feat(agent): new stores use the int8 vector index by default
`MemoryConfig::quantized_index` now defaults to `true`. It holds a quarter of the index memory and, with the exact re-score, is faster at equal recall on every configuration measured: 1.63x the queries per second on x86-64 (AVX2) and 1.18x on a Raspberry Pi 5 (NEON SDOT), with builds 1.8x and 2.3x faster. The one argument for keeping it off — that int8 search was slower on ARM — did not survive being measured. Existing stores do not change. A store written with v2.6.0 or later keeps its persisted setting. One written before the setting existed has no stored value, and it loads as `false` rather than as the new default, so reopening it never changes how its index is held. That case is guarded by a real store written with the v2.5.0 CLI, committed as `tests/fixtures/store_v2_5_0.h5` (6.8 KB): the test asserts it reopens with an f32 index and still searches, and it fails if the load default is changed to `true`. The CLI needed more than a new default. `create --quantized-index` assigned its value straight into the config, so under the new default every CLI-created store would have been forced back to f32 unless the caller knew to ask for int8. It is replaced by `--f32-index`, which only ever switches the default off; `--quantized-index` is still accepted, hidden, as a no-op, and the two conflict. The whole agent suite passes under the new default, including the brute-force recall oracle, now running on int8 plus re-score without being asked to. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
+5
-4
@@ -364,10 +364,11 @@ cargo install --path crates/clawhdf5-cli
|
||||
clawhdf5 --path agent.h5 create --agent-id my-agent --dim 384 --wal
|
||||
```
|
||||
|
||||
Add `--quantized-index` to store the vector index's copy of the embeddings as
|
||||
int8. That roughly halves a loaded store's memory at about 13% fewer queries
|
||||
per second, with recall unchanged — the query path re-scores candidates
|
||||
against the exact embeddings. The setting is recorded in the file.
|
||||
New stores hold the vector index's copy of the embeddings as int8, which
|
||||
roughly halves a loaded store's memory and is faster at equal recall — the
|
||||
query path re-scores candidates against the exact embeddings. Pass
|
||||
`--f32-index` to keep an f32 index instead. The setting is recorded in the
|
||||
file, and stores created before it existed keep their f32 index.
|
||||
|
||||
Output:
|
||||
```json
|
||||
|
||||
Reference in New Issue
Block a user