feat(agent): new stores use the int8 vector index by default

`MemoryConfig::quantized_index` now defaults to `true`. It holds a
quarter of the index memory and, with the exact re-score, is faster at
equal recall on every configuration measured: 1.63x the queries per
second on x86-64 (AVX2) and 1.18x on a Raspberry Pi 5 (NEON SDOT), with
builds 1.8x and 2.3x faster. The one argument for keeping it off — that
int8 search was slower on ARM — did not survive being measured.

Existing stores do not change. A store written with v2.6.0 or later
keeps its persisted setting. One written before the setting existed has
no stored value, and it loads as `false` rather than as the new default,
so reopening it never changes how its index is held. That case is
guarded by a real store written with the v2.5.0 CLI, committed as
`tests/fixtures/store_v2_5_0.h5` (6.8 KB): the test asserts it reopens
with an f32 index and still searches, and it fails if the load default
is changed to `true`.

The CLI needed more than a new default. `create --quantized-index`
assigned its value straight into the config, so under the new default
every CLI-created store would have been forced back to f32 unless the
caller knew to ask for int8. It is replaced by `--f32-index`, which only
ever switches the default off; `--quantized-index` is still accepted,
hidden, as a no-op, and the two conflict.

The whole agent suite passes under the new default, including the
brute-force recall oracle, now running on int8 plus re-score without
being asked to.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-21 18:27:59 -07:00
co-authored by Claude Opus 5
parent 6598a7d02f
commit 8b85d9364b
9 changed files with 124 additions and 16 deletions
+5 -2
View File
@@ -39,8 +39,11 @@ Cargo workspace with 16 crates under `crates/` (plus `libaec-sys`, an internal F
(plain closest-M capped recall on clustered data: 0.31 recall@10 at 100K). Its
graph is saved to `<store>.h5.ann` at each checkpoint and reloaded by `open()`
(tied to the checkpoint by a generation id; stale/damaged sidecars are
ignored and the index rebuilt). `MemoryConfig::quantized_index` (off by
default, persisted) stores the index's own copy of the embeddings as `i8`,
ignored and the index rebuilt). `MemoryConfig::quantized_index` (**on by
default** for new stores, persisted; stores predating the setting load as
`false` and keep their f32 index — guarded by
`tests/fixtures/store_v2_5_0.h5`; CLI opt-out is `create --f32-index`)
stores the index's own copy of the embeddings as `i8`,
which roughly halves a loaded store's memory (2.72x -> 1.74x the raw vectors
at 100K); because quantised distances are approximate and `ef` cannot
compensate, the query path then re-scores the candidate pool against the