feat(agent): new stores use the int8 vector index by default

`MemoryConfig::quantized_index` now defaults to `true`. It holds a
quarter of the index memory and, with the exact re-score, is faster at
equal recall on every configuration measured: 1.63x the queries per
second on x86-64 (AVX2) and 1.18x on a Raspberry Pi 5 (NEON SDOT), with
builds 1.8x and 2.3x faster. The one argument for keeping it off — that
int8 search was slower on ARM — did not survive being measured.

Existing stores do not change. A store written with v2.6.0 or later
keeps its persisted setting. One written before the setting existed has
no stored value, and it loads as `false` rather than as the new default,
so reopening it never changes how its index is held. That case is
guarded by a real store written with the v2.5.0 CLI, committed as
`tests/fixtures/store_v2_5_0.h5` (6.8 KB): the test asserts it reopens
with an f32 index and still searches, and it fails if the load default
is changed to `true`.

The CLI needed more than a new default. `create --quantized-index`
assigned its value straight into the config, so under the new default
every CLI-created store would have been forced back to f32 unless the
caller knew to ask for int8. It is replaced by `--f32-index`, which only
ever switches the default off; `--quantized-index` is still accepted,
hidden, as a no-op, and the two conflict.

The whole agent suite passes under the new default, including the
brute-force recall oracle, now running on int8 plus re-score without
being asked to.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-21 18:27:59 -07:00
co-authored by Claude Opus 5
parent 6598a7d02f
commit 8b85d9364b
9 changed files with 124 additions and 16 deletions
Binary file not shown.
@@ -284,3 +284,63 @@ fn degenerate_hnsw_parameters_do_not_panic() {
let results = mem.hybrid_search(&vectors[7], "", 1.0, 0.0, 5);
assert_eq!(results[0].index, 7, "exact match should still rank first");
}
#[test]
fn new_stores_default_to_the_quantized_index() {
// int8 is the default because it is smaller and, with an exact re-score,
// faster at equal recall on every platform measured (see BENCHMARKS.md).
let dir = TempDir::new().unwrap();
let config = MemoryConfig::new(dir.path().join("mem.h5"), "agent", 8);
assert!(config.quantized_index);
let path = config.path.clone();
let mut mem = HDF5Memory::create(config).unwrap();
let mut seed = 3;
let vectors: Vec<Vec<f32>> = (0..40).map(|_| make_vector(&mut seed, 8)).collect();
for (i, v) in vectors.iter().enumerate() {
mem.save(entry(&format!("c{i}"), v.clone(), "t")).unwrap();
}
assert_eq!(
mem.hybrid_search(&vectors[11], "", 1.0, 0.0, 1)[0].index,
11
);
mem.flush_wal().unwrap();
drop(mem);
assert!(HDF5Memory::open(&path).unwrap().config().quantized_index);
}
#[test]
fn a_store_written_before_the_setting_existed_stays_f32() {
// `store_v2_5_0.h5` was written by the v2.5.0 CLI, before
// `quantized_index` or the HNSW parameters were persisted, so it carries
// none of them. Flipping the default for new stores must not reach back
// and change how an existing store's index is held.
let dir = TempDir::new().unwrap();
let path = dir.path().join("legacy.h5");
std::fs::copy(
concat!(
env!("CARGO_MANIFEST_DIR"),
"/tests/fixtures/store_v2_5_0.h5"
),
&path,
)
.unwrap();
let bytes = std::fs::read(&path).unwrap();
assert!(
!bytes.windows(15).any(|w| w == b"quantized_index"),
"the fixture must predate the setting, or it tests nothing"
);
let mut mem = HDF5Memory::open(&path).unwrap();
assert!(
!mem.config().quantized_index,
"an old store must reopen with an f32 index"
);
assert_eq!(mem.config().hnsw_m, 16);
assert_eq!(mem.config().hnsw_ef_construction, 64);
assert_eq!(mem.count(), 6);
// And it still searches: entry 3's own embedding finds it first.
let hit = mem.hybrid_search(&[3.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "", 1.0, 0.0, 1);
assert_eq!(hit[0].index, 3);
}