feat(agent): new stores use the int8 vector index by default

`MemoryConfig::quantized_index` now defaults to `true`. It holds a
quarter of the index memory and, with the exact re-score, is faster at
equal recall on every configuration measured: 1.63x the queries per
second on x86-64 (AVX2) and 1.18x on a Raspberry Pi 5 (NEON SDOT), with
builds 1.8x and 2.3x faster. The one argument for keeping it off — that
int8 search was slower on ARM — did not survive being measured.

Existing stores do not change. A store written with v2.6.0 or later
keeps its persisted setting. One written before the setting existed has
no stored value, and it loads as `false` rather than as the new default,
so reopening it never changes how its index is held. That case is
guarded by a real store written with the v2.5.0 CLI, committed as
`tests/fixtures/store_v2_5_0.h5` (6.8 KB): the test asserts it reopens
with an f32 index and still searches, and it fails if the load default
is changed to `true`.

The CLI needed more than a new default. `create --quantized-index`
assigned its value straight into the config, so under the new default
every CLI-created store would have been forced back to f32 unless the
caller knew to ask for int8. It is replaced by `--f32-index`, which only
ever switches the default off; `--quantized-index` is still accepted,
hidden, as a no-op, and the two conflict.

The whole agent suite passes under the new default, including the
brute-force recall oracle, now running on int8 plus re-score without
being asked to.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-21 18:27:59 -07:00
co-authored by Claude Opus 5
parent 6598a7d02f
commit 8b85d9364b
9 changed files with 124 additions and 16 deletions
+23
View File
@@ -2,6 +2,29 @@
## Unreleased
### Upgrade Notes
- **New stores use the int8 vector index by default.**
`MemoryConfig::quantized_index` now defaults to `true`: a quarter of the
index memory, builds 1.8x (x86-64) and 2.3x (Raspberry Pi 5) faster, and
searches 1.63x and 1.18x faster at equal recall, measured on every
configuration tested. **Existing stores are unaffected** — a store written
with v2.6.0 or later keeps its persisted setting, and one written before the
setting existed opens as `false` and keeps its f32 index. Set
`quantized_index = false`, or pass `create --f32-index` to the CLI, to opt
out. The CLI's `--quantized-index` is still accepted but is now a no-op.
### Defaults
- `clawhdf5-agent`: `MemoryConfig::quantized_index` defaults to `true` for new
stores. The reason it had been off — that int8 search was slower on ARM —
did not survive measurement (see Corrections). Stores that predate the
setting still load it as `false`, so reopening one never changes how its
index is held; a store written by the v2.5.0 CLI is now a test fixture that
guards exactly that, and the test fails if the load default is changed.
- `clawhdf5-cli`: `create --f32-index` opts out. `create` used to assign
`--quantized-index` straight into the config, which under the new default
would have forced every CLI-created store back to f32 unless the caller
knew to ask; it now only ever switches the default off.
### Performance
- `clawhdf5-accel`: **`dot_i8` has aarch64 kernels** — `SDOT` for CPUs with
the ARMv8.2 dot-product extension (Cortex-A76 and later, Neoverse-N1, every