feat(agent): new stores use the int8 vector index by default
`MemoryConfig::quantized_index` now defaults to `true`. It holds a quarter of the index memory and, with the exact re-score, is faster at equal recall on every configuration measured: 1.63x the queries per second on x86-64 (AVX2) and 1.18x on a Raspberry Pi 5 (NEON SDOT), with builds 1.8x and 2.3x faster. The one argument for keeping it off — that int8 search was slower on ARM — did not survive being measured. Existing stores do not change. A store written with v2.6.0 or later keeps its persisted setting. One written before the setting existed has no stored value, and it loads as `false` rather than as the new default, so reopening it never changes how its index is held. That case is guarded by a real store written with the v2.5.0 CLI, committed as `tests/fixtures/store_v2_5_0.h5` (6.8 KB): the test asserts it reopens with an f32 index and still searches, and it fails if the load default is changed to `true`. The CLI needed more than a new default. `create --quantized-index` assigned its value straight into the config, so under the new default every CLI-created store would have been forced back to f32 unless the caller knew to ask for int8. It is replaced by `--f32-index`, which only ever switches the default off; `--quantized-index` is still accepted, hidden, as a no-op, and the two conflict. The whole agent suite passes under the new default, including the brute-force recall oracle, now running on int8 plus re-score without being asked to. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
@@ -2,6 +2,29 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Upgrade Notes
|
||||
- **New stores use the int8 vector index by default.**
|
||||
`MemoryConfig::quantized_index` now defaults to `true`: a quarter of the
|
||||
index memory, builds 1.8x (x86-64) and 2.3x (Raspberry Pi 5) faster, and
|
||||
searches 1.63x and 1.18x faster at equal recall, measured on every
|
||||
configuration tested. **Existing stores are unaffected** — a store written
|
||||
with v2.6.0 or later keeps its persisted setting, and one written before the
|
||||
setting existed opens as `false` and keeps its f32 index. Set
|
||||
`quantized_index = false`, or pass `create --f32-index` to the CLI, to opt
|
||||
out. The CLI's `--quantized-index` is still accepted but is now a no-op.
|
||||
|
||||
### Defaults
|
||||
- `clawhdf5-agent`: `MemoryConfig::quantized_index` defaults to `true` for new
|
||||
stores. The reason it had been off — that int8 search was slower on ARM —
|
||||
did not survive measurement (see Corrections). Stores that predate the
|
||||
setting still load it as `false`, so reopening one never changes how its
|
||||
index is held; a store written by the v2.5.0 CLI is now a test fixture that
|
||||
guards exactly that, and the test fails if the load default is changed.
|
||||
- `clawhdf5-cli`: `create --f32-index` opts out. `create` used to assign
|
||||
`--quantized-index` straight into the config, which under the new default
|
||||
would have forced every CLI-created store back to f32 unless the caller
|
||||
knew to ask; it now only ever switches the default off.
|
||||
|
||||
### Performance
|
||||
- `clawhdf5-accel`: **`dot_i8` has aarch64 kernels** — `SDOT` for CPUs with
|
||||
the ARMv8.2 dot-product extension (Cortex-A76 and later, Neoverse-N1, every
|
||||
|
||||
Reference in New Issue
Block a user