docs: the int8 index is faster than f32 on AVX2, not slower
Measured with `clawhdf5_accel::dot_i8` in place, medians of three alternating runs at N = 100 000 x 384, same binary: build f32 3197 ms int8 1778 ms int8+re-score 1826 ms ef = 64 f32 13 399 QPS @ 0.9945 int8+re-score 21 848 QPS @ 0.9940 So at equal recall the quantised index is 1.63x the queries per second and 1.8x the build speed, holding a quarter of the vectors. The earlier "~13% of QPS" figure compared a scalar int8 loop against hand-written AVX2 f32 kernels and was measuring the missing kernel; it is kept in BENCHMARKS.md with that explanation rather than quietly replaced. Still off by default, now for portability rather than performance: the kernel is AVX2-only and aarch64 falls back to scalar, where the original trade applies. A NEON kernel would settle it. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
@@ -44,8 +44,10 @@ Cargo workspace with 16 crates under `crates/` (plus `libaec-sys`, an internal F
|
||||
which roughly halves a loaded store's memory (2.72x -> 1.74x the raw vectors
|
||||
at 100K); because quantised distances are approximate and `ef` cannot
|
||||
compensate, the query path then re-scores the candidate pool against the
|
||||
exact embeddings, which holds recall at the f32 index's level and costs
|
||||
~13% of QPS. `hybrid_search` keeps one incremental BM25
|
||||
exact embeddings, which holds recall at the f32 index's level. On AVX2 it is
|
||||
also 1.63x the QPS and 1.8x the build speed (`clawhdf5_accel::dot_i8`); it
|
||||
stays off by default only because that kernel is AVX2-only and aarch64 falls
|
||||
back to scalar. `hybrid_search` keeps one incremental BM25
|
||||
index for the life of the store and never writes the store: Hebbian
|
||||
activation boosts are persisted by the next checkpoint (or on drop), not per
|
||||
query. Measure any search-path change with
|
||||
|
||||
Reference in New Issue
Block a user