docs: the int8 index is faster than f32 on AVX2, not slower
Measured with `clawhdf5_accel::dot_i8` in place, medians of three alternating runs at N = 100 000 x 384, same binary: build f32 3197 ms int8 1778 ms int8+re-score 1826 ms ef = 64 f32 13 399 QPS @ 0.9945 int8+re-score 21 848 QPS @ 0.9940 So at equal recall the quantised index is 1.63x the queries per second and 1.8x the build speed, holding a quarter of the vectors. The earlier "~13% of QPS" figure compared a scalar int8 loop against hand-written AVX2 f32 kernels and was measuring the missing kernel; it is kept in BENCHMARKS.md with that explanation rather than quietly replaced. Still off by default, now for portability rather than performance: the kernel is AVX2-only and aarch64 falls back to scalar, where the original trade applies. A NEON kernel would settle it. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
@@ -2,6 +2,20 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Performance
|
||||
- `clawhdf5-accel`: **`dot_i8`, a runtime-dispatched int8 dot product** (AVX2:
|
||||
sign-extend each half to `i16`, then `madd_epi16`; scalar fallback
|
||||
elsewhere). The quantised HNSW index used a scalar loop while the `f32` path
|
||||
it was measured against ran AVX2, so the ~13% throughput cost recorded for
|
||||
`MemoryConfig::quantized_index` was a missing kernel rather than a property
|
||||
of int8. With the kernel, at N = 100 000 x 384 and equal recall, the
|
||||
quantised index answers **1.63x as many queries per second** (21 848 vs
|
||||
13 399 at ef=64, recall 0.9940 vs 0.9945) and builds **1.8x faster** (1778
|
||||
vs 3197 ms) — on top of holding a quarter of the vectors. Medians of three
|
||||
alternating runs. It remains off by default only because the kernel is
|
||||
AVX2-only and aarch64 falls back to the scalar loop. Integer arithmetic, so
|
||||
the SIMD path is tested to agree with scalar bit for bit.
|
||||
|
||||
### Correctness
|
||||
- `clawhdf5-format`: **datasets indexed by an Extensible Array returned wrong
|
||||
data beyond their first few dozen chunks.** One unlimited dimension gives a
|
||||
|
||||
Reference in New Issue
Block a user