docs: the int8 index is faster than f32 on AVX2, not slower

Measured with `clawhdf5_accel::dot_i8` in place, medians of three
alternating runs at N = 100 000 x 384, same binary:

  build      f32 3197 ms   int8 1778 ms   int8+re-score 1826 ms
  ef = 64    f32 13 399 QPS @ 0.9945   int8+re-score 21 848 QPS @ 0.9940

So at equal recall the quantised index is 1.63x the queries per second
and 1.8x the build speed, holding a quarter of the vectors. The earlier
"~13% of QPS" figure compared a scalar int8 loop against hand-written
AVX2 f32 kernels and was measuring the missing kernel; it is kept in
BENCHMARKS.md with that explanation rather than quietly replaced.

Still off by default, now for portability rather than performance: the
kernel is AVX2-only and aarch64 falls back to scalar, where the original
trade applies. A NEON kernel would settle it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-20 17:30:56 -07:00
co-authored by Claude Opus 5
parent 97e65f2adf
commit dea02f5214
4 changed files with 48 additions and 9 deletions
+14
View File
@@ -2,6 +2,20 @@
## Unreleased
### Performance
- `clawhdf5-accel`: **`dot_i8`, a runtime-dispatched int8 dot product** (AVX2:
sign-extend each half to `i16`, then `madd_epi16`; scalar fallback
elsewhere). The quantised HNSW index used a scalar loop while the `f32` path
it was measured against ran AVX2, so the ~13% throughput cost recorded for
`MemoryConfig::quantized_index` was a missing kernel rather than a property
of int8. With the kernel, at N = 100 000 x 384 and equal recall, the
quantised index answers **1.63x as many queries per second** (21 848 vs
13 399 at ef=64, recall 0.9940 vs 0.9945) and builds **1.8x faster** (1778
vs 3197 ms) — on top of holding a quarter of the vectors. Medians of three
alternating runs. It remains off by default only because the kernel is
AVX2-only and aarch64 falls back to the scalar loop. Integer arithmetic, so
the SIMD path is tested to agree with scalar bit for bit.
### Correctness
- `clawhdf5-format`: **datasets indexed by an Extensible Array returned wrong
data beyond their first few dozen chunks.** One unlimited dimension gives a