docs: the int8 index is faster than f32 on AVX2, not slower

Measured with `clawhdf5_accel::dot_i8` in place, medians of three
alternating runs at N = 100 000 x 384, same binary:

  build      f32 3197 ms   int8 1778 ms   int8+re-score 1826 ms
  ef = 64    f32 13 399 QPS @ 0.9945   int8+re-score 21 848 QPS @ 0.9940

So at equal recall the quantised index is 1.63x the queries per second
and 1.8x the build speed, holding a quarter of the vectors. The earlier
"~13% of QPS" figure compared a scalar int8 loop against hand-written
AVX2 f32 kernels and was measuring the missing kernel; it is kept in
BENCHMARKS.md with that explanation rather than quietly replaced.

Still off by default, now for portability rather than performance: the
kernel is AVX2-only and aarch64 falls back to scalar, where the original
trade applies. A NEON kernel would settle it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

BENCHMARKS.md

@@ -88,11 +88,31 @@ back**, because the loss is in the distances rather than in the graph
 
 Re-scoring closes the gap: the store already holds the exact embeddings, so
 the query path re-scores the candidate pool against them before fusion. That
-is done automatically whenever the index is quantised. What it costs is
+is done automatically whenever the index is quantised.
-throughput — about 13% of QPS and 16% of build time at 100 000 x 384. So the
+
-setting trades ~13% of query speed for ~36% of the process's memory at equal
+**On AVX2 this costs nothing — it pays.** The first measurement of this put
-recall. It is **off by default**: the right side of that trade depends on
+the cost at ~13% of QPS and ~16% of build time, but that compared a scalar
-whether the deployment is short of memory or short of CPU.
+int8 loop against `clawhdf5-accel`'s hand-written AVX2 kernels for `f32`:
+the gap was a missing kernel, not a property of int8. With
+`clawhdf5_accel::dot_i8` (AVX2: sign-extend to `i16`, then `madd_epi16`),
+medians of three alternating runs at N = 100 000, same binary:
+
+| | f32 | int8 | int8 + re-score |
+|---|---:|---:|---:|
+| build | 3197 ms | **1778 ms** | 1826 ms |
+| QPS at ef = 64 | 13 399 | 29 195 | **21 848** |
+| recall@10 at ef = 64 | 0.9945 | 0.9625 | **0.9940** |
+
+So at equal recall the quantised index answers **1.63x as many queries per
+second**, builds **1.8x faster**, and holds a quarter of the vectors. (Compare
+only at equal `ef`: with re-scoring the harness raises `ef` to at least the
+candidate pool, so the `ef = 16` and `ef = 32` rows are not like-for-like.)
+
+It is still **off by default**, for portability rather than performance: the
+int8 kernel is AVX2-only, and on aarch64 — including `clawhdf5-android` — it
+falls back to the scalar loop, where the original trade still applies. A NEON
+kernel would remove that caveat. On an x86-64 deployment, turning it on is a
+win on every axis measured.
 
 A measurement trap worth recording: the synthetic `clustered` generator in the
 `clawhdf5-ann` tests draws clusters far tighter than any real embedding, so

CHANGELOG.md

@@ -2,6 +2,20 @@
 
 ## Unreleased
 
+### Performance
+- `clawhdf5-accel`: **`dot_i8`, a runtime-dispatched int8 dot product** (AVX2:
+  sign-extend each half to `i16`, then `madd_epi16`; scalar fallback
+  elsewhere). The quantised HNSW index used a scalar loop while the `f32` path
+  it was measured against ran AVX2, so the ~13% throughput cost recorded for
+  `MemoryConfig::quantized_index` was a missing kernel rather than a property
+  of int8. With the kernel, at N = 100 000 x 384 and equal recall, the
+  quantised index answers **1.63x as many queries per second** (21 848 vs
+  13 399 at ef=64, recall 0.9940 vs 0.9945) and builds **1.8x faster** (1778
+  vs 3197 ms) — on top of holding a quarter of the vectors. Medians of three
+  alternating runs. It remains off by default only because the kernel is
+  AVX2-only and aarch64 falls back to the scalar loop. Integer arithmetic, so
+  the SIMD path is tested to agree with scalar bit for bit.
+
 ### Correctness
 - `clawhdf5-format`: **datasets indexed by an Extensible Array returned wrong
   data beyond their first few dozen chunks.** One unlimited dimension gives a

CLAUDE.md

@@ -44,8 +44,10 @@ Cargo workspace with 16 crates under `crates/` (plus `libaec-sys`, an internal F
   which roughly halves a loaded store's memory (2.72x -> 1.74x the raw vectors
   at 100K); because quantised distances are approximate and `ef` cannot
   compensate, the query path then re-scores the candidate pool against the
-  exact embeddings, which holds recall at the f32 index's level and costs
+  exact embeddings, which holds recall at the f32 index's level. On AVX2 it is
-  ~13% of QPS. `hybrid_search` keeps one incremental BM25
+  also 1.63x the QPS and 1.8x the build speed (`clawhdf5_accel::dot_i8`); it
+  stays off by default only because that kernel is AVX2-only and aarch64 falls
+  back to scalar. `hybrid_search` keeps one incremental BM25
   index for the life of the store and never writes the store: Hebbian
   activation boosts are persisted by the next checkpoint (or on drop), not per
   query. Measure any search-path change with

README.md

@@ -437,8 +437,11 @@ ClawhDF5's agent memory design draws from 15+ recent papers:
 copy of the embeddings as `i8`, roughly halving a loaded store's memory
 (2.72x -> 1.74x the raw vectors at 100k x 384). Quantised distances are
 approximate, so the query path re-scores the candidate pool against the exact
-embeddings the store already holds — recall matches the `f32` index, at about
+embeddings the store already holds, which keeps recall at the `f32` index's
-13% fewer queries per second. See `BENCHMARKS.md`, "Quantising the index copy".
+level. On AVX2 it is also **faster** — 1.63x the queries per second and 1.8x
+the build speed at equal recall — because the int8 kernel is SIMD too. It
+stays off by default only because that kernel is AVX2-only and aarch64 falls
+back to a scalar loop. See `BENCHMARKS.md`, "Quantising the index copy".
 | `parallel` | no | Rayon parallel search |
 | `fast-math` | no | BLAS matrix-vector multiply |
 | `accelerate` | no | Apple Accelerate / AMX (macOS) |