quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs: measured ARM numbers, and a correction

Browse Source 
On a Raspberry Pi 5 at N = 100 000 and equal recall (0.9940 vs 0.9945),
medians of three runs:

  f32             33 413 ms build   6 164 QPS
  int8 scalar     18 950 ms         ~6 190 QPS   (what v2.7.0 shipped)
  int8 NEON      ~17 000 ms          6 640 QPS
  int8 SDOT       14 464 ms          7 267 QPS   1.18x f32, 2.3x build

The docs said quantised search stayed off by default because aarch64
"falls back to the scalar loop, where the original trade still
applies" — that it was ~13% slower than f32 there, as on x86. That was
extrapolated rather than measured, and it was wrong: x86's portable
baseline is SSE2 against hand-written AVX2 f32 kernels, but on aarch64
NEON is the baseline and the scalar loop vectorises well, so it already
matched f32. Corrected in BENCHMARKS.md, README.md and CLAUDE.md; the
released v2.7.0 changelog entry is left as it was and the correction is
recorded in a new one.

Labelled as Pi 5 figures throughout — a Pi's memory bandwidth and cache
are far below an M-series or flagship phone, so the ratios will move.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-21 17:34:51 -07:00
co-authored by Claude Opus 5
parent 56a8c2f3d0
commit 114a2dfcba
4 changed files with 65 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+6 -4
View File
@@ -44,10 +44,12 @@ Cargo workspace with 16 crates under `crates/` (plus `libaec-sys`, an internal F
which roughly halves a loaded store's memory (2.72x -> 1.74x the raw vectors
at 100K); because quantised distances are approximate and `ef` cannot
compensate, the query path then re-scores the candidate pool against the
exact embeddings, which holds recall at the f32 index's level. On AVX2 it is
also 1.63x the QPS and 1.8x the build speed (`clawhdf5_accel::dot_i8`); it
stays off by default only because that kernel is AVX2-only and aarch64 falls
back to scalar. `hybrid_search` keeps one incremental BM25
exact embeddings, which holds recall at the f32 index's level. It is also
faster at equal recall: 1.63x the QPS on x86-64 (AVX2) and 1.18x on a
Raspberry Pi 5 (`clawhdf5_accel::dot_i8`, NEON `SDOT` via inline asm since
the intrinsic is unstable; plain NEON on pre-dotprod cores). The aarch64
code is `cfg`'d out on x86, so x86 CI never compiles or lints it — test it
on real ARM (`rpivision02`, 10.0.2.3, is a Pi 5). `hybrid_search` keeps one incremental BM25
index for the life of the store and never writes the store: Hebbian
activation boosts are persisted by the next checkpoint (or on drop), not per
query. Measure any search-path change with