quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs(bench): record search harness numbers after the HNSW heuristic

Browse Source 
Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
osobh
2026-09-19 07:43:52 -07:00
co-authored by Claude Fable 5.1
parent 65d219c409
commit 61424d1418
1 changed files with 45 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
BENCHMARKS.md
+45
View File
@@ -94,6 +94,51 @@ build: 9752.6 ms (10254 vectors/s) · exact scan: 40 QPS, p50 24648 µs
| 100000 | 1486 | 713.0 | 354.5 | 10486.5 | 883.51 | 975.23 | 1.1 | | 100000 | 1486 | 713.0 | 354.5 | 10486.5 | 883.51 | 975.23 | 1.1 |
wrote /tmp/claude-1000/-home-osobh-projects-clawhdf5/422f755e-dd25-4c35-8613-5439087e3aaa/scratchpad/baseline_full.json wrote /tmp/claude-1000/-home-osobh-projects-clawhdf5/422f755e-dd25-4c35-8613-5439087e3aaa/scratchpad/baseline_full.json
### After: HNSW neighbour-selection heuristic
Same harness, same data, after replacing closest-M neighbour selection with the
HNSW paper's diversity heuristic (Algorithm 4, keeping pruned connections) for
both new links and back-link pruning. Recall@10 at `ef = 64`: **0.87 → 1.00**
(1K), **0.67 → 1.00** (10K), **0.31 → 0.98** (100K), and it now rises with
`ef` as it should. The cost is a slower build (extra distance evaluations per
insert: ~3.5x at 10K); the distance-kernel work that follows targets that.
### HNSW, N = 1000, dim = 384, M = 16, ef_construction = 64
build: 221.3 ms (4519 vectors/s) · exact scan: 3851 QPS, p50 258 µs
| ef | recall@10 | QPS | p50 µs | p99 µs |
|---:|---:|---:|---:|---:|
| 16 | 0.9990 | 54760 | 18 | 29 |
| 32 | 1.0000 | 40422 | 24 | 44 |
| 64 | 1.0000 | 27744 | 36 | 51 |
| 128 | 1.0000 | 13164 | 74 | 106 |
| 256 | 1.0000 | 6879 | 144 | 175 |
### HNSW, N = 10000, dim = 384, M = 16, ef_construction = 64
build: 2733.5 ms (3658 vectors/s) · exact scan: 423 QPS, p50 2362 µs
| ef | recall@10 | QPS | p50 µs | p99 µs |
|---:|---:|---:|---:|---:|
| 16 | 0.9975 | 31321 | 27 | 61 |
| 32 | 1.0000 | 32427 | 29 | 48 |
| 64 | 1.0000 | 22738 | 42 | 62 |
| 128 | 1.0000 | 10055 | 99 | 129 |
| 256 | 1.0000 | 4649 | 214 | 266 |
### HNSW, N = 100000, dim = 384, M = 16, ef_construction = 64
build: 36472.8 ms (2742 vectors/s) · exact scan: 40 QPS, p50 24644 µs
| ef | recall@10 | QPS | p50 µs | p99 µs |
|---:|---:|---:|---:|---:|
| 16 | 0.9235 | 11394 | 82 | 194 |
| 32 | 0.9675 | 12788 | 73 | 161 |
| 64 | 0.9840 | 10406 | 91 | 186 |
| 128 | 0.9990 | 7633 | 126 | 248 |
| 256 | 0.9990 | 2823 | 352 | 510 |
## Vector Search Latency ## Vector Search Latency
Brute-force cosine similarity over 384-dimensional embeddings (OpenAI text-embedding-3-small size). Brute-force cosine similarity over 384-dimensional embeddings (OpenAI text-embedding-3-small size).