bench: search harness — HNSW recall vs speed, end-to-end hybrid_search latency

New clawhdf5-bench binary `search_harness`, the measurement baseline for the
search hot-path work. On deterministic clustered 384-dim data it reports HNSW
build time and, per ef, recall@10 against an exact scan, QPS and p50/p99; and
for HDF5Memory: ingest, checkpoint, open, first-query-after-open and
steady-state hybrid_search latency at 1K/10K (and 100K with --full). Optional
JSON output for tracking.

Baseline recorded in BENCHMARKS.md. It shows two problems: HNSW recall@10 does
not respond to ef and falls from 0.87 (1K) to 0.31 (100K) on clustered data,
and end-to-end hybrid_search is ~1000x slower than its vector stage because
every query rebuilds BM25 and rewrites the .h5 file.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

BENCHMARKS.md

@@ -28,6 +28,72 @@
 
 ---
 
+## Search harness baseline (v2.3.0)
+
+Produced by `cargo run --release -p clawhdf5-bench --bin search_harness -- --full`
+on deterministic **clustered** synthetic data (384-dim, unit-normalised; points =
+cluster centre + noise — uniform random vectors are nearly equidistant in high
+dimension and say nothing about embeddings). Recall is measured against an exact
+brute-force scan, 200 queries. This is the *before* picture for the search
+hot-path work; every change to that path should be justified by a re-run.
+
+Two things stand out:
+
+* **HNSW recall does not respond to `ef`** and degrades sharply with size
+  (0.87 → 0.67 → 0.31 recall@10 at 1K / 10K / 100K). Latency plateaus at the same
+  point, i.e. the search exhausts the nodes it can reach: on clustered data the
+  graph is poorly connected. The index selects neighbours by plain top-M
+  distance rather than the HNSW paper's diversity heuristic.
+* **End-to-end `hybrid_search` is ~1000x slower than its vector stage** (49 ms
+  vs ~0.03 ms at 10K; 884 ms at 100K). Each query rebuilds the BM25 index from
+  scratch and rewrites the whole `.h5` file. The first query after `open()`
+  additionally rebuilds the HNSW index (10.5 s at 100K).
+
+### HNSW, N = 1000, dim = 384, M = 16, ef_construction = 64
+
+build: 72.4 ms (13818 vectors/s) · exact scan: 3854 QPS, p50 258 µs
+
+| ef | recall@10 | QPS | p50 µs | p99 µs |
+|---:|---:|---:|---:|---:|
+| 16 | 0.8710 | 59484 | 16 | 31 |
+| 32 | 0.8730 | 46302 | 21 | 25 |
+| 64 | 0.8730 | 31683 | 31 | 44 |
+| 128 | 0.8730 | 24715 | 40 | 49 |
+| 256 | 0.8730 | 24788 | 40 | 50 |
+
+### HNSW, N = 10000, dim = 384, M = 16, ef_construction = 64
+
+build: 802.5 ms (12461 vectors/s) · exact scan: 418 QPS, p50 2363 µs
+
+| ef | recall@10 | QPS | p50 µs | p99 µs |
+|---:|---:|---:|---:|---:|
+| 16 | 0.6695 | 44031 | 19 | 51 |
+| 32 | 0.6705 | 45066 | 22 | 30 |
+| 64 | 0.6705 | 32746 | 30 | 41 |
+| 128 | 0.6705 | 27542 | 36 | 51 |
+| 256 | 0.6705 | 27754 | 36 | 49 |
+
+### HNSW, N = 100000, dim = 384, M = 16, ef_construction = 64
+
+build: 9752.6 ms (10254 vectors/s) · exact scan: 40 QPS, p50 24648 µs
+
+| ef | recall@10 | QPS | p50 µs | p99 µs |
+|---:|---:|---:|---:|---:|
+| 16 | 0.3085 | 18046 | 57 | 84 |
+| 32 | 0.3110 | 21621 | 43 | 75 |
+| 64 | 0.3130 | 20015 | 49 | 70 |
+| 128 | 0.3135 | 15822 | 63 | 99 |
+| 256 | 0.3135 | 15308 | 66 | 124 |
+
+### End to end: `HDF5Memory::hybrid_search` (k = 10, weights 0.7 / 0.3)
+
+| N | ingest ms | checkpoint ms | open ms | first query ms | p50 ms | p99 ms | QPS |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 1000 | 11 | 3.9 | 0.9 | 68.1 | 5.48 | 5.57 | 182.5 |
+| 10000 | 114 | 32.2 | 10.9 | 845.0 | 48.56 | 78.65 | 19.8 |
+| 100000 | 1486 | 713.0 | 354.5 | 10486.5 | 883.51 | 975.23 | 1.1 |
+wrote /tmp/claude-1000/-home-osobh-projects-clawhdf5/422f755e-dd25-4c35-8613-5439087e3aaa/scratchpad/baseline_full.json
+
 ## Vector Search Latency
 
 Brute-force cosine similarity over 384-dimensional embeddings (OpenAI text-embedding-3-small size).