fix(ann): HNSW neighbour-selection heuristic — recall 0.31 -> 0.98 at 100K

Neighbours were selected as the plain closest-M, for both a new node's links
and back-link pruning. On clustered data every link of a node inside a tight
cluster then goes to that same cluster, so clusters become islands a search
entering elsewhere can never reach: recall@10 was 0.87 / 0.67 / 0.31 at
1K / 10K / 100K (384-dim) and flat in ef. Uniform random data — all the
existing tests used — does not show it.

Implement the HNSW paper's Algorithm 4 with keepPrunedConnections: accept a
candidate only if it is closer to the node than to every neighbour already
accepted, then fill spare slots with the closest rejected ones. Recall@10 at
ef=64 is now 1.00 / 1.00 / 0.98 and rises with ef; uniform data improves
slightly. Build is ~3.5x slower at 10K (extra distance evaluations), to be
recovered by the distance-kernel work. The needless rayon fan-out over <=33
distances in prune_connections is gone.

Tests: a clustered-data recall test for bulk build and incremental insert
(scores 0.43 with the old selection), and a unit test of the selection rule.
Harness gains --uniform and --ann-only; before/after in BENCHMARKS.md.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

CHANGELOG.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## Unreleased
+
+### Search
+- `clawhdf5-ann`: **HNSW recall fix.** Neighbours were chosen as the plain
+  closest-M, which on clustered data (what embeddings look like) turns each
+  cluster into an island: recall@10 was 0.87 / 0.67 / 0.31 at 1K / 10K / 100K
+  vectors and did not improve with `ef`. The index now uses the HNSW paper's
+  diversity heuristic (Algorithm 4 with kept pruned connections) when linking a
+  new node and when pruning back-links: recall@10 at `ef = 64` is 1.00 / 1.00 /
+  0.98 and responds to `ef`. Builds are slower (~3.5x at 10K). Existing
+  persisted indexes keep their old graph until rebuilt; the agent rebuilds its
+  index from the cache, so stores pick this up automatically.
+- `clawhdf5-bench`: new `search_harness` binary — HNSW recall@10 / QPS / latency
+  per `ef` against an exact scan, and end-to-end `hybrid_search` timings, on
+  deterministic clustered (or `--uniform`) data. Baseline in `BENCHMARKS.md`.
+
 ## v2.3.0 (2026-09-19)
 
 ### Upgrade Notes