osobhandClaude Fable 5.1
65d219c409
Neighbours were selected as the plain closest-M, for both a new node's links and back-link pruning. On clustered data every link of a node inside a tight cluster then goes to that same cluster, so clusters become islands a search entering elsewhere can never reach: recall@10 was 0.87 / 0.67 / 0.31 at 1K / 10K / 100K (384-dim) and flat in ef. Uniform random data — all the existing tests used — does not show it. Implement the HNSW paper's Algorithm 4 with keepPrunedConnections: accept a candidate only if it is closer to the node than to every neighbour already accepted, then fill spare slots with the closest rejected ones. Recall@10 at ef=64 is now 1.00 / 1.00 / 0.98 and rises with ef; uniform data improves slightly. Build is ~3.5x slower at 10K (extra distance evaluations), to be recovered by the distance-kernel work. The needless rayon fan-out over <=33 distances in prune_connections is gone. Tests: a clustered-data recall test for bulk build and incremental insert (scores 0.43 with the old selection), and a unit test of the selection rule. Harness gains --uniform and --ann-only; before/after in BENCHMARKS.md. Co-Authored-By: Claude Fable 5.1 <[email protected]>
clawhdf5-ann
HNSW approximate nearest neighbor index stored as HDF5.
Features
- Build and query HNSW indexes persisted in HDF5 format
- Pure Rust, no C dependencies
- Efficient similarity search for high-dimensional vectors
Usage
use clawhdf5_ann::HnswIndex;
let index = HnswIndex::from_hdf5("vectors.h5").unwrap();
let neighbors = index.search(&query, 10);
License
MIT