fix(ann): deletions near the query no longer shrink search results

search() collected ef candidates, then filtered out soft-deleted nodes, then took k. When the records nearest a query had been deleted, every candidate was a tombstone and the search returned fewer than k results — 39 of 40 queries in the new test, which deletes each query's 40 nearest neighbours. search_layer takes an optional skip mask: a skipped node is still pushed onto the candidate queue (a tombstone is a valid waypoint) but never into the result heap, so the ef result slots hold live nodes only. Build and insert pass no mask. Recall and speed without deletions are unchanged. Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-19 13:37:29 -07:00
parent db4a067fe8
commit 41db450c92
2 changed files with 77 additions and 9 deletions
@@ -1,5 +1,15 @@
 # Changelog

+## Unreleased
+
+### Search
+- `clawhdf5-ann`: `HnswIndex::search` returned fewer than `k` results — often
+  none — when the records nearest the query had been deleted: it collected `ef`
+  candidates, *then* dropped the deleted ones, *then* took `k`. Deleted nodes
+  are now traversed as waypoints but never occupy a result slot, so a search
+  returns the `k` nearest live records. Matters for any store that deletes or
+  supersedes memories without compacting straight away.
+
 ## v2.4.0 (2026-09-19)

 ### Upgrade Notes