feat(agent): build the vector index in parallel by default

`parallel` joins the agent's default features, so the HNSW bulk build uses the
thread pool: cold index build at 10K records 1152 -> ~380 ms in a same-moment
A/B (the graph is identical either way). Nothing else on the measured paths
changes — ingest, checkpoint, open and steady-state query times are the same
with the feature on or off. Adds rayon to the default dependency set; opt out
with `--no-default-features --features float16,hnsw`.

Harness: `--e2e-only` runs the end-to-end section without the index
benchmarks. Note for anyone comparing numbers: this machine's absolute timings
drifted ~1.5x over a long session, so only same-moment A/B runs are
comparable.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
osobh
2026-09-19 13:52:23 -07:00
co-authored by Claude Fable 5.1
parent 8803d0754b
commit f507803ec1
4 changed files with 12 additions and 4 deletions
+2
View File
@@ -33,6 +33,8 @@ Cargo workspace with 16 crates under `crates/` (plus `libaec-sys`, an internal F
the approximate `clawhdf5-ann` index for the vector stage (the index mirrors
the cache and self-heals on drift). Build the agent with
`--no-default-features --features float16` to force the exact linear cosine scan.
The agent's `parallel` feature (also default) builds the index on a thread
pool; the graph is identical with or without it.
The index uses the HNSW paper's diversity heuristic for neighbour selection
(plain closest-M capped recall on clustered data: 0.31 recall@10 at 100K). Its
graph is saved to `<store>.h5.ann` at each checkpoint and reloaded by `open()`