bench: search harness — HNSW recall vs speed, end-to-end hybrid_search latency
New clawhdf5-bench binary `search_harness`, the measurement baseline for the search hot-path work. On deterministic clustered 384-dim data it reports HNSW build time and, per ef, recall@10 against an exact scan, QPS and p50/p99; and for HDF5Memory: ingest, checkpoint, open, first-query-after-open and steady-state hybrid_search latency at 1K/10K (and 100K with --full). Optional JSON output for tracking. Baseline recorded in BENCHMARKS.md. It shows two problems: HNSW recall@10 does not respond to ef and falls from 0.87 (1K) to 0.31 (100K) on clustered data, and end-to-end hybrid_search is ~1000x slower than its vector stage because every query rebuilds BM25 and rewrites the .h5 file. Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
a3ad548f84
commit
eb99de1020
@@ -13,6 +13,10 @@ path = "src/bin/longmemeval_bench.rs"
|
||||
name = "memory_arena"
|
||||
path = "src/bin/memory_arena.rs"
|
||||
|
||||
[[bin]]
|
||||
name = "search_harness"
|
||||
path = "src/bin/search_harness.rs"
|
||||
|
||||
[[bin]]
|
||||
name = "footprint_bench"
|
||||
path = "src/bin/footprint_bench.rs"
|
||||
@@ -48,6 +52,7 @@ harness = false
|
||||
|
||||
[dependencies]
|
||||
clawhdf5-agent = { path = "../clawhdf5-agent" }
|
||||
clawhdf5-ann = { path = "../clawhdf5-ann" }
|
||||
clawhdf5-io = { path = "../clawhdf5-io" }
|
||||
mpi = { version = "0.8", optional = true }
|
||||
serde = { workspace = true }
|
||||
|
||||
Reference in New Issue
Block a user