perf(agent): maintain a persistent flat embedding buffer for BLAS/Accelerate search

blas_cosine_batch and accelerate_cosine_batch_vecs re-flattened the
entire Vec<Vec<f32>> corpus into a fresh Vec<f32> on every single
query before running the batch matmul — an O(N·dim) copy paid per
query when fast-math/accelerate/openblas is enabled, even though a
flat fast-path (blas_cosine_batch_flat / accelerate_cosine_batch)
already existed for pre-flattened input.

Add MemoryCache::embeddings_flat, a contiguous [N × embedding_dim]
buffer maintained incrementally in push/update/compact (O(1) amortized
append, O(dim) in-place overwrite, O(n) rebuild only on compact/bulk
load). schema.rs's direct-push load path calls the new rebuild_flat()
explicitly. flat_embeddings() now just clones the already-maintained
buffer instead of rebuilding it.

Thread the flat buffer through strategy::search_with_metrics as a new
vectors_flat parameter, used only by the Blas/Accelerate arms (now
calling the *_flat variants); other strategies are unaffected. No
current caller wires search_with_metrics into the production query
path yet (only its own tests exercise it) — this fixes the identified
per-query re-flatten and makes the flat buffer available for whenever
that wiring lands.

INT-16
This commit is contained in:
ClawHDF5 Coding Agent
2026-08-17 00:41:23 +00:00
parent 1efd82c841
commit 45a38ba260
3 changed files with 185 additions and 8 deletions
+1
View File
@@ -427,6 +427,7 @@ fn load_memory_group(
cache.tombstones = tombstones;
cache.norms = norms;
cache.activation_weights = activation_weights;
cache.rebuild_flat();
Ok(cache)
}