perf(agent): maintain a persistent flat embedding buffer for BLAS/Accelerate search
blas_cosine_batch and accelerate_cosine_batch_vecs re-flattened the entire Vec<Vec<f32>> corpus into a fresh Vec<f32> on every single query before running the batch matmul — an O(N·dim) copy paid per query when fast-math/accelerate/openblas is enabled, even though a flat fast-path (blas_cosine_batch_flat / accelerate_cosine_batch) already existed for pre-flattened input. Add MemoryCache::embeddings_flat, a contiguous [N × embedding_dim] buffer maintained incrementally in push/update/compact (O(1) amortized append, O(dim) in-place overwrite, O(n) rebuild only on compact/bulk load). schema.rs's direct-push load path calls the new rebuild_flat() explicitly. flat_embeddings() now just clones the already-maintained buffer instead of rebuilding it. Thread the flat buffer through strategy::search_with_metrics as a new vectors_flat parameter, used only by the Blas/Accelerate arms (now calling the *_flat variants); other strategies are unaffected. No current caller wires search_with_metrics into the production query path yet (only its own tests exercise it) — this fixes the identified per-query re-flatten and makes the flat buffer available for whenever that wiring lands. INT-16
This commit is contained in:
@@ -427,6 +427,7 @@ fn load_memory_group(
|
||||
cache.tombstones = tombstones;
|
||||
cache.norms = norms;
|
||||
cache.activation_weights = activation_weights;
|
||||
cache.rebuild_flat();
|
||||
|
||||
Ok(cache)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user