perf(agent): store embeddings once, not twice
MemoryCache held every embedding in two places: a `Vec<Vec<f32>>` and a flattened copy for the batched kernels, kept in lock-step on every push, update and compaction. A store loaded from disk therefore carried the corpus twice, plus one heap allocation per entry. A new `cache::Embeddings` owns just the flat `[N x dim]` buffer and indexes into it, so `embeddings[i]` still reads as a `&[f32]` row. The batch kernels take a `VectorSet` (implemented for both `Embeddings` and `Vec<Vec<f32>>`) instead of `&[Vec<f32>]`, so their callers and tests are unchanged. Loading no longer unflattens what it just read. 100k 384-dim entries, reopened from disk: 505 -> 357 MiB, 3.44x -> 2.43x the raw vectors. Recall (1.0000 at ef=64) and query latency are unchanged. Rows are now always exactly `dim` long, shorter ones zero-padded. The old representation allowed ragged rows, which silently misaligned the flattened copy — every row after a wrong-length embedding — and `update` carried a comment about falling back to a rebuild to avoid exactly that. It is now unrepresentable. A record saved without an embedding holds a zero row and is told apart by its norm, which is what `total_embeddings` now counts. Measured with a counting allocator rather than RSS: freeing a structure returns its pages to the allocator's pool, not the OS, so an RSS reading from inside the process showed the two representations as identical. Breaking: MemoryCache::embeddings changes type, embeddings_flat is replaced by flat_embeddings(), rebuild_flat() is a deprecated no-op. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
@@ -466,7 +466,7 @@ impl ClawhdfBackend {
|
||||
let record = MemoryRecord {
|
||||
id: i as u64,
|
||||
chunk: cache.chunks[i].clone(),
|
||||
embedding: cache.embeddings[i].clone(),
|
||||
embedding: cache.embeddings[i].to_vec(),
|
||||
tier: MemoryTier::Working,
|
||||
importance: cache.activation_weights[i],
|
||||
access_count: 0,
|
||||
@@ -717,11 +717,13 @@ impl MemoryBackend for ClawhdfBackend {
|
||||
|
||||
let total_records = cache.count_active();
|
||||
|
||||
// A record saved without an embedding occupies a zero row, so "has an
|
||||
// embedding" is "has a non-zero norm" rather than "row is non-empty".
|
||||
let total_embeddings = cache
|
||||
.embeddings
|
||||
.norms
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(i, emb)| cache.tombstones[*i] == 0 && !emb.is_empty())
|
||||
.filter(|(i, norm)| cache.tombstones[*i] == 0 && **norm > 0.0)
|
||||
.count();
|
||||
|
||||
let file_size_bytes = std::fs::metadata(&self.hdf5_path)
|
||||
|
||||
Reference in New Issue
Block a user