osobhandClaude Opus 5.5
00b0cb0035
consolidation_efficiency never finished: stopped after 19 minutes on one core while building its 100K case. Not the consolidation cycle (linear: 17 us at 100 records, 2.16 ms at 10K) but the setup — every add_memory scores the new record's novelty against the whole working tier, the benchmark lets that tier reach 50K, and each comparison recomputed both norms: ~5e9 comparisons of three passes each. ImportanceScorer::score_surprise now computes the new record's norm once, takes each comparison in one fused, 8-lane pass (dot product and the other norm together), and splits a working tier of 4096+ records across threads with the `parallel` feature. Same results: tested against the old cosine formula, including shorter, empty and zero vectors and the parallel path. The work stays quadratic in the working-tier size by design; with regular consolidation the tier stays near working_capacity (100) and inserts are cheap. The complete run takes 8 min 10 s on tank and fills in the 100K cycle row (46.66 ms) and the memory-reduction table, which had never been published. The binary no longer prints a record-count ratio as a "BM25 Speedup" (never measured; Part 1 measures search latency) or claims sub-linear cycle scaling (its own numbers grow slightly faster than linearly). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5-agent
HDF5-backed persistent memory store for on-device AI agents.
Built on clawhdf5, clawhdf5-agent provides a vector-searchable memory backend optimized for edge AI workloads. Store embeddings, text chunks, and metadata in a single HDF5 file with SIMD-accelerated similarity search.
Features
- Persistent vector store in HDF5 format
- Cosine similarity and L2 distance search
- SIMD-accelerated via clawhdf5-accel (AVX2, NEON)
- Optional GPU acceleration via clawhdf5-gpu
- Memory-mapped access for large stores
- f16 storage support for compact embeddings
Usage
[dependencies]
clawhdf5-agent = "2.1.0"
License
MIT