osobhandClaude Opus 5
1d767e3b93
`read_from_disk` memory-mapped the file and then copied the entire mapping into a `Vec` to hand to `File::from_bytes` — but `File::open` memory-maps it itself whenever the facade's `mmap` feature is on, which it is by default. So every open mapped the file, memcpy'd all of it, and parsed the copy. Store open at 100k x 384: 455 ms -> 327 ms, about 28% faster (two runs after the change, 326.8 and 328.1 ms). Peak memory is unchanged, which is worth saying because the opposite is the natural assumption. The footprint harness now tracks a high-water mark next to the retained figure, and it shows the peak falling after the parse, during the index build — so a buffer allocated and freed inside the parse never reaches it. Confirmed rather than assumed: holding a deliberate extra copy of the whole file across the parse leaves the peak exactly where it was, which is also what proved the instrument was working before trusting its answer. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
clawhdf5-agent
HDF5-backed persistent memory store for on-device AI agents.
Built on clawhdf5, clawhdf5-agent provides a vector-searchable memory backend optimized for edge AI workloads. Store embeddings, text chunks, and metadata in a single HDF5 file with SIMD-accelerated similarity search.
Features
- Persistent vector store in HDF5 format
- Cosine similarity and L2 distance search
- SIMD-accelerated via clawhdf5-accel (AVX2, NEON)
- Optional GPU acceleration via clawhdf5-gpu
- Memory-mapped access for large stores
- f16 storage support for compact embeddings
Usage
[dependencies]
clawhdf5-agent = "2.1.0"
License
MIT