osobhandClaude Fable 5.1
943b9141e3
flush() writes the new .h5 and only then truncates the WAL. A crash in that window left a .h5 that already contained the pending entries AND a WAL that still listed them, and open() replayed the WAL unconditionally — every pending entry came back twice. A checkpoint now records a WalMark in /meta (wal_applied_len/wal_applied_crc): the byte length and chained CRC of the WAL prefix it folded in. On open, if the WAL's v3 CRC chain passes through exactly that position, the entries up to it are skipped; otherwise (the normal case: the WAL was truncated) everything is replayed. No WAL format change; files without the attributes behave as before. WalFile tracks its chain length alongside running_crc and resumes both on reopen. Also make the checkpoint and snapshot durable as a unit: sync the temp file before the rename and the parent directory after it, so a power loss can't leave an empty or partial .h5 under the final name. This is per-checkpoint cost only; individual WAL appends remain unsynced by design. Co-Authored-By: Claude Fable 5.1 <[email protected]>
clawhdf5-agent
HDF5-backed persistent memory store for on-device AI agents.
Built on clawhdf5, clawhdf5-agent provides a vector-searchable memory backend optimized for edge AI workloads. Store embeddings, text chunks, and metadata in a single HDF5 file with SIMD-accelerated similarity search.
Features
- Persistent vector store in HDF5 format
- Cosine similarity and L2 distance search
- SIMD-accelerated via clawhdf5-accel (AVX2, NEON)
- Optional GPU acceleration via clawhdf5-gpu
- Memory-mapped access for large stores
- f16 storage support for compact embeddings
Usage
[dependencies]
clawhdf5-agent = "2.1.0"
License
MIT