Two write-side bugs, both present in every release (the first at least
since v2.1.0), made libhdf5 refuse files written by clawhdf5. Our own
reader ignores both fields, and the interop suites only ever wrote f64
from our side, so nothing here caught them.
- Every f32 dataset: "sign bit position out of bounds". The float
datatype encoder hard-coded the sign bit's position (bits 8-15 of the
class bit field) to 63, which is right only for f64. It is now derived
from the type: bit_offset + bit_precision - 1. This covered every
agent store's embeddings, norms and activation weights.
- Every empty dataset: "invalid dataset size, likely file corruption".
It was written with a real address and size 0, which trips libhdf5's
`addr + size <= addr` overflow check. An empty contiguous dataset now
gets the undefined address, as libhdf5 writes it. This covered every
agent store without sessions or a knowledge graph.
Agent stores are rewritten in full at each checkpoint, so they become
readable at their next checkpoint on a fixed build; other files with f32
or empty datasets need rewriting. Both are recorded in
docs/known-issues.md.
Tests: the sign position byte for f32/f64, and h5py reading our f32
datasets (plain and chunked + deflate) bit for bit.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
