osobhandClaude Opus 5.5
9ea44d473d
A type-1 (raw data chunk) B-tree key holds the chunk size, the filter mask and one offset per dimension, and those offsets are always 8 bytes: they are dataset coordinates, not file addresses. The reader used the superblock's size-of-offsets for them, so in a file with 4-byte offsets every key was misparsed. Unfiltered chunked datasets read as zeros (with stray bytes where a misread address landed on data) and filtered ones failed with "deflate: truncated stream". Only the sibling and child addresses follow size-of-offsets now. The unit-test B-tree builder wrote keys the same wrong way, which is why its tests passed; it now matches the format. Regression: h5py_four_byte_offsets_chunked_reads (h5py, set_sizes(4, 4) and (4, 8); 1-D and 2-D, unfiltered and gzip) and the unit test collect_chunks_with_four_byte_addresses. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT