osobhandClaude Opus 5.5
85eb7f5ce2
HDF5 1.4/1.6-era files store the layout as version 1 or 2: version, dimensionality, class, 5 reserved bytes, an address (contiguous and chunked only), dimensionality 32-bit sizes (with the trailing element-size dimension) and, for compact storage, a 32-bit size and the raw data. They failed with InvalidLayoutVersion — 84 of the 686 files in the audit sweep, 205 datasets. Map them onto the existing variants: chunked uses the same version-1 B-tree chunk index as version 3 and is reported as version 3, so every chunked read path (filters, selections, caches) applies unchanged. Contiguous size is the product of the stored dimensions, which is what libhdf5 computes from the dataspace; a disagreement fails the reader's size check instead of returning wrong data. Fixtures are HDF5's own deflate.h5 (v1, chunked + deflate) and h5ex_g_iterate.h5 (v2, contiguous, one unallocated dataset); the new interop test compares every dataset byte for byte against h5py. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT