osobhandClaude Opus 5
b41272487a
Every structure in both chunk indexes — header, index block, super block, data block and each data block page — carries a Jenkins lookup3 checksum, and all of them were parsed past and ignored. What that costs is not a warning but correct data. Flip one low bit of a chunk address and the index still has the right shape, the address still lands inside the file, and the reader returns whatever bytes now sit there as that chunk's contents. Nothing else in the parse can tell. Verified in both directions. The checksums accept files written by HDF5 2.0 from 100 to 200 000 chunks — dense, sparse, gzip-filtered and paged — which also confirms the block layouts byte for byte, since a wrong offset would fail every file. And an interop test corrupts an address to check the read fails instead of returning data: removing the verification makes that test fail with "corruption produced data instead of an error", which is what it is there to prove. The first version of that test passed with verification disabled — it corrupted a byte a structural check already rejected, so it proved nothing. Worth recording, since a test that passes for the wrong reason looks exactly like coverage. Hand-built fixtures now stamp real checksums, as HDF5 writers do, and the Extensible Array ones no longer describe the superseded layout. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT