osobhandClaude Fable 5.1
f3d63dbdcd
shuffle_decompress — on the read path of every compressed dataset, since shuffle is applied automatically before compression — was the naive `result[i * es + j] = data[j * n + i]`: a multiply and two bounds checks per byte. It now interleaves fixed-width arrays of byte planes for element sizes 2/4/8/16 (bounds checks hoisted, vectorisable), with a chunked generic fallback. The write-side shuffle was already optimised; this was the asymmetry the survey flagged. Modest wall-clock effect now that decode is parallel (chunked+deflate full read ~70 -> ~66 ms). Round-trip test over element sizes 1-24 and several lengths. Co-Authored-By: Claude Fable 5.1 <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT