osobhandClaude Opus 5.5
585e14d5e2
A chunk's filter mask has one bit per pipeline filter; bit i set means filter i was not applied to that chunk (an optional filter that declined, or a direct chunk write). Every read path treated any nonzero mask as "no filters applied" and returned the stored bytes, so a chunk that skipped only gzip in a shuffle+gzip pipeline came back still shuffled (h5py write_direct_chunk with filter_mask=0b10: 8 of 32 values wrong). decompress_chunk_masked undoes the filters the mask leaves set and skips the rest; an unsupported filter is no longer an error when the chunk skipped it. The full, cached, sweep, indexed, parallel and selection (partial_read) paths all use it, and a chunk is copied straight from the file only when every filter was skipped. decompress_chunk is the mask-0 case. Regression: h5py_partial_filter_mask_skips_only_masked_filters (1-D shuffle+gzip with masks 0, 0b10 and 0b11; 2-D with 0b01; full and hyperslab reads) and filter_mask_skips_only_the_masked_filters. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT