osobhandClaude Opus 5.5
585e14d5e2
A chunk's filter mask has one bit per pipeline filter; bit i set means filter i was not applied to that chunk (an optional filter that declined, or a direct chunk write). Every read path treated any nonzero mask as "no filters applied" and returned the stored bytes, so a chunk that skipped only gzip in a shuffle+gzip pipeline came back still shuffled (h5py write_direct_chunk with filter_mask=0b10: 8 of 32 values wrong). decompress_chunk_masked undoes the filters the mask leaves set and skips the rest; an unsupported filter is no longer an error when the chunk skipped it. The full, cached, sweep, indexed, parallel and selection (partial_read) paths all use it, and a chunk is copied straight from the file only when every filter was skipped. decompress_chunk is the mask-0 case. Regression: h5py_partial_filter_mask_skips_only_masked_filters (1-D shuffle+gzip with masks 0, 0b10 and 0b11; 2-D with 0b01; full and hyperslab reads) and filter_mask_skips_only_the_masked_filters. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5
Pure-Rust HDF5 reader/writer — no C dependencies.
Features
- Read and write HDF5 files entirely in Rust
- Memory-mapped I/O for large files (
mmapfeature, enabled by default) - Parallel chunk reads via Rayon (
parallelfeature) - Lazy dataset access for minimal memory usage
- h5py-compatible file output
Usage
use clawhdf5::File;
let file = File::open("data.h5").unwrap();
let dataset = file.dataset("/group/data").unwrap();
let values: Vec<f64> = dataset.read_1d().unwrap();
License
MIT