osobhandClaude Opus 5.5
d9e4dfb6e6
ruzstd reserves a frame's declared window (up to its 100 MiB default) when a decoder is reset for a new frame, before decoding anything. The Blosc, Blosc2 and bitshuffle decoders reuse one decoder per chunk, so a Blosc2 chunk of two 16-byte streams, each declaring a 96 MiB window, allocated 128 MiB. zstd_decode_into now sets the decoder's maximum window to twice the stream's output (at least 128 KiB): c-blosc, c-blosc2 and bitshuffle compress each block in one call with its size known, so libzstd's window never exceeds the block. Found by tracking peak allocation in the Blosc2 fuzz test. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT