docs: CHANGELOG for the Blosc2 allocation bounds
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -21,6 +21,21 @@
|
|||||||
python-blosc2 4.13.1 for what hdf5plugin never writes
|
python-blosc2 4.13.1 for what hdf5plugin never writes
|
||||||
(`crates/clawhdf5-format/tests/fixtures/blosc2/`); the decoder is
|
(`crates/clawhdf5-format/tests/fixtures/blosc2/`); the decoder is
|
||||||
fuzzed. Conformance: 576 of 697 files ok (was 575) — h5ex_d_blosc2.
|
fuzzed. Conformance: 576 of 697 files ok (was 575) — h5ex_d_blosc2.
|
||||||
|
- **A crafted Blosc2 chunk cannot allocate more than a few times its HDF5
|
||||||
|
chunk size.** Found in review before release: the sizes a frame declares
|
||||||
|
sized the decoder's buffers. A 173-byte frame whose offsets chunk claimed
|
||||||
|
2 GiB was decoded in full for a 1 MiB chunk; an empty chunk allocated its
|
||||||
|
declared block size twice (about 1 GiB); a B2ND chunk was decoded whole
|
||||||
|
with its padding (up to 16x the chunk); and a Zstandard stream's declared
|
||||||
|
window (up to 100 MiB) was reserved as the decoder was reused, which also
|
||||||
|
affected Blosc 1 and bitshuffle with Zstandard. Now the offsets chunk is
|
||||||
|
capped at the chunk size, block sizes are clamped to the chunk, B2ND
|
||||||
|
blocks are placed as they are decoded (padding is never held), and the
|
||||||
|
Zstandard window is capped at twice the stream's output (at least
|
||||||
|
128 KiB). A B2ND chunk may no longer be larger than its array, which
|
||||||
|
hdf5-blosc2 never writes. `tests/blosc2_alloc_bounds.rs` measures peak
|
||||||
|
allocation for these frames and for 45,000 fuzzed ones: at most 6x the
|
||||||
|
chunk size, twice the input and 2 MiB of Zstandard state.
|
||||||
|
|
||||||
### Concurrent reads (2026-09-26)
|
### Concurrent reads (2026-09-26)
|
||||||
- **Full reads of chunked datasets scale with threads again when rayon's
|
- **Full reads of chunked datasets scale with threads again when rayon's
|
||||||
|
|||||||
Reference in New Issue
Block a user