From 7515e5dcbdd13c3399cf79f3f8371e6a92adb1e2 Mon Sep 17 00:00:00 2001 From: osobh Date: Sat, 26 Sep 2026 11:35:19 -0500 Subject: [PATCH] docs: CHANGELOG for the Blosc2 allocation bounds Co-Authored-By: Claude Opus 5.5 (1M context) --- CHANGELOG.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index aebeb93..51e3040 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,21 @@ python-blosc2 4.13.1 for what hdf5plugin never writes (`crates/clawhdf5-format/tests/fixtures/blosc2/`); the decoder is fuzzed. Conformance: 576 of 697 files ok (was 575) — h5ex_d_blosc2. +- **A crafted Blosc2 chunk cannot allocate more than a few times its HDF5 + chunk size.** Found in review before release: the sizes a frame declares + sized the decoder's buffers. A 173-byte frame whose offsets chunk claimed + 2 GiB was decoded in full for a 1 MiB chunk; an empty chunk allocated its + declared block size twice (about 1 GiB); a B2ND chunk was decoded whole + with its padding (up to 16x the chunk); and a Zstandard stream's declared + window (up to 100 MiB) was reserved as the decoder was reused, which also + affected Blosc 1 and bitshuffle with Zstandard. Now the offsets chunk is + capped at the chunk size, block sizes are clamped to the chunk, B2ND + blocks are placed as they are decoded (padding is never held), and the + Zstandard window is capped at twice the stream's output (at least + 128 KiB). A B2ND chunk may no longer be larger than its array, which + hdf5-blosc2 never writes. `tests/blosc2_alloc_bounds.rs` measures peak + allocation for these frames and for 45,000 fuzzed ones: at most 6x the + chunk size, twice the input and 2 MiB of Zstandard state. ### Concurrent reads (2026-09-26) - **Full reads of chunked datasets scale with threads again when rayon's