From 17fc8b19645f0ac0c5c66cdfdb01286f019250b6 Mon Sep 17 00:00:00 2001 From: osobh Date: Sat, 26 Sep 2026 01:24:18 -0500 Subject: [PATCH] docs: changelog and known issues for the plugin-filter review fixes The short-decoding chunk (wrong data, pre-existing), the Blosc header underflow (crash) and filter 32023 registration, each with its date and what it changes; the conformance count is unchanged at 573 of 697. Co-Authored-By: Claude Opus 5.5 (1M context) --- CHANGELOG.md | 21 +++++++++++++++++++++ docs/known-issues.md | 11 ++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9318dbc..796c459 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,27 @@ missing feature ("unsupported filter: 32026 (Blosc2, not implemented by clawhdf5)"). - **Not implemented:** Blosc2 (32026) and ZFP (32013) remain a clear error. +- **Wrong data: a chunk that decodes short read as zeros** (pre-existing, every + filter). HDF5 stores every chunk at the full chunk size, so a filter + pipeline that decodes to fewer bytes means a corrupt chunk; every chunk + reader (full, cached, selection, parallel, partial) padded it with zeros. + It is now an error naming the chunk ("chunk at [16] decoded to 16 bytes, + expected 32"), via the new `filters::decompress_chunk_exact`. libhdf5 + returns the rest of such a chunk uninitialised, or fails when the filter + checks. A Blosc frame declaring no data for a non-empty chunk is an error + too. Legitimate edge chunks are unaffected (they are stored full-size, + filtered or not); conformance is unchanged at 573 of 697, with no file + changing class. +- **Crash: a hostile Blosc chunk panicked** in builds with overflow checks + (debug builds, `cargo test`, `maturin develop`): a frame size below the + 16-byte header underflowed. It is now an error. Every new decoder (LZF, + bitshuffle, bzip2, Blosc/BloscLZ) is fuzzed with random and mutated frames + in the unit tests. +- **`register_filter(32023, ..)` works with the `pcodec` feature.** 32023 is + Granular BitRound's ID; the built-in entry there only reads clawhdf5 + <= 2.7.0's pcodec chunks (filter name `"pcodec"`), so a registered codec now + handles every other chunk with that ID, and writes. It was refused as + "built in". ### Upgrade Notes - **HDF5 correctness audit (2026-09-25).** A sweep of 686 public files (the diff --git a/docs/known-issues.md b/docs/known-issues.md index 53c4996..11c2ea9 100644 --- a/docs/known-issues.md +++ b/docs/known-issues.md @@ -140,7 +140,16 @@ fill-value item that did is fixed). 697 ok). **Still open:** Blosc2 (32026 — hdf5plugin stores each chunk as a Blosc2 super-chunk frame, and n-D chunks as B2ND arrays) and ZFP (32013); both fail with an `UnsupportedFilter` error that names the filter, and - either can be plugged in with `filter_registry::register_filter`. + either can be plugged in with `filter_registry::register_filter` (32023, + Granular BitRound, too, since 2026-09-26 even with the `pcodec` feature). +- **Wrong data: a chunk whose filters decode to fewer bytes than the chunk + read with zeros for the missing bytes** (any filter; found reviewing the plugin + filters). **Fixed 2026-09-26:** it is an error naming the chunk. A corrupt + chunk must never read as zeros. Unfiltered chunks are read at their stored + size and are not checked this way. +- **Crash:** a hostile Blosc chunk (frame size below its header) panicked in + builds with overflow checks. **Fixed 2026-09-26**; the new decoders are + fuzzed in the unit tests. - **Header checks:** on 12 CVE datasets libhdf5 rejects a corrupt header and we read data anyway. We need stricter header checks. - **Writer:**