docs: changelog and known issues for the plugin-filter review fixes
The short-decoding chunk (wrong data, pre-existing), the Blosc header underflow (crash) and filter 32023 registration, each with its date and what it changes; the conformance count is unchanged at 573 of 697. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -31,6 +31,27 @@
|
|||||||
missing feature ("unsupported filter: 32026 (Blosc2, not implemented by
|
missing feature ("unsupported filter: 32026 (Blosc2, not implemented by
|
||||||
clawhdf5)").
|
clawhdf5)").
|
||||||
- **Not implemented:** Blosc2 (32026) and ZFP (32013) remain a clear error.
|
- **Not implemented:** Blosc2 (32026) and ZFP (32013) remain a clear error.
|
||||||
|
- **Wrong data: a chunk that decodes short read as zeros** (pre-existing, every
|
||||||
|
filter). HDF5 stores every chunk at the full chunk size, so a filter
|
||||||
|
pipeline that decodes to fewer bytes means a corrupt chunk; every chunk
|
||||||
|
reader (full, cached, selection, parallel, partial) padded it with zeros.
|
||||||
|
It is now an error naming the chunk ("chunk at [16] decoded to 16 bytes,
|
||||||
|
expected 32"), via the new `filters::decompress_chunk_exact`. libhdf5
|
||||||
|
returns the rest of such a chunk uninitialised, or fails when the filter
|
||||||
|
checks. A Blosc frame declaring no data for a non-empty chunk is an error
|
||||||
|
too. Legitimate edge chunks are unaffected (they are stored full-size,
|
||||||
|
filtered or not); conformance is unchanged at 573 of 697, with no file
|
||||||
|
changing class.
|
||||||
|
- **Crash: a hostile Blosc chunk panicked** in builds with overflow checks
|
||||||
|
(debug builds, `cargo test`, `maturin develop`): a frame size below the
|
||||||
|
16-byte header underflowed. It is now an error. Every new decoder (LZF,
|
||||||
|
bitshuffle, bzip2, Blosc/BloscLZ) is fuzzed with random and mutated frames
|
||||||
|
in the unit tests.
|
||||||
|
- **`register_filter(32023, ..)` works with the `pcodec` feature.** 32023 is
|
||||||
|
Granular BitRound's ID; the built-in entry there only reads clawhdf5
|
||||||
|
<= 2.7.0's pcodec chunks (filter name `"pcodec"`), so a registered codec now
|
||||||
|
handles every other chunk with that ID, and writes. It was refused as
|
||||||
|
"built in".
|
||||||
|
|
||||||
### Upgrade Notes
|
### Upgrade Notes
|
||||||
- **HDF5 correctness audit (2026-09-25).** A sweep of 686 public files (the
|
- **HDF5 correctness audit (2026-09-25).** A sweep of 686 public files (the
|
||||||
|
|||||||
+10
-1
@@ -140,7 +140,16 @@ fill-value item that did is fixed).
|
|||||||
697 ok). **Still open:** Blosc2 (32026 — hdf5plugin stores each chunk as a
|
697 ok). **Still open:** Blosc2 (32026 — hdf5plugin stores each chunk as a
|
||||||
Blosc2 super-chunk frame, and n-D chunks as B2ND arrays) and ZFP (32013);
|
Blosc2 super-chunk frame, and n-D chunks as B2ND arrays) and ZFP (32013);
|
||||||
both fail with an `UnsupportedFilter` error that names the filter, and
|
both fail with an `UnsupportedFilter` error that names the filter, and
|
||||||
either can be plugged in with `filter_registry::register_filter`.
|
either can be plugged in with `filter_registry::register_filter` (32023,
|
||||||
|
Granular BitRound, too, since 2026-09-26 even with the `pcodec` feature).
|
||||||
|
- **Wrong data: a chunk whose filters decode to fewer bytes than the chunk
|
||||||
|
read with zeros for the missing bytes** (any filter; found reviewing the plugin
|
||||||
|
filters). **Fixed 2026-09-26:** it is an error naming the chunk. A corrupt
|
||||||
|
chunk must never read as zeros. Unfiltered chunks are read at their stored
|
||||||
|
size and are not checked this way.
|
||||||
|
- **Crash:** a hostile Blosc chunk (frame size below its header) panicked in
|
||||||
|
builds with overflow checks. **Fixed 2026-09-26**; the new decoders are
|
||||||
|
fuzzed in the unit tests.
|
||||||
- **Header checks:** on 12 CVE datasets libhdf5 rejects a corrupt header and
|
- **Header checks:** on 12 CVE datasets libhdf5 rejects a corrupt header and
|
||||||
we read data anyway. We need stricter header checks.
|
we read data anyway. We need stricter header checks.
|
||||||
- **Writer:**
|
- **Writer:**
|
||||||
|
|||||||
Reference in New Issue
Block a user