2026-09-26 02:42:54 +00:00
1 changed files with 25 additions and 0 deletions
@@ -179,6 +179,31 @@
  `float16` rounding matches numpy's bit for bit on 4 020 probe values,
  including ties, subnormals and the overflow boundary), and an agent store —
  `f32` and `float16` — opened by h5py with every dataset decoded.
+- `clawhdf5-format` filters, checked against libhdf5 + hdf5plugin:
+  - **LZ4 (32004) now uses the registered HDF5 LZ4 format** (8-byte BE size,
+    4-byte BE block size, BE-length-prefixed blocks). Our old framing (4-byte
+    LE size + one block) was readable only by clawhdf5, and we could not read
+    libhdf5's (`h5ex_d_lz4.h5`). Old clawhdf5 LZ4 chunks still read; they are
+    told apart unambiguously (a registered chunk starts with four zero bytes).
+  - **Zstd (32015) frames now record the content size**, which libhdf5's zstd
+    plugin needs; h5py could not read our zstd datasets.
+  - **Pcodec moved from filter ID 32023 to 480.** 32023 is registered to
+    Granular BitRound, whose decode is a pass-through — libhdf5 with that
+    plugin would have returned compressed bytes as data. Pcodec has no
+    registered ID; 480 is in the registry's private range (256–511) and only
+    clawhdf5 can read it. Chunks written under 32023 with the filter name
+    `pcodec` (clawhdf5 ≤ 2.7.0) still read.
+  - **SZIP decode matches libhdf5.** It returned garbage or zeros with no
+    error for libhdf5-written files (the 4-byte size prefix, 32/64-bit
+    byte-plane interleaving, reference interval, scanline padding and byte
+    order were all handled wrongly) and rejected 64-bit data.
+  - N-Bit honours libhdf5's "need not compress" flag (multi-filter pipelines
+    such as `tfilters.h5` failed) and reads enum/no-op members.
+  - Scale-offset `float` decode uses libhdf5's single-precision arithmetic
+    (was 1 ULP off for some values).
+  - A pipeline with Fletcher32 ahead of the compressor (h5py
+    `set_fletcher32()` then `set_deflate()`) no longer fails with "deflate:
+    output exceeds size limit".

 ### Storage
 - `clawhdf5-format`: **half-precision datasets.**