quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

fix(format): stop writing pcodec under Granular BitRound's filter ID

Browse Source 
Pcodec chunks were written as filter 32023, which the HDF Group registry
assigns to Granular BitRound (GBR). Pcodec has no registered ID (checked
2026-09-25 against hdf5_plugins/docs/RegisteredFilterPlugins.md, which
ends at 32033 with no pcodec entry). GBR's decode is a pass-through, so
libhdf5 with that plugin loaded would have returned the compressed bytes
as the dataset's values.

Write pcodec as 480, from the registry's testing/private range (256-511),
named "pcodec (clawhdf5 private)", and document it as non-interoperable:
only clawhdf5 with the `pcodec` feature reads it. Chunks under 32023 are
still read as pcodec when the filter is named exactly "pcodec" (what
clawhdf5 <= 2.7.0 wrote); any other 32023 is UnsupportedFilter.

Test: pcodec_uses_private_id_and_reads_legacy_32023.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-25 21:06:07 -05:00
co-authored by Claude Opus 5.5
parent 183d96ee26
commit 06dda26d85
5 changed files with 88 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -696,7 +696,7 @@ stores keep their setting. Opt out with `float16 = false` or
| `fast-checksum` | no | crc32fast-accelerated checksums |
| `lz4` | no | LZ4 block compression filter (id 32004) |
| `zstd` | no | Zstandard compression filter (id 32015) |
| `pcodec` | no | Pcodec lossless numerical codec (id 32023, via `pco` crate) |
| `pcodec` | no | Pcodec lossless numerical codec (via `pco` crate). Private, unregistered filter id 480: **only clawhdf5 can read these datasets** (h5py/libhdf5 cannot). Files from clawhdf5 <= 2.7.0 used id 32023, which is registered to Granular BitRound; they still read. |
| `system-zlib` | no | System zlib backend for deflate (C) |
| `blake3_hash` | no | BLAKE3 content hashing for provenance |
| `szip` | no | SZIP filter (id 4) via libaec (C, through the internal `libaec-sys` crate) |