fix(format): stop writing pcodec under Granular BitRound's filter ID

Pcodec chunks were written as filter 32023, which the HDF Group registry
assigns to Granular BitRound (GBR). Pcodec has no registered ID (checked
2026-09-25 against hdf5_plugins/docs/RegisteredFilterPlugins.md, which
ends at 32033 with no pcodec entry). GBR's decode is a pass-through, so
libhdf5 with that plugin loaded would have returned the compressed bytes
as the dataset's values.

Write pcodec as 480, from the registry's testing/private range (256-511),
named "pcodec (clawhdf5 private)", and document it as non-interoperable:
only clawhdf5 with the `pcodec` feature reads it. Chunks under 32023 are
still read as pcodec when the filter is named exactly "pcodec" (what
clawhdf5 <= 2.7.0 wrote); any other 32023 is UnsupportedFilter.

Test: pcodec_uses_private_id_and_reads_legacy_32023.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-25 21:06:07 -05:00
co-authored by Claude Opus 5.5
parent 183d96ee26
commit 06dda26d85
5 changed files with 88 additions and 10 deletions
+5 -4
View File
@@ -11,8 +11,8 @@ use crate::chunk_cache::{CACHE_LINE_SIZE, align_to_cache_line};
use crate::ea_writer;
use crate::error::FormatError;
use crate::filter_pipeline::{
FILTER_DEFLATE, FILTER_FLETCHER32, FILTER_LZ4, FILTER_PCODEC, FILTER_SHUFFLE, FILTER_ZSTD,
FilterDescription, FilterPipeline,
FILTER_DEFLATE, FILTER_FLETCHER32, FILTER_LZ4, FILTER_PCODEC, FILTER_PCODEC_NAME,
FILTER_SHUFFLE, FILTER_ZSTD, FilterDescription, FilterPipeline,
};
use crate::filters::compress_chunk;
/// Round a file offset up to the next cache-line boundary.
@@ -44,7 +44,8 @@ pub struct ChunkOptions {
pub lz4: bool,
/// Zstandard compression level (1-22), None = no zstd. Filter ID 32015.
pub zstd_level: Option<u32>,
/// Pcodec lossless numerical compression. Filter ID 32023.
/// Pcodec lossless numerical compression. Private, unregistered filter
/// ID [`FILTER_PCODEC`] (480): only clawhdf5 can read it.
pub pcodec: bool,
}
@@ -115,7 +116,7 @@ impl ChunkOptions {
if self.pcodec {
filters.push(FilterDescription {
filter_id: FILTER_PCODEC,
name: Some("pcodec".into()),
name: Some(FILTER_PCODEC_NAME.into()),
flags: 0,
client_data: vec![element_size],
});