feat(format): bitshuffle filter (32008), with its LZ4 and Zstandard modes

hdf5plugin's Bitshuffle failed with UnsupportedFilter(32008). The new
`bitshuffle` feature (pure Rust: lz4_flex, and ruzstd for Zstandard — the
`zstd` feature's libzstd is not needed) decodes all three modes of
bshuf_h5filter.c — transpose only, LZ4 and Zstandard blocks behind the
12-byte header — including the default and explicit block sizes, the
shorter last block rounded down to a multiple of 8 elements, and the
untransposed trailing elements. Sizes read from the chunk are bounded by
the chunk size.

It also encodes: DatasetBuilder::with_bitshuffle(BitshuffleCompression)
or PluginFilter::Bitshuffle { block_size, compression } writes the filter
with hdf5plugin's cd_values and no automatic byte shuffle. ruzstd has one
compression level (about zstd's 1); the requested level is recorded.

The bit transpose is checked bit for bit against a one-bit-at-a-time model
(which matched hdf5plugin's output) and is shared with blosc next.
Interop: hdf5plugin writes none/LZ4/Zstandard at default and explicit
block sizes and levels over the 12-case matrix, read byte for byte; our
three modes at two block sizes read back through hdf5plugin. Both fail
with the decoder removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 00:04:05 -05:00
co-authored by Claude Opus 5.5
parent e38f9123db
commit 07094e34a9
8 changed files with 509 additions and 3 deletions
+2
View File
@@ -73,6 +73,8 @@ pub mod fill_value;
pub mod filter_pipeline;
pub mod filter_registry;
pub mod filters;
#[cfg(feature = "bitshuffle")]
mod filters_bitshuffle;
#[cfg(feature = "lzf")]
pub mod filters_lzf;
mod filters_szip;