feat(format): bitshuffle filter (32008), with its LZ4 and Zstandard modes
hdf5plugin's Bitshuffle failed with UnsupportedFilter(32008). The new
`bitshuffle` feature (pure Rust: lz4_flex, and ruzstd for Zstandard — the
`zstd` feature's libzstd is not needed) decodes all three modes of
bshuf_h5filter.c — transpose only, LZ4 and Zstandard blocks behind the
12-byte header — including the default and explicit block sizes, the
shorter last block rounded down to a multiple of 8 elements, and the
untransposed trailing elements. Sizes read from the chunk are bounded by
the chunk size.
It also encodes: DatasetBuilder::with_bitshuffle(BitshuffleCompression)
or PluginFilter::Bitshuffle { block_size, compression } writes the filter
with hdf5plugin's cd_values and no automatic byte shuffle. ruzstd has one
compression level (about zstd's 1); the requested level is recorded.
The bit transpose is checked bit for bit against a one-bit-at-a-time model
(which matched hdf5plugin's output) and is shared with blosc next.
Interop: hdf5plugin writes none/LZ4/Zstandard at default and explicit
block sizes and levels over the 12-case matrix, read byte for byte; our
three modes at two block sizes read back through hdf5plugin. Both fail
with the decoder removed.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -73,6 +73,8 @@ pub mod fill_value;
|
||||
pub mod filter_pipeline;
|
||||
pub mod filter_registry;
|
||||
pub mod filters;
|
||||
#[cfg(feature = "bitshuffle")]
|
||||
mod filters_bitshuffle;
|
||||
#[cfg(feature = "lzf")]
|
||||
pub mod filters_lzf;
|
||||
mod filters_szip;
|
||||
|
||||
Reference in New Issue
Block a user