feat(format): LZF filter (32000), read and write, pure Rust

h5py's built-in compression="lzf" failed with UnsupportedFilter(32000). The
new `lzf` feature (no dependencies, on by default in clawhdf5-format and
the facade) decodes the raw liblzf stream h5py's filter stores, bounded by
the chunk size, and encodes it: DatasetBuilder::with_lzf() (or
with_plugin_filter(PluginFilter::Lzf)) writes the filter with h5py's
cd_values (filter version 4, liblzf 0x0105, chunk size in bytes), flagged
optional as h5py does. ChunkOptions gains a `plugin` field for the plugin
filters; build_pipeline_for_chunk passes the chunk size to filters that
record it.

tests/plugin_filters_interop.rs: h5py writes LZF (alone, with shuffle, with
shuffle+fletcher32) over 12 dtype/shape/chunk/data cases with partial edge
chunks and incompressible data, and every dataset reads byte for byte equal
to its unfiltered twin; our LZF output (1-D and 2-D, edge chunks, with and
without shuffle) reads back in h5py. Both fail with the decoder removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

This commit is contained in:

osobh

2026-09-26 00:01:22 -05:00

co-authored by

Claude Opus 5.5

parent d16544b928

commit e38f9123db

9 changed files with 629 additions and 9 deletions

									
										crates/clawhdf5-format/src/lib.rs
									
		+2
		
												View File
												
				@@ -73,6 +73,8 @@ pub mod fill_value;

				pub mod filter_pipeline;

				pub mod filter_registry;

				pub mod filters;

				#[cfg(feature = "lzf")]

				pub mod filters_lzf;

				mod filters_szip;

				pub mod fixed_array;

				pub mod float16;