docs: plugin filters and the filter registry

CHANGELOG (Unreleased): LZF, bitshuffle, bzip2 and Blosc read and write
in pure Rust, their features, the ChunkOptions::plugin field (breaking for
struct-literal construction), the filter registry, the named
UnsupportedFilter message, and Blosc2/ZFP still unimplemented.
README: the clawhdf5-format feature table gains lzf (default),
bitshuffle, bzip2, blosc and plugin-filters, with how to write them and
what is not implemented; no speed claims. docs/known-issues.md: the audit's
filter gap is marked fixed 2026-09-26 for LZF/bitshuffle/bzip2/Blosc,
Blosc2 and ZFP still open. CLAUDE.md: the clawhdf5-filters row no longer
says "No Blosc". clawhdf5-format's crate docs list the new features.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

CHANGELOG.md

@@ -2,6 +2,36 @@
 
 ## Unreleased
 
+### Plugin filters (2026-09-26)
+- **LZF, bitshuffle, bzip2 and Blosc read and write, in pure Rust.** Files
+  written by h5py with `compression="lzf"`, or with hdf5plugin's
+  `Bitshuffle`, `BZip2` and `Blosc`, failed with `UnsupportedFilter`. New
+  `clawhdf5-format`/`clawhdf5` features: `lzf` (32000, **on by default**, no
+  dependencies), `bitshuffle` (32008: transpose only, LZ4 and Zstandard
+  modes), `bzip2` (307), `blosc` (32001: Blosc 1 frames with BloscLZ,
+  LZ4/LZ4HC, Snappy, Zlib and Zstandard codecs and byte/bit shuffle;
+  BloscLZ is decoded by a port of c-blosc 1.21's decoder, and cannot be
+  written), and `plugin-filters` for all four. None compiles C: Zstandard is
+  ruzstd, bzip2 is libbz2-rs-sys. Write with `DatasetBuilder::with_lzf()`,
+  `with_bitshuffle(..)`, `with_bzip2(..)`, `with_blosc(..)` or
+  `with_plugin_filter(PluginFilter::..)`; `ChunkOptions` gains a `plugin`
+  field (**breaking** for code that builds `ChunkOptions` with a struct
+  literal and no `..Default::default()`). Tested both ways against h5py 3.16
+  + hdf5plugin 7.1 over 1-3-D shapes with partial edge chunks, 1-8-byte
+  types in both byte orders and incompressible data
+  (`crates/clawhdf5/tests/plugin_filters_interop.rs`). Conformance: 573 of
+  697 files ok (was 569) — h5ex_d_lzf/bshuf/bzip2/blosc.
+- **Filter registry.** Filters are looked up by ID in
+  `clawhdf5_format::filter_registry` instead of a `match`: the built-in
+  table (per build), then codecs registered at run time with
+  `register_filter(id, codec)` — a decoding closure or a `FilterCodec` that
+  can also encode. Built-in IDs cannot be overridden; a registered decoder's
+  output is held to the chunk-size bound. Unknown IDs still fail with
+  `UnsupportedFilter(id)`, whose message now names known filters and the
+  missing feature ("unsupported filter: 32026 (Blosc2, not implemented by
+  clawhdf5)").
+- **Not implemented:** Blosc2 (32026) and ZFP (32013) remain a clear error.
+
 ### Upgrade Notes
 - **HDF5 correctness audit (2026-09-25).** A sweep of 686 public files (the
   libhdf5 test files, the HDF Group's CVE reproducers, pyfive, netcdf-c,

CLAUDE.md

@@ -11,7 +11,7 @@ Cargo workspace with 16 crates under `crates/` (plus `libaec-sys`, an internal F
 |-------|------|
 | `clawhdf5-format` | HDF5 binary spec parser (superblock, B-tree, heap) — also holds shared type definitions and physical constants |
 | `clawhdf5-io` | Read/write implementation |
-| `clawhdf5-filters` | Deflate backends (zlib-rs, zlib-ng, Apple Compression); the HDF5 filter pipeline and the other codecs (LZ4, Zstd, SZIP, N-Bit, scale-offset, pcodec) live in `clawhdf5-format`. No Blosc. |
+| `clawhdf5-filters` | Deflate backends (zlib-rs, zlib-ng, Apple Compression); the HDF5 filter pipeline, the filter registry (`clawhdf5_format::filter_registry`) and the other codecs (LZ4, Zstd, SZIP, N-Bit, scale-offset, pcodec, and the pure-Rust plugin filters LZF, bitshuffle, bzip2, Blosc 1) live in `clawhdf5-format`. No Blosc2 or ZFP. |
 | `clawhdf5-derive` | Proc-macro derive for HDF5-serializable structs |
 | `clawhdf5` | Main facade crate |
 | `clawhdf5-netcdf4` | NetCDF-4 compatibility layer |

README.md

@@ -594,7 +594,7 @@ clawhdf5 workspace (16 crates, ~86K lines of Rust in src/, ~104K with tests
 ├── Core HDF5
 │   ├── clawhdf5-format      — Binary parser/writer (no_std-capable), shared type definitions
 │   ├── clawhdf5-io          — I/O abstraction (file/memory readers; optional mmap, async, HSDS, MPI)
-│   ├── clawhdf5-filters     — Fast deflate path (zlib-ng); lz4/zstd/pcodec/szip filters live in clawhdf5-format
+│   ├── clawhdf5-filters     — Fast deflate path (zlib-ng); the filter registry and the lz4/zstd/pcodec/szip/LZF/bitshuffle/bzip2/Blosc filters live in clawhdf5-format
 │   ├── clawhdf5-derive      — Proc macros
 │   ├── clawhdf5             — High-level API
 │   ├── clawhdf5-netcdf4     — NetCDF-4 support
@@ -700,6 +700,22 @@ stores keep their setting. Opt out with `float16 = false` or
 | `system-zlib` | no | System zlib backend for deflate (C) |
 | `blake3_hash` | no | BLAKE3 content hashing for provenance |
 | `szip` | no | SZIP filter (id 4) via libaec (C, through the internal `libaec-sys` crate) |
+| `lzf` | **yes** | LZF filter (id 32000), h5py's built-in `compression="lzf"`: read and write. No dependencies |
+| `bitshuffle` | no | Bitshuffle filter (id 32008) with its LZ4 and Zstandard modes: read and write. Pure Rust (lz4_flex, ruzstd) |
+| `bzip2` | no | bzip2 filter (id 307): read and write. Pure Rust (the `bzip2` crate's libbz2-rs-sys backend compiles no C) |
+| `blosc` | no | Blosc 1 filter (id 32001): reads BloscLZ, LZ4/LZ4HC, Snappy, Zlib and Zstandard frames with byte or bit shuffle; writes LZ4, Snappy, Zlib or Zstandard (not BloscLZ). Pure Rust |
+| `plugin-filters` | no | All four above |
+
+Blosc2 (32026) and ZFP (32013) are not implemented: reading them fails with
+`UnsupportedFilter`, whose message names the filter. Any other filter can be
+supplied at run time with `filter_registry::register_filter` (a decoder
+closure, or a `FilterCodec` that also encodes). The facade (`clawhdf5`)
+forwards `lzf`, `bitshuffle`, `bzip2`, `blosc` and `plugin-filters`. Write
+with `DatasetBuilder::with_lzf()`, `with_bitshuffle(..)`, `with_bzip2(..)`
+and `with_blosc(..)`; h5py + hdf5plugin read the result (tested both ways in
+`crates/clawhdf5/tests/plugin_filters_interop.rs`). The pure-Rust Zstandard
+encoder has one level (about zstd's level 1); no speed or ratio claims are
+made for these codecs.
 
 ### `clawhdf5-ann`
 

crates/clawhdf5-format/src/lib.rs

@@ -43,6 +43,14 @@
 //! | `checksum` | yes | Jenkins lookup3 checksum validation |
 //! | `deflate` | yes | Deflate (gzip) compression via `flate2` |
 //! | `provenance` | yes | SHINES provenance — SHA-256 hashing & verification |
+//! | `lzf` | yes | LZF filter (32000), h5py's `compression="lzf"` |
+//! | `bitshuffle` | no | Bitshuffle filter (32008), none/LZ4/Zstandard |
+//! | `bzip2` | no | bzip2 filter (307) |
+//! | `blosc` | no | Blosc 1 filter (32001) |
+//! | `plugin-filters` | no | The four above |
+//!
+//! Filters are looked up by ID in [`filter_registry`], which also takes
+//! codecs registered at run time for other IDs.
 
 #![cfg_attr(not(feature = "std"), no_std)]
 

docs/known-issues.md

@@ -133,7 +133,14 @@ fill-value item that did is fixed).
   - x87 long double and binary128 are refused.
   - N-Bit on 64-bit scale-offset data and some N-Bit parameter layouts fail.
 - **Filters:** blosc, blosc2, bitshuffle, bzip2, LZF and zfp are not
-  implemented.
+  implemented. **Fixed 2026-09-26** for LZF (default-on `lzf` feature),
+  bitshuffle, bzip2 and Blosc 1 (`bitshuffle`, `bzip2`, `blosc`, or
+  `plugin-filters` for all), read and write, pure Rust; h5ex_d_lzf,
+  h5ex_d_bshuf, h5ex_d_bzip2 and h5ex_d_blosc now read (conformance 573 of
+  697 ok). **Still open:** Blosc2 (32026 — hdf5plugin stores each chunk as a
+  Blosc2 super-chunk frame, and n-D chunks as B2ND arrays) and ZFP (32013);
+  both fail with an `UnsupportedFilter` error that names the filter, and
+  either can be plugged in with `filter_registry::register_filter`.
 - **Header checks:** on 12 CVE datasets libhdf5 rejects a corrupt header and
   we read data anyway. We need stricter header checks.
 - **Writer:**