Merge branch 'feat/p3-zfp' into feat/p3-range-zfp-edit

# Conflicts:
#	CHANGELOG.md
#	crates/clawhdf5-format/Cargo.toml

CHANGELOG.md

@@ -149,6 +149,30 @@
   converted parser falling back to the whole file fails it. Milestones M2
   and M3 extend it.
 
+### ZFP (2026-09-26)
+- **ZFP (filter 32013, H5Z-ZFP) reads, in pure Rust.** It was the last
+  filter in the conformance corpus that failed with `UnsupportedFilter`.
+  New feature `zfp` (`clawhdf5-format` and `clawhdf5`, included in
+  `plugin-filters`, no dependencies) ports the zfp 1.0.1 decoder and the
+  decompression half of H5Z-ZFP 1.1.1: every mode (fixed rate, fixed
+  precision, fixed accuracy, reversible, expert), int32, int64, float and
+  double, 1-4-D fields with partial blocks, and headers written by
+  big-endian machines (values byte-swapped as H5Z-ZFP does). Read only:
+  there is no ZFP encoder. The decoder is deterministic, so lossy modes have
+  one right answer, and clawhdf5 returns exactly libzfp's values.
+- Tests: `crates/clawhdf5/tests/zfp_interop.rs` has h5py + hdf5plugin 7.1
+  (H5Z-ZFP 1.1.1, zfp 1.0.1) write 2205 datasets over 16 mode settings
+  (including expert parameters at their edges) x the four types x 1-4-D
+  shapes with partial edge chunks, partial blocks and unit chunk
+  dimensions x smooth, noisy, wide-range, zero and inf/NaN data; each must
+  read byte for byte as h5py reads it. `tests/zfp_alloc_bounds.rs` fuzzes
+  headers and streams under a counting allocator: no panics, and the output
+  is allocated only when it matches the chunk's size and the stream holds at
+  least a bit per block. A stream that ends before the decoder is done is an
+  error (libzfp reads past its buffer).
+- Conformance on tank (2026-09-26, `conformance/run.sh --no-fetch`): 600 of
+  697 files ok (599 before); `h5ex_d_zfp.h5` now reads.
+
 ### Chunked full reads (2026-09-26)
 - **Chunks are decoded straight into the output, into reused buffers.** A
   full read of a chunked dataset faulted in about three times its size in

CLAUDE.md

@@ -11,7 +11,7 @@ Cargo workspace with 18 crates under `crates/` (plus `libaec-sys`, an internal F
 |-------|------|
 | `clawhdf5-format` | HDF5 binary spec parser (superblock, B-tree, heap) — also holds shared type definitions and physical constants |
 | `clawhdf5-io` | Read/write implementation |
-| `clawhdf5-filters` | Deflate backends (zlib-rs, zlib-ng, Apple Compression); the HDF5 filter pipeline, the filter registry (`clawhdf5_format::filter_registry`) and the other codecs (LZ4, Zstd, SZIP, N-Bit, scale-offset, pcodec, and the pure-Rust plugin filters LZF, bitshuffle, bzip2, Blosc 1, and Blosc2 read-only) live in `clawhdf5-format`. No ZFP. |
+| `clawhdf5-filters` | Deflate backends (zlib-rs, zlib-ng, Apple Compression); the HDF5 filter pipeline, the filter registry (`clawhdf5_format::filter_registry`) and the other codecs (LZ4, Zstd, SZIP, N-Bit, scale-offset, pcodec, and the pure-Rust plugin filters LZF, bitshuffle, bzip2, Blosc 1, and Blosc2 and ZFP read-only) live in `clawhdf5-format`. |
 | `clawhdf5-derive` | Proc-macro derive for HDF5-serializable structs |
 | `clawhdf5` | Main facade crate |
 | `clawhdf5-netcdf4` | NetCDF-4 compatibility layer |

README.md

@@ -780,14 +780,14 @@ stores keep their setting. Opt out with `float16 = false` or
 | `bzip2` | no | bzip2 filter (id 307): read and write. Pure Rust (the `bzip2` crate's libbz2-rs-sys backend compiles no C) |
 | `blosc` | no | Blosc 1 filter (id 32001): reads BloscLZ, LZ4/LZ4HC, Snappy, Zlib and Zstandard frames with byte or bit shuffle; writes LZ4, Snappy, Zlib or Zstandard (not BloscLZ). Pure Rust |
 | `blosc2` | no | Blosc2 filter (id 32026), read only: hdf5plugin's frames and B2ND (n-D) chunks, BloscLZ, LZ4/LZ4HC, Zlib and Zstandard, with shuffle, bit shuffle, delta or truncated precision. Pure Rust |
-| `plugin-filters` | no | All five above |
+| `zfp` | no | ZFP filter (id 32013, H5Z-ZFP), read only: every mode (rate, precision, accuracy, reversible, expert) for int32, int64, float and double, 1-4-D, returning exactly libzfp's values. Pure Rust, no dependencies |
+| `plugin-filters` | no | All six above |
 
-ZFP (32013) is not implemented: reading it fails with `UnsupportedFilter`,
+clawhdf5 cannot write Blosc2 or ZFP. Any other
-whose message names the filter. clawhdf5 cannot write Blosc2. Any other
 filter can be supplied at run time with `filter_registry::register_filter` (a
 decoder closure, or a `FilterCodec` that also encodes). The facade
-(`clawhdf5`) forwards `lzf`, `bitshuffle`, `bzip2`, `blosc`, `blosc2` and
+(`clawhdf5`) forwards `lzf`, `bitshuffle`, `bzip2`, `blosc`, `blosc2`, `zfp`
-`plugin-filters`. Write
+and `plugin-filters`. Write
 with `DatasetBuilder::with_lzf()`, `with_bitshuffle(..)`, `with_bzip2(..)`
 and `with_blosc(..)`; h5py + hdf5plugin read the result (tested both ways in
 `crates/clawhdf5/tests/plugin_filters_interop.rs`). The pure-Rust Zstandard

crates/clawhdf5-format/Cargo.toml

@@ -78,8 +78,11 @@ bzip2 = ["dep:bzip2", "std"]
 blosc = ["lz4_flex", "ruzstd", "snap", "deflate", "std"]
 # Blosc2 (32026), read-only: frames, B2ND arrays, and the Blosc codecs above.
 blosc2 = ["blosc"]
+# ZFP (32013, H5Z-ZFP), read-only: every mode, for int32, int64, float and
+# double fields of 1 to 4 dimensions.
+zfp = []
 # Every plugin filter above.
-plugin-filters = ["lzf", "bitshuffle", "bzip2", "blosc", "blosc2"]
+plugin-filters = ["lzf", "bitshuffle", "bzip2", "blosc", "blosc2", "zfp"]
 # Test instrumentation: per-thread counts of heap objects read (see
 # `lookup_stats`), so tests can bound the cost of a name lookup.
 lookup-stats = ["std"]

crates/clawhdf5-format/src/filter_pipeline.rs

@@ -27,7 +27,8 @@ pub const FILTER_LZF: u16 = 32000;
 pub const FILTER_BLOSC: u16 = 32001;
 /// Bitshuffle, optionally with LZ4 or Zstandard (hdf5plugin's `Bitshuffle`).
 pub const FILTER_BITSHUFFLE: u16 = 32008;
-/// ZFP lossy floating-point compression (hdf5plugin's `Zfp`). Not supported.
+/// ZFP lossy (and lossless) compression of numeric arrays (H5Z-ZFP;
+/// hdf5plugin's `Zfp`). Read-only, with the `zfp` feature.
 pub const FILTER_ZFP: u16 = 32013;
 /// Blosc 2 (hdf5plugin's `Blosc2`).
 pub const FILTER_BLOSC2: u16 = 32026;

crates/clawhdf5-format/src/filter_registry.rs

@@ -6,7 +6,7 @@
 //!   build: the HDF5 standard filters (deflate, shuffle, Fletcher32, szip,
 //!   N-Bit, scale-offset) and the plugin filters whose cargo features are
 //!   enabled (LZ4, Zstandard, pcodec, LZF, bitshuffle, bzip2, blosc,
-//!   blosc2).
+//!   blosc2, zfp).
 //!   [`builtin_filters`] lists them.
 //! * **Registered filters** (`std` only) — codecs the application supplies
 //!   for any other ID with [`register_filter`] (a [`FilterCodec`], or just a
@@ -162,7 +162,7 @@ pub fn known_filter(id: u16) -> Option<(&'static str, Option<&'static str>)> {
         32001 => ("Blosc", Some("blosc")),
         32004 => ("LZ4", Some("lz4")),
         32008 => ("bitshuffle", Some("bitshuffle")),
-        32013 => ("ZFP", None),
+        32013 => ("ZFP", Some("zfp")),
         32015 => ("Zstandard", Some("zstd")),
         32019 => ("JPEG", None),
         32022 => ("BitGroom", None),
@@ -455,8 +455,10 @@ pub(crate) mod tests {
         let msg = FormatError::UnsupportedFilter(32026).to_string();
         assert!(msg.contains("Blosc2") && msg.contains("`blosc2`"), "{msg}");
         let msg = FormatError::UnsupportedFilter(32013).to_string();
+        assert!(msg.contains("ZFP") && msg.contains("`zfp`"), "{msg}");
+        let msg = FormatError::UnsupportedFilter(32019).to_string();
         assert!(
-            msg.contains("ZFP") && msg.contains("not implemented"),
+            msg.contains("JPEG") && msg.contains("not implemented"),
             "{msg}"
         );
         let msg = FormatError::UnsupportedFilter(32000).to_string();

crates/clawhdf5-format/src/filters.rs

@@ -434,6 +434,13 @@ pub(crate) static BUILTIN_FILTERS: &[BuiltinFilter] = &[
         decode: crate::filters_bitshuffle::bitshuffle_decode,
         encode: Some(crate::filters_bitshuffle::bitshuffle_encode),
     },
+    #[cfg(feature = "zfp")]
+    BuiltinFilter {
+        id: crate::filter_pipeline::FILTER_ZFP,
+        name: "zfp",
+        decode: crate::filters_zfp::zfp_decode,
+        encode: None,
+    },
     #[cfg(feature = "zstd")]
     BuiltinFilter {
         id: FILTER_ZSTD,

crates/clawhdf5-format/src/lib.rs

@@ -95,6 +95,8 @@ mod filters_bzip2;
 #[cfg(feature = "lzf")]
 pub mod filters_lzf;
 mod filters_szip;
+#[cfg(feature = "zfp")]
+pub mod filters_zfp;
 pub mod fixed_array;
 pub mod float16;
 pub mod fractal_heap;

crates/clawhdf5-format/tests/zfp_alloc_bounds.rs

@@ -0,0 +1,274 @@
+//! Crafted ZFP filter parameters and streams cannot make the decoder panic
+//! or allocate out of proportion to the chunk it decodes.
+//!
+//! The field size comes from the filter's `cd_values` (up to 2^48 values),
+//! not from the chunk: the decoder allocates the output only when it matches
+//! the chunk's size (or, when that is unknown, is within the 256 MiB
+//! ceiling), and only when the stream is long enough to hold a bit per
+//! block. Peak heap use is measured with a counting global allocator; the
+//! tests share it, so each holds `SERIAL` for its whole run.
+#![cfg(feature = "zfp")]
+
+use std::alloc::{GlobalAlloc, Layout, System};
+use std::sync::Mutex;
+use std::sync::atomic::{AtomicUsize, Ordering};
+
+use clawhdf5_format::filters_zfp::zfp_decompress;
+
+struct Counting;
+
+static CURRENT: AtomicUsize = AtomicUsize::new(0);
+static PEAK: AtomicUsize = AtomicUsize::new(0);
+static SERIAL: Mutex<()> = Mutex::new(());
+
+unsafe impl GlobalAlloc for Counting {
+    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
+        let p = unsafe { System.alloc(layout) };
+        if !p.is_null() {
+            let now = CURRENT.fetch_add(layout.size(), Ordering::Relaxed) + layout.size();
+            PEAK.fetch_max(now, Ordering::Relaxed);
+        }
+        p
+    }
+
+    unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
+        let p = unsafe { System.alloc_zeroed(layout) };
+        if !p.is_null() {
+            let now = CURRENT.fetch_add(layout.size(), Ordering::Relaxed) + layout.size();
+            PEAK.fetch_max(now, Ordering::Relaxed);
+        }
+        p
+    }
+
+    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
+        unsafe { System.dealloc(ptr, layout) };
+        CURRENT.fetch_sub(layout.size(), Ordering::Relaxed);
+    }
+}
+
+#[global_allocator]
+static ALLOC: Counting = Counting;
+
+/// Bytes allocated at the peak of `f`, above what was live when it started.
+fn peak_during<T>(f: impl FnOnce() -> T) -> (T, usize) {
+    let base = CURRENT.load(Ordering::Relaxed);
+    PEAK.store(base, Ordering::Relaxed);
+    let out = f();
+    (out, PEAK.load(Ordering::Relaxed).saturating_sub(base))
+}
+
+fn lock() -> std::sync::MutexGuard<'static, ()> {
+    SERIAL.lock().unwrap_or_else(|e| e.into_inner())
+}
+
+/// What decoding may hold: the output (at most the limit, and at most a
+/// 4-D block of doubles, 2 KiB, per bit of input), and a little more.
+fn bound(max_output: usize, input: &[u8]) -> usize {
+    let limit = if max_output == 0 {
+        256 << 20
+    } else {
+        max_output
+    };
+    limit.min(input.len() * 8 * 2048) + 4096
+}
+
+/// An LSB-first bit writer.
+#[derive(Default)]
+struct Bits {
+    v: Vec<u8>,
+    n: usize,
+}
+
+impl Bits {
+    fn put(&mut self, x: u64, bits: usize) {
+        for i in 0..bits {
+            if self.n.is_multiple_of(8) {
+                self.v.push(0);
+            }
+            if (x >> i) & 1 == 1 {
+                *self.v.last_mut().unwrap() |= 1 << (self.n % 8);
+            }
+            self.n += 1;
+        }
+    }
+}
+
+/// H5Z-ZFP `cd_values`: a version word and a zfp header for a field of
+/// `ztype` (0 int32, 1 int64, 2 float, 3 double) and sizes `n` (fastest
+/// first), with `mode` (12 bits, or 64 when `long`).
+fn cd_values(ztype: u64, n: &[u64], mode: u64, long: bool) -> Vec<u32> {
+    let mut b = Bits::default();
+    for c in b"zfp" {
+        b.put(*c as u64, 8);
+    }
+    b.put(5, 8);
+    let dims = n.len();
+    let mut meta = 0u64;
+    let bits = [48, 24, 16, 12][dims - 1];
+    for &v in n.iter().rev() {
+        meta = (meta << bits) + v - 1;
+    }
+    meta = (meta << 2) + dims as u64 - 1;
+    meta = (meta << 2) + ztype;
+    b.put(meta, 52);
+    b.put(mode, if long { 64 } else { 12 });
+    b.v.resize(b.v.len().div_ceil(4) * 4, 0);
+    let mut cd = vec![0x1001_1111u32];
+    cd.extend(
+        b.v.chunks(4)
+            .map(|w| u32::from_le_bytes(w.try_into().unwrap())),
+    );
+    cd
+}
+
+fn elem(ztype: u64) -> usize {
+    if ztype & 1 == 0 { 4 } else { 8 }
+}
+
+/// A 1-D field of 2^32 doubles (32 GiB) in a 1-byte chunk: refused for
+/// its size, with or without the chunk size known, before anything is
+/// allocated.
+#[test]
+fn huge_fields_are_refused_without_allocating() {
+    let _g = lock();
+    for (n, ztype) in [
+        (vec![1u64 << 32], 3),
+        (vec![1 << 24, 1 << 24], 3),
+        (vec![4096; 4], 1),
+    ] {
+        let cd = cd_values(ztype, &n, 2176, false);
+        for max_output in [0usize, 1 << 20] {
+            let (r, peak) = peak_during(|| zfp_decompress(&[0xff], &cd, max_output));
+            assert!(r.is_err(), "{n:?}: decoded {:?} bytes", r.map(|v| v.len()));
+            assert!(peak < 4096, "{n:?}: peak {peak} bytes");
+        }
+    }
+}
+
+/// A field of the chunk's size whose stream is too short for its blocks
+/// is refused before the output is allocated.
+#[test]
+fn short_streams_are_refused_before_allocating() {
+    let _g = lock();
+    let n = [1u64 << 18];
+    let cd = cd_values(2, &n, 2176, false);
+    let size = (1 << 18) * 4;
+    let (r, peak) = peak_during(|| zfp_decompress(&[0u8; 100], &cd, size));
+    assert!(r.is_err());
+    assert!(peak < 4096, "peak {peak} bytes");
+    // A stream with a bit per block: all-zero blocks, which decode.
+    let input = vec![0u8; (1 << 16) / 8];
+    let out = zfp_decompress(&input, &cd, size).unwrap();
+    assert_eq!(out, vec![0u8; size]);
+}
+
+/// xorshift64*: deterministic, so a failure reproduces.
+struct Rng(u64);
+
+impl Rng {
+    fn next(&mut self) -> u64 {
+        let mut x = self.0;
+        x ^= x >> 12;
+        x ^= x << 25;
+        x ^= x >> 27;
+        self.0 = x;
+        x.wrapping_mul(0x2545_F491_4F6C_DD1D)
+    }
+
+    fn below(&mut self, n: u64) -> u64 {
+        self.next() % n.max(1)
+    }
+}
+
+/// A mode word: one of the four short forms, or the 64-bit expert form
+/// with parameters at and past their edges (maxbits below a float block's
+/// exponent, minbits past maxbits, precision 0, minexp at the reversible
+/// boundary).
+fn mode(rng: &mut Rng) -> (u64, bool) {
+    match rng.below(6) {
+        0 => (rng.below(2048), false),
+        1 => (2048 + rng.below(128), false),
+        2 => (2176, false),
+        3 => (2177 + rng.below(4094 - 2177 + 1), false),
+        _ => {
+            fn pick(rng: &mut Rng, v: [u64; 6]) -> u64 {
+                v[rng.below(6) as usize]
+            }
+            let r = [rng.below(2000), rng.below(0x8000), rng.below(40)];
+            let minbits = pick(rng, [0, 1, 11, r[0], r[1], 1]);
+            let r = [rng.below(0x8000), rng.below(40)];
+            let maxbits = pick(rng, [minbits, minbits + r[1], 7, 11, r[0], 0x7fff]);
+            let maxprec = rng.below(0x80);
+            let r = [rng.below(0x8000), rng.below(400)];
+            let minexp = pick(
+                rng,
+                [
+                    r[0],
+                    16495 - 1074,
+                    16495 - 1075,
+                    16495 + r[1] - 200,
+                    16495 - 1074,
+                    r[0],
+                ],
+            );
+            let m = ((((minexp << 7) + maxprec) << 15) + maxbits) << 15;
+            ((m + minbits) << 12 | 0xfff, true)
+        }
+    }
+}
+
+/// Random fields, modes and streams (random bytes, runs of ones, mostly
+/// zeros; of every length), decoded with the chunk size known and not,
+/// and with header words that are random too.
+#[test]
+fn fuzzed_headers_and_streams_stay_within_the_allocation_bound() {
+    let _g = lock();
+    let mut rng = Rng(0x2f9);
+    let mut decoded = 0;
+    for i in 0..20_000 {
+        let ztype = rng.below(4);
+        let dims = 1 + rng.below(4) as usize;
+        let max = [300, 40, 14, 8][dims - 1];
+        let n: Vec<u64> = (0..dims).map(|_| 1 + rng.below(max)).collect();
+        let (m, long) = mode(&mut rng);
+        let mut cd = cd_values(ztype, &n, m, long);
+        if rng.below(10) == 0 {
+            let at = rng.below(cd.len() as u64) as usize;
+            cd[at] ^= 1 << rng.below(32);
+        }
+        if rng.below(20) == 0 {
+            cd.truncate(rng.below(cd.len() as u64 + 1) as usize);
+        }
+        let len = match rng.below(4) {
+            0 => rng.below(8),
+            1 => rng.below(300),
+            _ => rng.below(20_000),
+        } as usize;
+        let input: Vec<u8> = match rng.below(3) {
+            0 => (0..len).map(|_| rng.next() as u8).collect(),
+            1 => (0..len)
+                .map(|_| [0, 0xff, rng.next() as u8][rng.below(3) as usize])
+                .collect(),
+            _ => (0..len)
+                .map(|_| [0, 0, 0, 1, 0x80, rng.next() as u8][rng.below(6) as usize])
+                .collect(),
+        };
+        let size = n.iter().product::<u64>() as usize * elem(ztype);
+        let max_output = if rng.below(4) == 0 { 0 } else { size };
+        let (r, peak) = peak_during(|| zfp_decompress(&input, &cd, max_output));
+        if let Ok(out) = &r {
+            decoded += 1;
+            if max_output != 0 {
+                assert_eq!(out.len(), max_output, "iteration {i}");
+            }
+        }
+        assert!(
+            peak <= bound(max_output, &input),
+            "iteration {i}: peak {peak} bytes for {n:?} from {} bytes ({:?})",
+            input.len(),
+            r.map(|v| v.len())
+        );
+    }
+    // Most inputs are streams zfp decodes without running out.
+    assert!(decoded > 5_000, "only {decoded} decoded");
+}

crates/clawhdf5/Cargo.toml

@@ -53,8 +53,10 @@ bitshuffle = ["clawhdf5-format/bitshuffle"]
 bzip2 = ["clawhdf5-format/bzip2"]
 blosc = ["clawhdf5-format/blosc"]
 blosc2 = ["clawhdf5-format/blosc2"]
+# ZFP (32013), read-only.
+zfp = ["clawhdf5-format/zfp"]
 # Every plugin filter.
-plugin-filters = ["lzf", "bitshuffle", "bzip2", "blosc", "blosc2"]
+plugin-filters = ["lzf", "bitshuffle", "bzip2", "blosc", "blosc2", "zfp"]
 # Dataset::verify_provenance() — recompute a dataset's SHA-256 and compare
 # against its stored _provenance_sha256 attribute. On by default, matching
 # clawhdf5-format's own default-on `provenance` feature.

crates/clawhdf5/tests/plugin_filters_interop.rs

@@ -495,16 +495,15 @@ fn lzf_written_by_clawhdf5_reads_in_h5py() {
     });
 }
 
-/// ZFP is not implemented, and Blosc2 is not in a build without the
+/// Blosc2 and ZFP are not in a build without their features: reading them
-/// `blosc2` feature: reading them must be a clear error naming the filter,
+/// must be a clear error naming the filter, never data.
-/// never data.
 #[test]
-fn unimplemented_filters_are_a_clear_error() {
+fn filters_left_out_of_the_build_are_a_clear_error() {
     if !have_python("h5py, hdf5plugin") {
         return;
     }
     let dir = tempfile::tempdir().unwrap();
-    let path = dir.path().join("unimplemented.h5");
+    let path = dir.path().join("left_out.h5");
     run_python(
         r#"
 import sys
@@ -517,7 +516,10 @@ with h5py.File(sys.argv[1], 'w') as f:
         &[path.to_str().unwrap()],
     );
     let file = File::open(&path).unwrap();
-    let mut missing = vec![("zfp", 32013u16, "ZFP")];
+    let mut missing = Vec::new();
+    if !cfg!(feature = "zfp") {
+        missing.push(("zfp", 32013u16, "ZFP"));
+    }
     if !cfg!(feature = "blosc2") {
         missing.push(("blosc2", 32026, "Blosc2"));
     }
@@ -526,7 +528,7 @@ with h5py.File(sys.argv[1], 'w') as f:
             .dataset(name)
             .unwrap()
             .read_selection(&Selection::All)
-            .expect_err("an unimplemented filter must not read");
+            .expect_err("a filter left out of the build must not read");
         let msg = err.to_string();
         assert!(
             msg.contains(&id.to_string()) && msg.contains(label),

crates/clawhdf5/tests/zfp_interop.rs

@@ -0,0 +1,277 @@
+//! ZFP (H5Z-ZFP, filter 32013) against libhdf5 + libzfp.
+//!
+//! h5py with hdf5plugin (H5Z-ZFP 1.1.1, zfp 1.0.1) writes datasets in every
+//! ZFP mode (fixed rate, precision and accuracy, reversible, expert) for
+//! each type ZFP supports (int32, int64, float, double), in 1 to 4
+//! dimensions, with chunks that are partial at the dataset's edges, blocks
+//! that are partial at the chunks' edges, and chunks with unit dimensions
+//! (a lower-dimensional ZFP field). Next to each it stores what h5py reads
+//! back, unfiltered, and clawhdf5 must read the ZFP dataset bit for bit
+//! equal to that: the decoder is deterministic, so lossy modes have one
+//! right answer.
+//!
+//! Skipped when python3 with h5py and hdf5plugin is unavailable, unless
+//! `CLAWHDF5_REQUIRE_INTEROP=1`.
+#![cfg(feature = "zfp")]
+
+use std::collections::BTreeMap;
+use std::process::Command;
+
+use clawhdf5::File;
+use clawhdf5_format::selection::Selection;
+
+fn python() -> String {
+    std::env::var("CLAWHDF5_PYTHON").unwrap_or_else(|_| "python3".to_string())
+}
+
+fn have_python() -> bool {
+    let ok = Command::new(python())
+        .args(["-c", "import h5py, hdf5plugin"])
+        .output()
+        .map(|o| o.status.success())
+        .unwrap_or(false);
+    if ok {
+        return true;
+    }
+    assert!(
+        !std::env::var("CLAWHDF5_REQUIRE_INTEROP").is_ok_and(|v| v == "1"),
+        "CLAWHDF5_REQUIRE_INTEROP=1 but python3 with h5py and hdf5plugin is not available"
+    );
+    eprintln!("SKIP: python3 with h5py and hdf5plugin not available");
+    false
+}
+
+fn run_python(script: &str, args: &[&str]) -> String {
+    let output = Command::new(python())
+        .arg("-c")
+        .arg(script)
+        .args(args)
+        .output()
+        .expect("failed to run python");
+    assert!(
+        output.status.success(),
+        "Python script failed:\nSTDOUT: {}\nSTDERR: {}",
+        String::from_utf8_lossy(&output.stdout),
+        String::from_utf8_lossy(&output.stderr)
+    );
+    String::from_utf8_lossy(&output.stdout).trim().to_string()
+}
+
+/// Writes `f{i}` (ZFP) and `r{i}` (h5py's reading of `f{i}`, unfiltered)
+/// for every mode x dtype x shape x data kind H5Z-ZFP accepts; each `f{i}`
+/// has a `case` attribute. Prints the number of pairs.
+const GENERATE: &str = r#"
+import sys
+import numpy as np, h5py, hdf5plugin
+path = sys.argv[1]
+MODES = [
+    ('rate 2.5', dict(rate=2.5)),
+    ('rate 8', dict(rate=8)),
+    ('rate 16', dict(rate=16)),
+    ('rate 31', dict(rate=31)),
+    ('rate 64', dict(rate=64)),
+    ('precision 6', dict(precision=6)),
+    ('precision 20', dict(precision=20)),
+    ('precision 64', dict(precision=64)),
+    ('accuracy 0.5', dict(accuracy=0.5)),
+    ('accuracy 1e-3', dict(accuracy=1e-3)),
+    ('accuracy 1e-12', dict(accuracy=1e-12)),
+    ('reversible', dict(reversible=True)),
+    ('expert', dict(minbits=24, maxbits=600, maxprec=30, minexp=-20)),
+    # maxbits past ZFP_MAX_BITS: H5Z-ZFP stores it as fixed precision.
+    ('expert maxbits 20000', dict(minbits=1, maxbits=20000, maxprec=40, minexp=-1074)),
+    # A budget of 10 bits a block: one bit left after a float's exponent.
+    # (Doubles are left out: 12 bits of exponent and flag overrun it, and
+    # libzfp's encoder then writes past its buffer.)
+    ('expert maxbits 10', dict(minbits=5, maxbits=10, maxprec=64, minexp=-100)),
+    ('expert minbits 900', dict(minbits=900, maxbits=4000, maxprec=64, minexp=-60)),
+]
+DTYPES = ['<f4', '<f8', '<i4', '<i8']
+SHAPES = [
+    ((37,), (16,)),
+    ((13, 22), (5, 8)),
+    ((9, 7, 6), (4, 5, 6)),
+    ((5, 6, 7, 3), (3, 5, 6, 3)),
+    ((4, 1, 30), (2, 1, 30)),
+    ((3, 10, 1, 9), (2, 10, 1, 9)),
+    ((64, 64), (64, 64)),
+]
+KINDS = ['smooth', 'noise', 'wide', 'zeros', 'special']
+rng = np.random.default_rng(13)
+
+def data(dt, shape, kind):
+    n = int(np.prod(shape))
+    d = np.dtype(dt)
+    idx = np.arange(n, dtype=np.float64)
+    if d.kind == 'f':
+        if kind == 'smooth':
+            v = 100 * np.sin(idx / 5.0) + idx / 3.0
+        elif kind == 'noise':
+            v = rng.normal(size=n) * 10.0 ** rng.uniform(-3, 3, n)
+        elif kind == 'wide':
+            lo, hi = (-150, 120) if d.itemsize == 4 else (-1075, 1000)
+            v = rng.choice([-1.0, 1.0], n) * np.exp2(rng.integers(lo, hi, n).astype(np.float64))
+            v[rng.random(n) < 0.1] = 0.0
+        elif kind == 'zeros':
+            v = np.zeros(n)
+            v[: n // 3] = 0.0
+        else:
+            v = rng.normal(size=n) * 1e3
+            v[rng.random(n) < 0.05] = np.inf
+            v[rng.random(n) < 0.05] = -np.inf
+            v[rng.random(n) < 0.05] = np.nan
+            v[rng.random(n) < 0.05] = -0.0
+        with np.errstate(over='ignore'):
+            return v.astype(dt).reshape(shape)
+    info = np.iinfo(d)
+    if kind == 'smooth':
+        v = ((idx * 7) % 1000 - 500).astype(np.int64)
+    elif kind == 'noise':
+        v = rng.integers(info.min, info.max, n, dtype=np.int64, endpoint=True)
+    elif kind == 'wide':
+        bits = rng.integers(0, d.itemsize * 8 - 2, n)
+        v = rng.integers(-(2 ** 20), 2 ** 20, n) << np.minimum(bits, d.itemsize * 8 - 22)
+    elif kind == 'zeros':
+        v = np.zeros(n, dtype=np.int64)
+    else:
+        v = rng.choice([info.min, info.max, 0, -1, 1], n)
+    return v.astype(dt).reshape(shape)
+
+# Written first and read back after the file is closed: h5py reads a chunk
+# still in libhdf5's chunk cache without decoding it.
+i = 0
+with h5py.File(path, 'w') as f:
+    for label, kw in MODES:
+        for dt in DTYPES:
+            if label == 'expert maxbits 10' and dt == '<f8':
+                continue
+            for shape, chunks in SHAPES:
+                for kind in KINDS:
+                    v = data(dt, shape, kind)
+                    name = f'f{i}'
+                    try:
+                        ds = f.create_dataset(name, data=v, chunks=chunks, **hdf5plugin.Zfp(**kw))
+                    except Exception:
+                        continue
+                    ds.attrs['case'] = f'{label}|{dt}|{shape}|{chunks}|{kind}'
+                    i += 1
+with h5py.File(path, 'a') as f:
+    for k in range(i):
+        f.create_dataset(f'r{k}', data=f[f'f{k}'][()])
+print(i)
+"#;
+
+#[test]
+fn every_mode_reads_bit_exact() {
+    if !have_python() {
+        return;
+    }
+    let dir = tempfile::tempdir().unwrap();
+    let path = dir.path().join("zfp.h5");
+    let n: usize = run_python(GENERATE, &[path.to_str().unwrap()])
+        .parse()
+        .unwrap();
+    let file = File::open(&path).unwrap();
+    // mode -> dtypes read
+    let mut seen: BTreeMap<String, Vec<String>> = BTreeMap::new();
+    let mut failures = Vec::new();
+    for i in 0..n {
+        let ds = file.dataset(&format!("f{i}")).unwrap();
+        let case = match ds.attrs().unwrap().get("case") {
+            Some(clawhdf5::AttrValue::String(s)) => s.clone(),
+            other => panic!("f{i}: case attribute {other:?}"),
+        };
+        let want = file
+            .dataset(&format!("r{i}"))
+            .unwrap()
+            .read_selection(&Selection::All)
+            .unwrap();
+        match ds.read_selection(&Selection::All) {
+            Ok(got) if got == want => {
+                let mut parts = case.split('|');
+                let mode = parts.next().unwrap().to_string();
+                let dt = parts.next().unwrap().to_string();
+                let e = seen.entry(mode).or_default();
+                if !e.contains(&dt) {
+                    e.push(dt);
+                }
+            }
+            Ok(got) => {
+                let first = got.iter().zip(&want).position(|(a, b)| a != b);
+                failures.push(format!("f{i} {case}: differs from byte {first:?}"));
+            }
+            Err(e) => failures.push(format!("f{i} {case}: {e}")),
+        }
+    }
+    assert!(
+        failures.is_empty(),
+        "{} of {n} datasets:\n{}",
+        failures.len(),
+        failures.join("\n")
+    );
+    // Every mode was exercised on every type H5Z-ZFP accepts it for.
+    for (mode, dts) in &seen {
+        assert!(dts.len() >= 2, "{mode}: only {dts:?}");
+    }
+    assert_eq!(seen.len(), 16, "{:?}", seen.keys());
+    eprintln!("{n} ZFP datasets bit-exact; modes and types: {seen:?}");
+}
+
+/// A header written on a big-endian machine: H5Z-ZFP finds the magic only
+/// after byte-swapping the `cd_values`, and then byte-swaps the decoded
+/// values, since the dataset's datatype is big-endian there. The file is
+/// made by swapping the header words of a little-endian one in place, so
+/// the datatype stays little-endian and libhdf5 reads the values swapped;
+/// clawhdf5 must read the same bytes.
+#[test]
+fn big_endian_header_swaps_the_values() {
+    if !have_python() {
+        return;
+    }
+    let dir = tempfile::tempdir().unwrap();
+    let path = dir.path().join("zfp_be.h5");
+    let n: usize = run_python(
+        r#"
+import sys, struct
+import numpy as np, h5py, hdf5plugin
+path = sys.argv[1]
+cases = [('<f4', dict(rate=12)), ('<f8', dict(precision=30)), ('<i4', dict(reversible=True)),
+         ('<i8', dict(accuracy=4))]
+cds = []
+with h5py.File(path, 'w') as f:
+    for k, (dt, kw) in enumerate(cases):
+        v = (np.arange(150) * 13 % 97 - 40).reshape(10, 15).astype(dt)
+        ds = f.create_dataset(f'f{k}', data=v, chunks=(4, 8), **hdf5plugin.Zfp(**kw))
+        cds.append(ds.id.get_create_plist().get_filter(0)[2])
+raw = bytearray(open(path, 'rb').read())
+for cd in cds:
+    le = struct.pack(f'<{len(cd)}I', *cd)
+    swapped = le[:4] + struct.pack(f'>{len(cd) - 1}I', *cd[1:])
+    at = raw.find(le)
+    assert at > 0 and raw.find(le, at + 1) < 0, 'cd_values not found once'
+    raw[at:at + len(le)] = swapped
+open(path, 'wb').write(raw)
+with h5py.File(path, 'a') as f:
+    for k in range(len(cases)):
+        f.create_dataset(f'r{k}', data=f[f'f{k}'][()])
+        f.create_dataset(f'o{k}', data=f[f'f{k}'][()].byteswap())
+print(len(cases))
+"#,
+        &[path.to_str().unwrap()],
+    )
+    .parse()
+    .unwrap();
+    let file = File::open(&path).unwrap();
+    for k in 0..n {
+        let read = |name: String| {
+            file.dataset(&name)
+                .unwrap()
+                .read_selection(&Selection::All)
+                .unwrap_or_else(|e| panic!("{name}: {e}"))
+        };
+        let got = read(format!("f{k}"));
+        assert!(got == read(format!("r{k}")), "f{k}: differs from h5py");
+        // Byte-swapped back, the values are the right ones.
+        assert!(got != read(format!("o{k}")), "f{k}: not swapped");
+    }
+}

docs/known-issues.md

@@ -293,10 +293,14 @@ fill-value item that did is fixed).
   697 ok, tank, `conformance/run.sh --no-fetch`). Blosc2 frames using
   dictionaries, lazy chunks, variable-length blocks, user-defined codecs or
   registered filters (e.g. bytedelta) are refused with an error.
-  **Still open:** ZFP (32013) fails with an `UnsupportedFilter` error that
+  **Fixed 2026-09-26** for ZFP (32013, `zfp` feature, also in
-  names the filter, and can be plugged in with
+  `plugin-filters`), read only: every H5Z-ZFP mode and type, bit-exact
-  `filter_registry::register_filter` (32023, Granular BitRound, too, since
+  against h5py + hdf5plugin 7.1 (`crates/clawhdf5/tests/zfp_interop.rs`);
-  2026-09-26 even with the `pcodec` feature). clawhdf5 cannot write Blosc2.
+  h5ex_d_zfp now reads (conformance 600 of 697 ok, tank,
+  `conformance/run.sh --no-fetch`). **Still open:** clawhdf5 cannot write
+  Blosc2 or ZFP. Other filters (32023, Granular BitRound, too, since
+  2026-09-26 even with the `pcodec` feature) can be plugged in with
+  `filter_registry::register_filter`.
 - **Wrong data: a chunk whose filters decode to fewer bytes than the chunk
   read with zeros for the missing bytes** (any filter; found reviewing the plugin
   filters). **Fixed 2026-09-26:** it is an error naming the chunk. A corrupt

scripts/ci-test.sh

@@ -67,7 +67,7 @@ run_step "cargo clippy --all-targets" cargo clippy \
 
 # 3. Clippy over clawhdf5-format's optional features, which the default
 #    workspace build never compiles (szip is left out: it needs libaec).
-#    plugin-filters = bitshuffle, bzip2, blosc, blosc2 (and the default-on lzf).
+#    plugin-filters = bitshuffle, bzip2, blosc, blosc2, zfp (and the default-on lzf).
 run_step "cargo clippy (format feature matrix)" cargo clippy \
     -p clawhdf5-format \
     --all-targets \
@@ -78,7 +78,7 @@ run_step "cargo clippy (format feature matrix)" cargo clippy \
 # dependencies (bitshuffle and blosc share code).
 plugin_filters_alone() {
     local f
-    for f in bitshuffle bzip2 blosc blosc2; do
+    for f in bitshuffle bzip2 blosc blosc2 zfp; do
         echo "--- $f"
         cargo clippy -p clawhdf5-format --all-targets --features "$f" -- -D warnings || return 1
     done
@@ -210,9 +210,10 @@ if "$PYTHON" -c "import h5py" >/dev/null 2>&1 || [ "${CLAWHDF5_REQUIRE_INTEROP:-
     # libhdf5's registered plugins) compile and run too.
     run_step "h5py interop (format, ignored tests)" cargo test \
         -p clawhdf5-format --features lz4,zstd --test writer_h5py_tests -- --include-ignored
-    # LZF, bitshuffle, bzip2 and Blosc both ways against h5py + hdf5plugin.
+    # LZF, bitshuffle, bzip2 and Blosc both ways against h5py + hdf5plugin;
+    # Blosc2 and ZFP (read-only) against what h5py reads.
     run_step "h5py interop (plugin filters)" cargo test \
-        -p clawhdf5 --features plugin-filters --test plugin_filters_interop
+        -p clawhdf5 --features plugin-filters --test plugin_filters_interop --test zfp_interop
 else
     echo ""
     echo "==> [h5py interop] SKIPPED: no h5py in $PYTHON"