Merge branch 'feat/p3-zfp' into feat/p3-range-zfp-edit

# Conflicts:
#	CHANGELOG.md
#	crates/clawhdf5-format/Cargo.toml
This commit is contained in:
osobh
2026-09-26 14:52:22 -05:00
15 changed files with 1709 additions and 27 deletions
+24
View File
@@ -149,6 +149,30 @@
converted parser falling back to the whole file fails it. Milestones M2
and M3 extend it.
### ZFP (2026-09-26)
- **ZFP (filter 32013, H5Z-ZFP) reads, in pure Rust.** It was the last
filter in the conformance corpus that failed with `UnsupportedFilter`.
New feature `zfp` (`clawhdf5-format` and `clawhdf5`, included in
`plugin-filters`, no dependencies) ports the zfp 1.0.1 decoder and the
decompression half of H5Z-ZFP 1.1.1: every mode (fixed rate, fixed
precision, fixed accuracy, reversible, expert), int32, int64, float and
double, 1-4-D fields with partial blocks, and headers written by
big-endian machines (values byte-swapped as H5Z-ZFP does). Read only:
there is no ZFP encoder. The decoder is deterministic, so lossy modes have
one right answer, and clawhdf5 returns exactly libzfp's values.
- Tests: `crates/clawhdf5/tests/zfp_interop.rs` has h5py + hdf5plugin 7.1
(H5Z-ZFP 1.1.1, zfp 1.0.1) write 2205 datasets over 16 mode settings
(including expert parameters at their edges) x the four types x 1-4-D
shapes with partial edge chunks, partial blocks and unit chunk
dimensions x smooth, noisy, wide-range, zero and inf/NaN data; each must
read byte for byte as h5py reads it. `tests/zfp_alloc_bounds.rs` fuzzes
headers and streams under a counting allocator: no panics, and the output
is allocated only when it matches the chunk's size and the stream holds at
least a bit per block. A stream that ends before the decoder is done is an
error (libzfp reads past its buffer).
- Conformance on tank (2026-09-26, `conformance/run.sh --no-fetch`): 600 of
697 files ok (599 before); `h5ex_d_zfp.h5` now reads.
### Chunked full reads (2026-09-26)
- **Chunks are decoded straight into the output, into reused buffers.** A
full read of a chunked dataset faulted in about three times its size in