quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs: ZFP reads (changelog, README feature table, known issues)

Browse Source 
ZFP (32013) was the one plugin filter still listed as UnsupportedFilter.
Conformance on tank, `conformance/run.sh --no-fetch` (2026-09-26): 600 of
697 files ok (baseline 599); h5ex_d_zfp.h5 is newly ok.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 13:08:47 -05:00
co-authored by Claude Opus 5.5
parent d97b3d703a
commit 2b7065998a
4 changed files with 38 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+24
View File
@@ -2,6 +2,30 @@
## Unreleased
### ZFP (2026-09-26)
- **ZFP (filter 32013, H5Z-ZFP) reads, in pure Rust.** It was the last
filter in the conformance corpus that failed with `UnsupportedFilter`.
New feature `zfp` (`clawhdf5-format` and `clawhdf5`, included in
`plugin-filters`, no dependencies) ports the zfp 1.0.1 decoder and the
decompression half of H5Z-ZFP 1.1.1: every mode (fixed rate, fixed
precision, fixed accuracy, reversible, expert), int32, int64, float and
double, 1-4-D fields with partial blocks, and headers written by
big-endian machines (values byte-swapped as H5Z-ZFP does). Read only:
there is no ZFP encoder. The decoder is deterministic, so lossy modes have
one right answer, and clawhdf5 returns exactly libzfp's values.
- Tests: `crates/clawhdf5/tests/zfp_interop.rs` has h5py + hdf5plugin 7.1
(H5Z-ZFP 1.1.1, zfp 1.0.1) write 2205 datasets over 16 mode settings
(including expert parameters at their edges) x the four types x 1-4-D
shapes with partial edge chunks, partial blocks and unit chunk
dimensions x smooth, noisy, wide-range, zero and inf/NaN data; each must
read byte for byte as h5py reads it. `tests/zfp_alloc_bounds.rs` fuzzes
headers and streams under a counting allocator: no panics, and the output
is allocated only when it matches the chunk's size and the stream holds at
least a bit per block. A stream that ends before the decoder is done is an
error (libzfp reads past its buffer).
- Conformance on tank (2026-09-26, `conformance/run.sh --no-fetch`): 600 of
697 files ok (599 before); `h5ex_d_zfp.h5` now reads.
### Chunked full reads (2026-09-26)
- **Chunks are decoded straight into the output, into reused buffers.** A
full read of a chunked dataset faulted in about three times its size in