quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

facade: zfp feature; ZFP read bit-exact against libhdf5 + libzfp

Browse Source 
The facade forwards `zfp` and adds it to `plugin-filters`.
tests/zfp_interop.rs: h5py + hdf5plugin (H5Z-ZFP 1.1.1, zfp 1.0.1)
write 2205 datasets over 16 modes (rate, precision, accuracy,
reversible, expert settings at their edges) x int32/int64/float/double
x 1-4-D shapes with partial edge chunks, partial blocks and unit chunk
dimensions x smooth/noisy/wide-range/zero/inf-NaN data; clawhdf5 must
read each byte for byte as h5py does (read back after closing the file:
h5py returns a chunk still in libhdf5's cache without decoding it). A
second test swaps a file's header words to what a big-endian writer
stores and checks the byte-swapped values match h5py's.

The left-out-filter test now expects ZFP only in builds without it; CI
lints `zfp` alone and runs the new test with the plugin-filter step.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 12:57:02 -05:00
co-authored by Claude Opus 5.5
parent 23a4784e72
commit d97b3d703a
4 changed files with 294 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/ci-test.sh
+5 -4
View File
@@ -67,7 +67,7 @@ run_step "cargo clippy --all-targets" cargo clippy \
# 3. Clippy over clawhdf5-format's optional features, which the default
# workspace build never compiles (szip is left out: it needs libaec).
# plugin-filters = bitshuffle, bzip2, blosc, blosc2 (and the default-on lzf).
# plugin-filters = bitshuffle, bzip2, blosc, blosc2, zfp (and the default-on lzf).
run_step "cargo clippy (format feature matrix)" cargo clippy \
-p clawhdf5-format \
--all-targets \
@@ -78,7 +78,7 @@ run_step "cargo clippy (format feature matrix)" cargo clippy \
# dependencies (bitshuffle and blosc share code).
plugin_filters_alone() {
local f
for f in bitshuffle bzip2 blosc blosc2; do
for f in bitshuffle bzip2 blosc blosc2 zfp; do
echo "--- $f"
cargo clippy -p clawhdf5-format --all-targets --features "$f" -- -D warnings || return 1
done
@@ -208,9 +208,10 @@ if "$PYTHON" -c "import h5py" >/dev/null 2>&1 || [ "${CLAWHDF5_REQUIRE_INTEROP:-
# libhdf5's registered plugins) compile and run too.
run_step "h5py interop (format, ignored tests)" cargo test \
-p clawhdf5-format --features lz4,zstd --test writer_h5py_tests -- --include-ignored
# LZF, bitshuffle, bzip2 and Blosc both ways against h5py + hdf5plugin.
# LZF, bitshuffle, bzip2 and Blosc both ways against h5py + hdf5plugin;
# Blosc2 and ZFP (read-only) against what h5py reads.
run_step "h5py interop (plugin filters)" cargo test \
-p clawhdf5 --features plugin-filters --test plugin_filters_interop
-p clawhdf5 --features plugin-filters --test plugin_filters_interop --test zfp_interop
else
echo ""
echo "==> [h5py interop] SKIPPED: no h5py in $PYTHON"