ci: lint all targets, run interop suites for real, compile benches

- clippy --all-targets plus a clawhdf5-format feature matrix (parallel, lz4,
  zstd, pcodec, fast-checksum); fix the accumulated lint backlog in test,
  bench and feature-gated code (no behaviour changes).
- Install python3 + h5py/numpy/netCDF4/xarray in the CI container and set
  CLAWHDF5_REQUIRE_INTEROP=1, which makes a missing interop dependency a test
  failure. Every h5py/netCDF4 interop test used to skip silently in CI. Run
  the #[ignore]d writer_h5py_tests suite explicitly.
- cargo bench --no-run so benches can't rot; fix bench.rs and memory_bench.rs,
  which no longer compiled against the current strategy/consolidation APIs.
- Optional fuzz smoke run via CLAWHDF5_FUZZ_SECONDS.
- CHANGELOG and docs/known-issues.md updated.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
osobh
2026-09-19 05:36:22 -07:00
co-authored by Claude Fable 5.1
parent 706189c3ef
commit bbe1baa208
30 changed files with 588 additions and 405 deletions
+34 -7
View File
@@ -72,11 +72,38 @@ generated yet; one written with the C API (`H5T_STD_REF`) is needed.
## `clawhdf5-gpu` `gpu_tests` can hang under the default parallel test runner
**Status:** open. Observed 2026-09-18 (RTX 5060 Ti, Linux).
**Status:** fixed 2026-09-19.
**Summary:** during `cargo test --workspace`, the `gpu_tests` binary sat idle
(~1% CPU) for 25+ minutes and had to be killed. Run single-threaded it passes
in seconds (20/20): `cargo test -p clawhdf5-gpu --test gpu_tests -- --test-threads=1`.
Suspected cause: several tests creating wgpu devices concurrently (possibly
compounded by the rest of the workspace's tests loading the machine). Not yet
root-caused; workaround is `--test-threads=1` for that crate.
**Summary:** during `cargo test --workspace` the `gpu_tests` binary sat idle for
25+ minutes. Every test created its own `wgpu::Instance` + device (requesting
adapter-maximum limits) concurrently, and readback used an unbounded
`device.poll(Wait)`.
**Fix:** tests hold a process-wide lock while they own a device, and
`GpuAccelerator` readback waits time out after 30 s with `GpuError::BufferMap`.
## Compound datatype versions 1 and 2 are mis-parsed (default libver files)
**Status:** fixed 2026-09-19. Found by adding a default-libver axis to the h5py
interop tests.
**Summary:** any compound dataset written with default libver bounds (plain
`h5py.File(path, 'w')`, datatype message version 1) failed to read, typically
with `Overflow("compound member 'x': byte_offset(0) + field_size(4136977) ...")`.
Only `libver='latest'` files (version 3+) and files written by clawhdf5 itself
worked, which is why the existing tests never caught it.
**Root cause:** `Datatype::parse` skipped 24 bytes of legacy per-member array
fields for v1 where the format has 28 (dimensionality 1 + reserved 3 +
permutation 4 + reserved 4 + 4 dimension sizes 16), and treated v2 like v1 minus
name padding, whereas v2 keeps the 8-byte name padding and has no array fields.
## Attributes with unsupported datatypes are silently dropped
**Status:** open.
**Summary:** `Dataset::attrs()` / `Group::attrs()` in the `clawhdf5` facade return
only attributes convertible to `AttrValue`. An attribute with, e.g., a compound
datatype is omitted from the map with no error or indication that it exists.
Planned: surface these as an explicit `AttrValue` variant (raw bytes + datatype)
or an error, as part of the "no silent skips" robustness work.