Files

T

osobhandClaude Fable 5.1 bbe1baa208 ci: lint all targets, run interop suites for real, compile benches

- clippy --all-targets plus a clawhdf5-format feature matrix (parallel, lz4,
  zstd, pcodec, fast-checksum); fix the accumulated lint backlog in test,
  bench and feature-gated code (no behaviour changes).
- Install python3 + h5py/numpy/netCDF4/xarray in the CI container and set
  CLAWHDF5_REQUIRE_INTEROP=1, which makes a missing interop dependency a test
  failure. Every h5py/netCDF4 interop test used to skip silently in CI. Run
  the #[ignore]d writer_h5py_tests suite explicitly.
- cargo bench --no-run so benches can't rot; fix bench.rs and memory_bench.rs,
  which no longer compiled against the current strategy/consolidation APIs.
- Optional fuzz smoke run via CLAWHDF5_FUZZ_SECONDS.
- CHANGELOG and docs/known-issues.md updated.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

2026-09-19 05:36:22 -07:00

5.0 KiB

Raw Blame History

Known Issues

Bugs found during development or downstream use, tracked here because this repository's issue tracker is disabled. One entry per bug; when an entry is fixed, record the fix in CHANGELOG.md and update its status here rather than deleting it.

Compound datatype message version 5 is not parsed (HDF5 2.0)

Status: fixed on main in a13ff51 (2026-06-03); not in the v2.1.0 tag, which was cut five commits earlier. Ships in the next release.

Reported by: M. Scot Breitenfeld (The HDF Group), 2026-09-08, against v2.1.0.

Summary: clawhdf5-format v2.1.0 rejects any dataset with a compound (struct) datatype written by an HDF5 2.0 library in libver='latest' mode: InvalidDatatypeVersion { class: 6, version: 5 }.

Reproduction (h5py 3.16.0 / HDF5 2.0.0):

import h5py, numpy as np
dt = np.dtype([('x', 'f8'), ('y', 'f8'), ('id', 'i4')])
data = np.array([(1.0, 2.0, 10), (3.0, 4.0, 20)], dtype=dt)
f = h5py.File('compound.h5', 'w', libver='latest')
f.create_dataset('particles', data=data)
f.close()

Committed as crates/clawhdf5-format/tests/writer_h5py_tests.rs::read_h5py_generated_compound (#[ignore]d; needs python3 with h5py on PATH). Run with cargo test -p clawhdf5-format --test writer_h5py_tests -- --include-ignored: v2.1.0 gives 25 passed / 1 failed; main passes everything.

Root cause: the compound (class 6) branch of Datatype::parse (crates/clawhdf5-format/src/datatype.rs) accepted only versions 1–4. Datatype message versions 4 and 5 changed only the Reference and Complex classes, so a v5-tagged compound uses the unchanged v3 member-list layout.

Fix: versions 3–5 are accepted for compound (class 6) and array (class 10) datatypes, and data layout message version 5 is accepted too (needed for every chunked dataset written by HDF5 2.0). Byte-level regression tests: test_compound_v5_from_hdf5_2_0, test_array_v5_from_hdf5_2_0.

Native complex datatype (class 11) is mis-parsed (HDF5 2.0)

Status: fixed 2026-09-18. Found while validating the report above.

Summary: HDF5 2.0 native complex types (H5T_COMPLEX_IEEE_F64LE etc.) were parsed as if they carried a compound-style member list. The properties are actually a single base floating-point datatype, so the parser produced a garbage datatype, or UnexpectedEof when the complex type was a compound member. h5py's default numpy-complex mapping is unaffected (it writes a {r, i} compound); only files using the native type through the C API / h5py low-level API hit this.

Fix: class 11 parses its base type and is surfaced as the equivalent {r, i} compound. Tests: test_complex_v5_from_hdf5_2_0, test_compound_with_complex_member_from_hdf5_2_0, writer_h5py_tests.rs::read_h5py_generated_native_complex.

Revised reference datatype (class 7, version 4) is not parsed

Status: open, unconfirmed against a real file.

Summary: HDF5 1.12+ H5T_STD_REF references use datatype version 4 with reference types 2–4 (object2 / region2 / attribute), which Datatype::parse rejects with InvalidReferenceType. h5py still writes the legacy v1 object/region references, which read correctly, so no reproducing file has been generated yet; one written with the C API (H5T_STD_REF) is needed.

`clawhdf5-gpu` `gpu_tests` can hang under the default parallel test runner

Status: fixed 2026-09-19.

Summary: during cargo test --workspace the gpu_tests binary sat idle for 25+ minutes. Every test created its own wgpu::Instance + device (requesting adapter-maximum limits) concurrently, and readback used an unbounded device.poll(Wait).

Fix: tests hold a process-wide lock while they own a device, and GpuAccelerator readback waits time out after 30 s with GpuError::BufferMap.

Compound datatype versions 1 and 2 are mis-parsed (default libver files)

Status: fixed 2026-09-19. Found by adding a default-libver axis to the h5py interop tests.

Summary: any compound dataset written with default libver bounds (plain h5py.File(path, 'w'), datatype message version 1) failed to read, typically with Overflow("compound member 'x': byte_offset(0) + field_size(4136977) ..."). Only libver='latest' files (version 3+) and files written by clawhdf5 itself worked, which is why the existing tests never caught it.

Root cause: Datatype::parse skipped 24 bytes of legacy per-member array fields for v1 where the format has 28 (dimensionality 1 + reserved 3 + permutation 4 + reserved 4 + 4 dimension sizes 16), and treated v2 like v1 minus name padding, whereas v2 keeps the 8-byte name padding and has no array fields.

Attributes with unsupported datatypes are silently dropped

Status: open.

Summary: Dataset::attrs() / Group::attrs() in the clawhdf5 facade return only attributes convertible to AttrValue. An attribute with, e.g., a compound datatype is omitted from the map with no error or indication that it exists. Planned: surface these as an explicit AttrValue variant (raw bytes + datatype) or an error, as part of the "no silent skips" robustness work.

5.0 KiB Raw Blame History Unescape Escape