osobhandClaude Fable 5.1
b55b7dbac5
Class 11 (datatype version 5) properties are a single base floating-point
datatype message, not a compound-style member list. The old parser read the
base type's bytes as member names, yielding a garbage datatype, and failed
with UnexpectedEof when a complex type was nested in a compound.
Parse the base type and surface the type as the equivalent {r, i} compound
(the shape h5py writes for numpy complex dtypes), with a size check against
the base type. Covered by byte-level tests taken from HDF5 2.0 output and an
h5py end-to-end test (writer_h5py_tests is now 27/27 against HDF5 2.0.0).
Found while validating a user report of InvalidDatatypeVersion
{ class: 6, version: 5 } against v2.1.0 (already fixed on main in a13ff51,
never released). Add docs/known-issues.md recording that report, this bug,
the open reference-v4 gap and a gpu_tests parallel-run hang; credit the
reporter in the changelog.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
3.8 KiB
Known Issues
Bugs found during development or downstream use, tracked here because this
repository's issue tracker is disabled. One entry per bug; when an entry is
fixed, record the fix in CHANGELOG.md and update its status here rather than
deleting it.
Compound datatype message version 5 is not parsed (HDF5 2.0)
Status: fixed on main in a13ff51 (2026-06-03); not in the v2.1.0
tag, which was cut five commits earlier. Ships in the next release.
Reported by: M. Scot Breitenfeld (The HDF Group), 2026-09-08, against v2.1.0.
Summary: clawhdf5-format v2.1.0 rejects any dataset with a compound
(struct) datatype written by an HDF5 2.0 library in libver='latest' mode:
InvalidDatatypeVersion { class: 6, version: 5 }.
Reproduction (h5py 3.16.0 / HDF5 2.0.0):
import h5py, numpy as np
dt = np.dtype([('x', 'f8'), ('y', 'f8'), ('id', 'i4')])
data = np.array([(1.0, 2.0, 10), (3.0, 4.0, 20)], dtype=dt)
f = h5py.File('compound.h5', 'w', libver='latest')
f.create_dataset('particles', data=data)
f.close()
Committed as crates/clawhdf5-format/tests/writer_h5py_tests.rs::read_h5py_generated_compound
(#[ignore]d; needs python3 with h5py on PATH). Run with
cargo test -p clawhdf5-format --test writer_h5py_tests -- --include-ignored:
v2.1.0 gives 25 passed / 1 failed; main passes everything.
Root cause: the compound (class 6) branch of Datatype::parse
(crates/clawhdf5-format/src/datatype.rs) accepted only versions 1–4. Datatype
message versions 4 and 5 changed only the Reference and Complex classes, so a
v5-tagged compound uses the unchanged v3 member-list layout.
Fix: versions 3–5 are accepted for compound (class 6) and array (class 10)
datatypes, and data layout message version 5 is accepted too (needed for every
chunked dataset written by HDF5 2.0). Byte-level regression tests:
test_compound_v5_from_hdf5_2_0, test_array_v5_from_hdf5_2_0.
Native complex datatype (class 11) is mis-parsed (HDF5 2.0)
Status: fixed 2026-09-18. Found while validating the report above.
Summary: HDF5 2.0 native complex types (H5T_COMPLEX_IEEE_F64LE etc.)
were parsed as if they carried a compound-style member list. The properties are
actually a single base floating-point datatype, so the parser produced a garbage
datatype, or UnexpectedEof when the complex type was a compound member. h5py's
default numpy-complex mapping is unaffected (it writes a {r, i} compound);
only files using the native type through the C API / h5py low-level API hit this.
Fix: class 11 parses its base type and is surfaced as the equivalent
{r, i} compound. Tests: test_complex_v5_from_hdf5_2_0,
test_compound_with_complex_member_from_hdf5_2_0,
writer_h5py_tests.rs::read_h5py_generated_native_complex.
Revised reference datatype (class 7, version 4) is not parsed
Status: open, unconfirmed against a real file.
Summary: HDF5 1.12+ H5T_STD_REF references use datatype version 4 with
reference types 2–4 (object2 / region2 / attribute), which Datatype::parse
rejects with InvalidReferenceType. h5py still writes the legacy v1
object/region references, which read correctly, so no reproducing file has been
generated yet; one written with the C API (H5T_STD_REF) is needed.
clawhdf5-gpu gpu_tests can hang under the default parallel test runner
Status: open. Observed 2026-09-18 (RTX 5060 Ti, Linux).
Summary: during cargo test --workspace, the gpu_tests binary sat idle
(~1% CPU) for 25+ minutes and had to be killed. Run single-threaded it passes
in seconds (20/20): cargo test -p clawhdf5-gpu --test gpu_tests -- --test-threads=1.
Suspected cause: several tests creating wgpu devices concurrently (possibly
compounded by the rest of the workspace's tests loading the machine). Not yet
root-caused; workaround is --test-threads=1 for that crate.