attrs() read every attribute of an object through extract_attributes_full,
so one attribute it could not read (a corrupt or unsupported attribute
message, or a heap object it could not locate) failed all of them — the
same shape as the huge-object bug, where one 8 KiB attribute hid every
attribute on a NetCDF file's root group.
- clawhdf5-format: new attribute::extract_attributes_tolerant returns the
attributes it could read plus one error per attribute it could not.
Errors in the attribute index itself (Attribute Info message, dense
heap header, B-tree) still fail, since then it is unknown which
attributes exist. extract_attributes_full is unchanged (strict); both
share one implementation.
- clawhdf5: attrs() on Group/Dataset, MmapGroup/MmapDataset and
LazyGroup/LazyDataset leaves an unreadable attribute out (documented),
and the new attrs_with_errors() returns the map with the per-attribute
errors. A value is either returned complete or not at all.
Regression test: one_unreadable_attribute_does_not_hide_the_others (h5py
writes 11 dense attributes; one message's version byte is corrupted;
before: attrs() failed with InvalidAttributeVersion(127), after: the 10
others come back with their values and one error is reported).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A version-1 shared message reference is version, type, six reserved bytes
and then an old-style symbol table entry: link-name offset (length size),
object header address, cache type, reserved, scratch. We read the address
straight after the reserved bytes, i.e. the link-name offset, and the
committed datatype lookup failed with InvalidObjectHeaderVersion (the bytes
checked in tcompound.h5: name offset 0x10, then 0x590 = /type1). Datasets
of 1.4/1.6-era files that use a committed datatype were unreadable.
Skip the name offset. parse_shared_ref has no length size, so add
parse_shared_ref_sized and use it in every internal caller;
parse_shared_ref keeps its signature and assumes length size == offset
size. The old parse_v1_ref unit test encoded the wrong layout and now uses
the real bytes.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A dataset created from a committed (named) datatype stores only a shared-
message reference to it. The facade parsed those reference bytes as the
datatype itself, producing `Time { size: 0 }` and unreadable data, and an
attribute using a committed datatype was silently dropped.
- shared_message::parse_shared_ref had the encoding wrong: it skipped six
reserved bytes for version 2 (only version 1 has them) and had the version 3
types inverted (1 is the SOHM heap, 2 is "committed, in another object
header"). Verified against h5py 3.16 / HDF5 2.0, which writes
`02 02 <address>` under both default and latest libver bounds. Resolution
now dispatches on which field the reference carries.
- New shared_message::message_data resolves a header message through the
indirection; the reader, lazy and mmap facades use it for datatype,
dataspace and filter-pipeline messages.
- AttributeMessage honours the v2/v3 flags (bit 0 datatype shared, bit 1
dataspace shared) via the new parse_in_file, used everywhere file data is
available. Parsing a shared attribute without file access is now
FormatError::UnresolvedSharedMessage instead of a garbage datatype.
- h5py interop test covering both libver settings.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
- clippy --all-targets plus a clawhdf5-format feature matrix (parallel, lz4,
zstd, pcodec, fast-checksum); fix the accumulated lint backlog in test,
bench and feature-gated code (no behaviour changes).
- Install python3 + h5py/numpy/netCDF4/xarray in the CI container and set
CLAWHDF5_REQUIRE_INTEROP=1, which makes a missing interop dependency a test
failure. Every h5py/netCDF4 interop test used to skip silently in CI. Run
the #[ignore]d writer_h5py_tests suite explicitly.
- cargo bench --no-run so benches can't rot; fix bench.rs and memory_bench.rs,
which no longer compiled against the current strategy/consolidation APIs.
- Optional fuzz smoke run via CLAWHDF5_FUZZ_SECONDS.
- CHANGELOG and docs/known-issues.md updated.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
osobh