Commit Graph
2 Commits
Author SHA1 Message Date
osobhandClaude Opus 5.5 36356ba8a1 fix(format): keep the array dimensions of compound v1 members
Compound datatype version 1 carries, per member, a dimensionality and four
dimension sizes (HDF5 before 1.4 had no array class). The parser skipped
those 28 bytes, so a member such as `f: f32[4]` came back as a single f32
at the member's offset: the compound's size was right but its members were
wrong. libhdf5 wraps such a member in an array type of the first
`dimensionality` sizes and ignores the permutation; do the same, and
reject a dimensionality above 4 as libhdf5 does.

Only files old enough to also use layout message v1 have these, so this
became reachable with the previous commit (tarrold.h5, tcompound.h5).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 21:55:31 -05:00
osobhandClaude Opus 5.5 85eb7f5ce2 feat(format): read Data Layout message versions 1 and 2
HDF5 1.4/1.6-era files store the layout as version 1 or 2: version,
dimensionality, class, 5 reserved bytes, an address (contiguous and chunked
only), dimensionality 32-bit sizes (with the trailing element-size
dimension) and, for compact storage, a 32-bit size and the raw data. They
failed with InvalidLayoutVersion — 84 of the 686 files in the audit sweep,
205 datasets.

Map them onto the existing variants: chunked uses the same version-1
B-tree chunk index as version 3 and is reported as version 3, so every
chunked read path (filters, selections, caches) applies unchanged.
Contiguous size is the product of the stored dimensions, which is what
libhdf5 computes from the dataspace; a disagreement fails the reader's size
check instead of returning wrong data.

Fixtures are HDF5's own deflate.h5 (v1, chunked + deflate) and
h5ex_g_iterate.h5 (v2, contiguous, one unallocated dataset); the new
interop test compares every dataset byte for byte against h5py.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-25 21:55:31 -05:00