Merge branch 'feat/p1-layout-v1v2' into fix/p1-read-gaps

This commit is contained in:
osobh
2026-09-25 22:40:26 -05:00
12 changed files with 573 additions and 16 deletions
+19
View File
@@ -253,6 +253,12 @@
- A pipeline with Fletcher32 ahead of the compressor (h5py
`set_fletcher32()` then `set_deflate()`) no longer fails with "deflate:
output exceeds size limit".
- `clawhdf5-format`: **HDF5 1.4/1.6-era files are readable.** Data Layout
message versions 1 and 2 (compact, contiguous, and chunked through the
version-1 B-tree) failed with `InvalidLayoutVersion` — 84 of the 686 files in
the 2026-09-25 audit sweep, 205 datasets. They now read as libhdf5 does;
checked byte for byte against h5py on HDF5's own test files
(`tests/legacy_format_interop.rs`).
### Storage
- `clawhdf5-format`: **half-precision datasets.**
@@ -311,6 +317,19 @@
- Two threads reading two chunked datasets through one `File` could get each
other's chunks (the shared chunk cache was switched between datasets
across separate lock acquisitions). The cache is now keyed by dataset.
- Compound datatype version 1 members with legacy array dimensions (HDF5
before 1.4, which had no array class) were read as a single scalar at
the member's offset; they are now array members, as in libhdf5
(`tarrold.h5`, `tcompound.h5`). Only reachable once layout versions 1/2
were readable, since the files that use it are that old.
- `clawhdf5-format` reader — errors on valid files: a version-1 shared
message (a committed datatype in HDF5 1.4/1.6-era files) was read as if the
object header address followed the reserved bytes; it follows a link-name
offset (the reference is an old-style symbol table entry), so the reader
followed the name offset and failed with `InvalidObjectHeaderVersion`
(`tcompound.h5`). New `shared_message::parse_shared_ref_sized` takes the
superblock's length size; `parse_shared_ref` assumes it equals the offset
size.
- `clawhdf5-format` reader — errors on valid files: enum and bool datasets
through the numeric readers; the "don't filter partial edge chunks" layout
flag; Fletcher32 ahead of deflate (NetCDF-4's order). Unknown-message flags