Merge branch 'fix/p1-userblock-shared' into fix/p1-read-gaps

# Conflicts:
#	crates/clawhdf5-format/src/attribute.rs
#	crates/clawhdf5-format/src/datatype.rs
#	crates/clawhdf5-format/src/shared_message.rs
#	docs/known-issues.md
This commit is contained in:
osobh
2026-09-25 22:40:55 -05:00
21 changed files with 1023 additions and 154 deletions
+35
View File
@@ -296,6 +296,41 @@
- CI keeps zlib-ng building and tested; the arm64 job no longer needs cmake.
### Correctness
- `clawhdf5-format` reader: an old-style group whose local heap has a free
list pointing outside the heap was listed with names read from the broken
heap (garbage names on `cve-2021-36977.h5` once its user block was
applied). libhdf5 refuses such a heap ("bad heap free list"); so do we now,
with `FormatError::InvalidLocalHeapFreeList`. As in libhdf5 the free list
is checked when the first name is read (`LocalHeap::validate_free_list`,
new), so an empty group with a damaged heap still lists as empty.
- **Files with a user block** (`h5py.File(..., userblock_size=N)`, `h5jam`;
the superblock at 512, 1024, …) could not be read: every address in the
file is relative to the superblock, but it was applied from byte 0
(`InvalidObjectHeaderVersion` on the root group). `File` (mmap, buffered,
`from_bytes`), `MmapFile`, `LazyFile`, `AsyncHDF5File`, the VOL readers,
the HNSW loader and external VDS sources now view the file from the
superblock on, using the signature's position as the base address as
libhdf5 does; `user_block_size()` reports the user block (h5py's
`userblock_size`), and `as_bytes()` returns the bytes from the superblock
on. **Breaking (format crate):** `Superblock::parse` refuses a non-zero
signature offset with `FormatError::UserBlockNotStripped`, since the
addresses it returns would be applied to the wrong bytes; pass the slice
from `signature::split_user_block` (new) and parse at offset 0.
- `clawhdf5-format` reader: version-1 shared messages (HDF5 1.6-era files,
e.g. a dataset using a committed datatype in libhdf5's `tcompound.h5`)
read the heap-offset field of the embedded symbol-table entry as the
target address and failed with `InvalidObjectHeaderVersion`. The address
is now read after it, as libhdf5 does. **Breaking (format crate):**
`shared_message::parse_shared_ref` takes `length_size`. A reference whose
target header has no message of the referenced type is now
`FormatError::SharedMessageTargetMissing` instead of returning the first
other message found there (which decoded as garbage).
- `clawhdf5-format` reader: array members of version-1 compound datatypes
(HDF5 1.6-era files, e.g. libhdf5's `tcompound.h5`) were read as a single
element: a `[4] i32` member came back as one `i32`, with the wrong size.
The legacy per-member dimension fields are now decoded into an array type,
as libhdf5 does; more than four dimensions, or a zero-sized one, is an
error.
- `clawhdf5-format` reader — **values returned wrong with no error:**
- Fixed Array and Extensible Array chunk indexes were laid out by the
dataset's current shape instead of its max shape (23 libhdf5 test files,