Merge branch 'fix/p1-userblock-shared' into fix/p1-read-gaps
# Conflicts: # crates/clawhdf5-format/src/attribute.rs # crates/clawhdf5-format/src/datatype.rs # crates/clawhdf5-format/src/shared_message.rs # docs/known-issues.md
This commit is contained in:
@@ -296,6 +296,41 @@
|
||||
- CI keeps zlib-ng building and tested; the arm64 job no longer needs cmake.
|
||||
|
||||
### Correctness
|
||||
- `clawhdf5-format` reader: an old-style group whose local heap has a free
|
||||
list pointing outside the heap was listed with names read from the broken
|
||||
heap (garbage names on `cve-2021-36977.h5` once its user block was
|
||||
applied). libhdf5 refuses such a heap ("bad heap free list"); so do we now,
|
||||
with `FormatError::InvalidLocalHeapFreeList`. As in libhdf5 the free list
|
||||
is checked when the first name is read (`LocalHeap::validate_free_list`,
|
||||
new), so an empty group with a damaged heap still lists as empty.
|
||||
- **Files with a user block** (`h5py.File(..., userblock_size=N)`, `h5jam`;
|
||||
the superblock at 512, 1024, …) could not be read: every address in the
|
||||
file is relative to the superblock, but it was applied from byte 0
|
||||
(`InvalidObjectHeaderVersion` on the root group). `File` (mmap, buffered,
|
||||
`from_bytes`), `MmapFile`, `LazyFile`, `AsyncHDF5File`, the VOL readers,
|
||||
the HNSW loader and external VDS sources now view the file from the
|
||||
superblock on, using the signature's position as the base address as
|
||||
libhdf5 does; `user_block_size()` reports the user block (h5py's
|
||||
`userblock_size`), and `as_bytes()` returns the bytes from the superblock
|
||||
on. **Breaking (format crate):** `Superblock::parse` refuses a non-zero
|
||||
signature offset with `FormatError::UserBlockNotStripped`, since the
|
||||
addresses it returns would be applied to the wrong bytes; pass the slice
|
||||
from `signature::split_user_block` (new) and parse at offset 0.
|
||||
- `clawhdf5-format` reader: version-1 shared messages (HDF5 1.6-era files,
|
||||
e.g. a dataset using a committed datatype in libhdf5's `tcompound.h5`)
|
||||
read the heap-offset field of the embedded symbol-table entry as the
|
||||
target address and failed with `InvalidObjectHeaderVersion`. The address
|
||||
is now read after it, as libhdf5 does. **Breaking (format crate):**
|
||||
`shared_message::parse_shared_ref` takes `length_size`. A reference whose
|
||||
target header has no message of the referenced type is now
|
||||
`FormatError::SharedMessageTargetMissing` instead of returning the first
|
||||
other message found there (which decoded as garbage).
|
||||
- `clawhdf5-format` reader: array members of version-1 compound datatypes
|
||||
(HDF5 1.6-era files, e.g. libhdf5's `tcompound.h5`) were read as a single
|
||||
element: a `[4] i32` member came back as one `i32`, with the wrong size.
|
||||
The legacy per-member dimension fields are now decoded into an array type,
|
||||
as libhdf5 does; more than four dimensions, or a zero-sized one, is an
|
||||
error.
|
||||
- `clawhdf5-format` reader — **values returned wrong with no error:**
|
||||
- Fixed Array and Extensible Array chunk indexes were laid out by the
|
||||
dataset's current shape instead of its max shape (23 libhdf5 test files,
|
||||
|
||||
Reference in New Issue
Block a user