fix(format): resolve VL elements as libhdf5 does
Checked with h5py on a patched file:
- a VL string with an embedded NUL reads up to the NUL (libhdf5 converts
VL strings to C strings); read_vl_strings returned "a\0b";
- an element whose global heap object is not length x base size bytes is
an error ("Expected global heap object size does not match"); we
returned the object cut to the length;
- a heap address of 0 is a null element whatever its length.
vl_data::VlResolver does this, caching each parsed heap collection:
read_vl_strings parsed the whole collection again for every element.
read_vl_strings and read_vl_bytes use it; check_element_size refuses a VL
type whose stored element size is not 4 + offset size + 4. The
conformance probe resolves VL values through VlResolver instead of its
own lenient copy (575 of 697, unchanged).
The new unit tests fail against the old read_vl_strings.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -16,6 +16,20 @@
|
||||
exhaustively destructures that variant; patterns with `..` are
|
||||
unaffected), and it is written back as stored. Tested against h5py
|
||||
(`crates/clawhdf5/tests/vl_offset4_interop.rs`).
|
||||
- **Wrong data: VL strings with an embedded NUL, and VL elements whose heap
|
||||
object has the wrong size.** libhdf5 hands VL strings over as C strings,
|
||||
so h5py reads `"a\0b"` as `"a"`; `read_vl_strings` returned the NUL and
|
||||
what followed. An element whose heap object is not exactly
|
||||
`length × base size` bytes is refused by libhdf5 ("Expected global heap
|
||||
object size does not match"); we returned the object cut or padded to
|
||||
the length. Both now behave as libhdf5, and a heap address of 0 is a null
|
||||
element (empty) whatever its length. The new
|
||||
`clawhdf5_format::vl_data::VlResolver` does this and parses each global
|
||||
heap collection once per read: `read_vl_strings` parsed the whole
|
||||
collection again for every element. `vl_data::check_element_size` refuses
|
||||
a VL datatype whose stored size is not 4 + offset size + 4 (libhdf5
|
||||
ignores the stored size). The conformance probe resolves VL elements
|
||||
with `VlResolver` too; conformance unchanged at 575 of 697.
|
||||
|
||||
### Plugin filters (2026-09-26)
|
||||
- **LZF, bitshuffle, bzip2 and Blosc read and write, in pure Rust.** Files
|
||||
|
||||
Reference in New Issue
Block a user