fix(format): resolve VL elements as libhdf5 does

Checked with h5py on a patched file:
- a VL string with an embedded NUL reads up to the NUL (libhdf5 converts
  VL strings to C strings); read_vl_strings returned "a\0b";
- an element whose global heap object is not length x base size bytes is
  an error ("Expected global heap object size does not match"); we
  returned the object cut to the length;
- a heap address of 0 is a null element whatever its length.

vl_data::VlResolver does this, caching each parsed heap collection:
read_vl_strings parsed the whole collection again for every element.
read_vl_strings and read_vl_bytes use it; check_element_size refuses a VL
type whose stored element size is not 4 + offset size + 4. The
conformance probe resolves VL values through VlResolver instead of its
own lenient copy (575 of 697, unchanged).

The new unit tests fail against the old read_vl_strings.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:20:19 -05:00
co-authored by Claude Opus 5.5
parent 10da8f0d09
commit f99587c27d
3 changed files with 329 additions and 98 deletions
+14
View File
@@ -16,6 +16,20 @@
exhaustively destructures that variant; patterns with `..` are
unaffected), and it is written back as stored. Tested against h5py
(`crates/clawhdf5/tests/vl_offset4_interop.rs`).
- **Wrong data: VL strings with an embedded NUL, and VL elements whose heap
object has the wrong size.** libhdf5 hands VL strings over as C strings,
so h5py reads `"a\0b"` as `"a"`; `read_vl_strings` returned the NUL and
what followed. An element whose heap object is not exactly
`length × base size` bytes is refused by libhdf5 ("Expected global heap
object size does not match"); we returned the object cut or padded to
the length. Both now behave as libhdf5, and a heap address of 0 is a null
element (empty) whatever its length. The new
`clawhdf5_format::vl_data::VlResolver` does this and parses each global
heap collection once per read: `read_vl_strings` parsed the whole
collection again for every element. `vl_data::check_element_size` refuses
a VL datatype whose stored size is not 4 + offset size + 4 (libhdf5
ignores the stored size). The conformance probe resolves VL elements
with `VlResolver` too; conformance unchanged at 575 of 697.
### Plugin filters (2026-09-26)
- **LZF, bitshuffle, bzip2 and Blosc read and write, in pure Rust.** Files