fix(tools,wasm): resolve VL data through the library's VlResolver
h5rs (dump, ls, diff, check --data) kept its own lenient VL decoder: a heap object longer than its element was cut to the element's length (libhdf5 and h5py refuse it), a null string printed "" where h5dump prints NULL, the stored element size was trusted, and every heap collection was kept as an owned copy for the whole run. It now resolves each element with VlResolver::element / string_element (new: one element in place, borrowing from the file), and refuses a VL type whose stored element size is not 4 + offset size + 4, as File does. H5::heap_object and its cache are gone. h5diff compares a null VL string equal to an empty one; so does h5rs diff. clawhdf5-wasm already resolved VL strings with read_vl_strings; it now uses VlResolver and checks the stored element size before reading, as File::read_string does. Tests (h5py writes the files, patched for "a\0b", a null element and mis-sized heap objects, with 8- and 4-byte offsets): - h5rs_interop dump_prints_vl_data_like_h5dump: byte-identical to h5dump; - dump_json_vl_values_match_h5py: h5py's values, errors where h5py fails; - check_data_flags_mis_sized_vl_heap_objects; - clawhdf5-wasm tests/vl_strings.rs: wasm, File and h5py agree. All four fail before. check --data over the 150 cve_hdf5 CVE and fuzzer files now passes 15 (h5dump rejects 8 of them), was 16 and 9: the stored-size check flags cve-2024-32608. h5rs-check-ok-files.sh --data: 0 of 422 flagged; h5rs-fuzz.sh: clean on 180 files. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -76,8 +76,13 @@ print the same bytes as h5dump 1.14.6 and as Debian's h5dump 1.14.5 (the
|
||||
was run in that image on 2026-09-26) — `dump_matches_h5dump` in
|
||||
`tests/h5rs_interop.rs` checks this, and `dump_shows_nul_padding_in_nested_strings`
|
||||
that null-padded strings show their NULs (`"a\000b"`) at any depth, as
|
||||
h5dump's do. Not covered by those tests: references, opaque, bitfield,
|
||||
variable-length sequences and virtual datasets. Known differences from
|
||||
h5dump's do. `dump_prints_vl_data_like_h5dump` covers variable-length
|
||||
strings (one with an embedded NUL, which prints up to the NUL; empty; null,
|
||||
which prints `NULL`), variable-length sequences, a VL compound member and
|
||||
a VL attribute, with 8- and 4-byte offsets. Not covered by those tests:
|
||||
references, opaque, bitfield, non-ASCII UTF-8 (h5dump prints each byte
|
||||
above 0x7f as a sign-extended octal escape, h5rs the character) and
|
||||
virtual datasets. Known differences from
|
||||
h5dump:
|
||||
|
||||
- Floats print at their own precision (a `float32` 0.1 prints as `0.1`),
|
||||
@@ -233,9 +238,11 @@ extension) and checks:
|
||||
(which catches corrupt compressed data and Fletcher-32 mismatches), and
|
||||
follows every variable-length element (strings and sequences, also inside
|
||||
compounds and arrays) of every dataset and attribute into its global heap
|
||||
collection: a collection that does not parse, a missing heap object, or a
|
||||
sequence longer than its heap object is a problem at the collection's
|
||||
address. Data the
|
||||
collection: a collection that does not parse or overlaps another, a missing
|
||||
heap object, or a heap object whose size is not exactly the element's
|
||||
length times its base size (libhdf5 refuses such an element) is a problem
|
||||
at the collection's address. Variable-length elements are resolved by the
|
||||
library's `VlResolver`, as `clawhdf5::File` resolves them. Data the
|
||||
tool cannot decode (a filter it does not implement, such as szip, or a
|
||||
dataset over `--max-bytes`) is a `note:`, not a problem. Every problem is
|
||||
printed with the address of the structure involved; the exit status is 0
|
||||
@@ -252,9 +259,10 @@ none at all without `--data`), and objects reachable only by external links. It
|
||||
clawhdf5's parsers, so it accepts what they accept: some header damage that
|
||||
libhdf5 refuses goes unreported. Of the 150 CVE and fuzzer files of the
|
||||
HDF Group's `cve_hdf5` corpus (`cvefiles/` and `fuzzerfiles/`),
|
||||
`check --data` passes 16, and h5dump 1.14.6 rejects 9 of those (tank,
|
||||
`check --data` passes 15, and h5dump 1.14.6 rejects 8 of those (tank,
|
||||
2026-09-26, `h5rs check --data F` and `h5dump F` per file; before the
|
||||
library's header checks it passed 28, of which h5dump rejects 21).
|
||||
library's header checks it passed 28, of which h5dump rejects 21, and 16
|
||||
and 9 before a VL type's stored element size was checked).
|
||||
|
||||
## Robustness
|
||||
|
||||
|
||||
Reference in New Issue
Block a user