fix(tools,wasm): resolve VL data through the library's VlResolver

h5rs (dump, ls, diff, check --data) kept its own lenient VL decoder:
a heap object longer than its element was cut to the element's length
(libhdf5 and h5py refuse it), a null string printed "" where h5dump
prints NULL, the stored element size was trusted, and every heap
collection was kept as an owned copy for the whole run. It now resolves
each element with VlResolver::element / string_element (new: one element
in place, borrowing from the file), and refuses a VL type whose stored
element size is not 4 + offset size + 4, as File does. H5::heap_object
and its cache are gone. h5diff compares a null VL string equal to an
empty one; so does h5rs diff.

clawhdf5-wasm already resolved VL strings with read_vl_strings; it now
uses VlResolver and checks the stored element size before reading, as
File::read_string does.

Tests (h5py writes the files, patched for "a\0b", a null element and
mis-sized heap objects, with 8- and 4-byte offsets):
- h5rs_interop dump_prints_vl_data_like_h5dump: byte-identical to h5dump;
- dump_json_vl_values_match_h5py: h5py's values, errors where h5py fails;
- check_data_flags_mis_sized_vl_heap_objects;
- clawhdf5-wasm tests/vl_strings.rs: wasm, File and h5py agree.
All four fail before. check --data over the 150 cve_hdf5 CVE and fuzzer
files now passes 15 (h5dump rejects 8 of them), was 16 and 9: the
stored-size check flags cve-2024-32608. h5rs-check-ok-files.sh --data:
0 of 422 flagged; h5rs-fuzz.sh: clean on 180 files.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 09:04:21 -05:00
co-authored by Claude Opus 5.5
parent 41b7837d0a
commit d345ffbf80
11 changed files with 533 additions and 152 deletions
+18 -2
View File
@@ -61,6 +61,22 @@
object running past its collection. Conformance unchanged at 575 of 697
(`crates/clawhdf5-format/tests/vl_heap_bounds.rs`).
- **Every reader resolves VL data the same way.** `h5rs` (`dump`, `ls`,
`diff`, `check --data`) had its own lenient VL decoder: a heap object
longer than the element's length was cut to it (h5py refuses it), a null
string printed `""` where h5dump prints `NULL`, the stored element size
was trusted, and each heap collection was kept as a copy for the whole
run. It now resolves through `VlResolver`, so `dump` matches h5dump byte
for byte on VL strings (`"a\0b"` as `"a"`, null as `NULL`), VL sequences
and 4-byte-offset files, `dump --json` gives h5py's values, and
`check --data` reports any heap object whose size is not exactly the
element's length × base size. `clawhdf5-wasm` already resolved VL strings
with `read_vl_strings`; it now uses `VlResolver` and refuses a VL type
whose stored element size disagrees with the file, as `File` does
(`crates/clawhdf5-tools/tests/h5rs_interop.rs`,
`crates/clawhdf5-wasm/tests/vl_strings.rs`). New
`VlResolver::element` / `string_element` resolve one element in place.
### Plugin filters (2026-09-26)
- **LZF, bitshuffle, bzip2 and Blosc read and write, in pure Rust.** Files
written by h5py with `compression="lzf"`, or with hdf5plugin's
@@ -264,10 +280,10 @@
printed with its address; exit 1 when there are any. libhdf5's h5check
reads only the 1.8 format. On the conformance corpus it passes all 418
files that both clawhdf5 and h5py read in full, and `check --data` flags
134 of the 150 CVE and fuzzer files of the `cve_hdf5` corpus (tank,
135 of the 150 CVE and fuzzer files of the `cve_hdf5` corpus (tank,
2026-09-26). `--data` also follows variable-length data into its global
heap collections and reports a damaged one at its address. It inherits
the library's tolerance, though: 9 of the 16 it passes are files h5dump
the library's tolerance, though: 8 of the 15 it passes are files h5dump
1.14.6 rejects (see `docs/known-issues.md`, header checks).
- Values over `--max-bytes` (default 1 GiB) are reported instead of read;
a panic is caught and reported as an internal error (exit 3).