h5rs (dump, ls, diff, check --data) kept its own lenient VL decoder:
a heap object longer than its element was cut to the element's length
(libhdf5 and h5py refuse it), a null string printed "" where h5dump
prints NULL, the stored element size was trusted, and every heap
collection was kept as an owned copy for the whole run. It now resolves
each element with VlResolver::element / string_element (new: one element
in place, borrowing from the file), and refuses a VL type whose stored
element size is not 4 + offset size + 4, as File does. H5::heap_object
and its cache are gone. h5diff compares a null VL string equal to an
empty one; so does h5rs diff.
clawhdf5-wasm already resolved VL strings with read_vl_strings; it now
uses VlResolver and checks the stored element size before reading, as
File::read_string does.
Tests (h5py writes the files, patched for "a\0b", a null element and
mis-sized heap objects, with 8- and 4-byte offsets):
- h5rs_interop dump_prints_vl_data_like_h5dump: byte-identical to h5dump;
- dump_json_vl_values_match_h5py: h5py's values, errors where h5py fails;
- check_data_flags_mis_sized_vl_heap_objects;
- clawhdf5-wasm tests/vl_strings.rs: wasm, File and h5py agree.
All four fail before. check --data over the 150 cve_hdf5 CVE and fuzzer
files now passes 15 (h5dump rejects 8 of them), was 16 and 9: the
stored-size check flags cve-2024-32608. h5rs-check-ok-files.sh --data:
0 of 422 flagged; h5rs-fuzz.sh: clean on 180 files.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
-c meant "list at most N differences" in h5rs, but in h5diff -c is
--compare (a flag) and the count is -n/--count=N, so a script moved over
from h5diff behaved differently: `h5diff -r -c 2 A B` exits 2 (the 2 is
taken as a file name) while h5rs exited 1.
The count is now -n/--count, -c/--compare is accepted (h5rs always lists
objects that are not comparable), and the --count=N, --delta=D,
--relative=R forms are accepted; exit codes equal h5diff's on 7 cases.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
An OBJ that was a soft link was resolved and its target object compared,
so two files whose /g/s both point at /z differed when /z did: exit 1,
where h5diff (without --follow-symlinks) compares the links' target paths
and exits 0.
A soft link is now compared as a link wherever it is, OBJ included.
--follow-symlinks compares the objects soft links lead to instead, walks
into soft-linked groups, resolves relative targets against the link's
group, and treats two dangling links as the same; exit codes equal
h5diff's on 14 cases. External links are never followed (documented).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
With a tolerance, integers were converted to f64 before comparing, so
int64/uint64 values above 2^53 that differ compared equal: -d 0 on 2^60
and 2^60 + 1 exited 0, where h5diff exits 1. Integer pairs are now
compared in i128 (the delta against floor(D), the relative quotient from
an exact difference), and the report prints the exact difference.
h5diff compares exactly when -p is below the f64 epsilon (2^60 and
2^60 + 1 differ at -p 1e-18, and nextafter(2, 0) and 2 at -p 1.5e-16);
h5rs now does the same.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The path walk skipped the second hard link to an object, so a file that
shares one dataset between /x and /y differed from a file holding two
identical copies: "</y> exists only in <B>", exit 1, where h5diff exits 0.
For a hard-linked group every member was reported the same way.
diff now enumerates every path below the start object (a hard link back
to an ancestor is recorded but not descended into), so each name is
compared. A group whose links cannot be read is now an error instead of
an empty group.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
New workspace crate clawhdf5-tools with one binary, h5rs, built only on the
clawhdf5 facade and clawhdf5-format (no libhdf5, no C):
- ls [-r] [-v] FILE[/path]: h5ls's listing (same text in its first two
columns) plus the datatype; -v adds address, link count, layout and chunk
index, chunk size, storage, filters, datatype and attributes.
- dump [--json] [-A] [-p] [-d PATH] FILE: h5dump DDL (byte-identical to
h5dump 1.14.6 on the test files) or hdf5-json.
- stat FILE: h5stat's object/link/rank/layout/filter/attribute counts, raw
data and total size.
- diff [-r] [-q] [-d D] [-p R] A B [OBJ1 [OBJ2]]: structural and value
differences, exit 0/1/2 like h5diff.
- check [--data] FILE: walks every object, parses every message, verifies
the checksums of every v2+ structure (including the fractal heap blocks
the library never checks), checks chunk indexes against their datasets
and raw data for out-of-file or overlapping extents; every problem with
its address.
Values over --max-bytes are reported, not read; dense-storage heaps are
verified before objects are read from them; panics are caught (exit 3).
Tests compare with h5ls, h5stat, h5dump and h5diff and with h5py's values,
and flip the checksum of every checksummed structure in a v1.14-format file.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
