fix(tools): h5rs check --data follows VL data into the global heap

The README said check skips only "global heap collections other than
those a value read touches", but read_dataset returns the raw heap IDs,
so no collection was ever read: a file whose global heap collection
claims a 4 GiB object passed `check --data` with no problems, while
h5dump (and h5rs dump/diff) fail on it.

With --data, every variable-length element (strings and sequences, also
inside compounds, arrays and nested sequences) of every dataset and
attribute is followed into its collection. A collection that does not
parse, a missing heap object, or a sequence longer than its heap object
is a problem at the collection's address, once per object; the summary
counts the collections read.

Measured on tank, 2026-09-26: the 418 fully-read conformance ok files
still pass (scripts/h5rs-check-ok-files.sh --data, 0 flagged), and
`check --data` now flags 152 of the 180 CVE-corpus files (was 147); of
the 28 it passes, h5dump 1.14.6 rejects 21 (was 26 of 33).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 01:22:27 -05:00
co-authored by Claude Opus 5.5
parent 386bd1d41e
commit b8492bd28d
5 changed files with 228 additions and 13 deletions
@@ -639,6 +639,44 @@ fn check_flags_every_corrupted_checksum() {
}
}
/// `check --data` follows variable-length elements into the global heap:
/// a damaged collection is reported at its address, for the dataset and
/// the attribute that point into it. Without --data it is not read.
#[test]
fn check_data_follows_vl_data_into_the_global_heap() {
let Some(f) = generate() else { return };
for name in LIBVERS {
let mut data = std::fs::read(f.path(name)).unwrap();
let gcols: Vec<usize> = (0..data.len() - 4)
.filter(|&i| &data[i..i + 4] == b"GCOL")
.collect();
assert!(!gcols.is_empty(), "{name}: no global heap");
// "GCOL" version(1) reserved(3) size(8), then heap object 1:
// index(2) refcount(2) reserved(4) size(8) — claim 4 GiB.
let at = gcols[0];
data[at + 24..at + 28].fill(0xff);
let bad = f.path(&format!("bad-gcol-{name}"));
std::fs::write(&bad, &data).unwrap();
let bad = bad.to_string_lossy().into_owned();
let o = h5rs(&["check", "--data", &bad]);
let s = stdout(&o);
assert_eq!(code(&o), 1, "{name}: {s}");
let want = format!("problem: {at:#x} /vlstr: variable-length data: global heap");
assert!(s.contains(&want), "{name}: no {want:?} in\n{s}");
// h5dump refuses the file too.
if tool_available("h5dump") {
assert_ne!(code(&run("h5dump", &[&bad])), 0, "{name}: h5dump read it");
}
assert_eq!(code(&h5rs(&["check", &bad])), 0, "{name}: without --data");
let ok = h5rs(&["check", "--data", &f.p(name)]);
assert!(
stdout(&ok).contains(&format!("global heap collections read: {}", gcols.len())),
"{name}: {}",
stdout(&ok)
);
}
}
#[test]
fn check_flags_a_truncated_file() {
let Some(f) = generate() else { return };