h5rs tools, browser reader, libhdf5 header checks, plugin filters, concurrency benchmark #14

Merged
osobh merged 60 commits from feat/p1-proof into main 2026-09-26 13:14:39 +00:00
3 changed files with 14 additions and 4 deletions
Showing only changes of commit 845a9d0125 - Show all commits
+4 -2
View File
@@ -142,8 +142,10 @@
that raw data lies inside the file without overlaps. Every problem is
printed with its address; exit 1 when there are any. libhdf5's h5check
reads only the 1.8 format. On the conformance corpus it passes all 418
files that both clawhdf5 and h5py read in full, and flags 149 of the
180 CVE reproducers.
files that both clawhdf5 and h5py read in full, and `check --data` flags
147 of the 180 files of the CVE corpus (tank, 2026-09-26). It inherits
the library's tolerance, though: 26 of the 33 it passes are files
h5dump 1.14.6 rejects (see `docs/known-issues.md`, header checks).
- Values over `--max-bytes` (default 1 GiB) are reported instead of read;
a panic is caught and reported as an internal error (exit 3).
`scripts/h5rs-fuzz.sh` runs every subcommand over a corpus (default the
+5 -1
View File
@@ -207,7 +207,11 @@ B-tree chunk indexes, and version 3 superblocks).
What it does not check: free-space manager and shared-message (SOHM) table
checksums, global heap collections other than those a value read touches,
and objects reachable only by external links.
and objects reachable only by external links. It validates with
clawhdf5's parsers, so it accepts what they accept: some header damage that
libhdf5 refuses goes unreported (of the 180 files of the HDF Group's CVE
corpus, `check --data` passes 33, and h5dump 1.14.6 rejects 26 of those;
tank, 2026-09-26).
## Robustness
+5 -1
View File
@@ -135,7 +135,11 @@ fill-value item that did is fixed).
- **Filters:** blosc, blosc2, bitshuffle, bzip2, LZF and zfp are not
implemented.
- **Header checks:** on 12 CVE datasets libhdf5 rejects a corrupt header and
we read data anyway. We need stricter header checks.
we read data anyway. We need stricter header checks. The same gap shows in
`h5rs check`, which validates with the library's parsers: of the 180
files of the CVE corpus, `check --data` passes 33, and h5dump 1.14.6
rejects 26 of those (measured on tank, 2026-09-26, with
`h5rs check --data F` and `h5dump F` per file).
- **Writer:**
- Nested groups beyond one level: path-like names are now refused, not
created.