diff --git a/CHANGELOG.md b/CHANGELOG.md index 306dd13..1518e19 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,27 @@ ## Unreleased +### Conformance: no our-errors or mismatches left (2026-09-27) +- The last 3 our-errors and 2 mismatches were documented as not ours but + still counted against us. Re-checked with evidence + (`docs/known-issues.md`, "Conformance: the last non-ok files"): + - The 2 mismatches were h5py's big-endian VL bug (elements returned with + the file's bytes under a little-endian dtype). `conformance/ref.py` now + checks the installed h5py has the bug and relabels such elements before + hashing, so their values are compared: `attr_datatypes.hdf5` and + `tcomplex_be.h5` are identical to clawhdf5's and now **ok**. + - The 3 our-errors are objects HDF5 2.0 reads by over-reading memory + (scale-offset codes past a chunk, unfiltered chunks shorter than a + chunk, an N-Bit parameter list one value short). h5py's values for them + change between runs, with `MALLOC_PERTURB_` and with import order, and + h5dump 1.14.6 prints others, so they are not the file's data and + clawhdf5 keeps refusing them. `conformance/ref_bugs.py` repeats that + check in every run, and such a file is the new class **ref-bug** only + while its values keep changing. +- Result (`conformance/run.sh --no-fetch`, tank, 2026-09-27): 602 of 697 + ok (baseline 600), 0 our-error, 0 mismatch, 3 ref-bug, 92 + h5py-cannot-read, no panic/hang/crash/oom. + ### Deterministic errors on damaged chunked datasets (2026-09-27) - A read through the file's chunk cache listed a damaged dataset's chunks in hash-map order, seeded per `File`, so two opens of the same file could diff --git a/CONFORMANCE.md b/CONFORMANCE.md index ff5b581..a900e17 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -13,15 +13,15 @@ fatal. This file is generated by `conformance/run.sh`; do not edit it by hand. | | | |---|---| -| date | 2026-09-27 00:34 UTC | -| clawhdf5 commit | `f37e7ae3263277319dba4bc39be5397194eb00c3` | +| date | 2026-09-28 03:33 UTC | +| clawhdf5 commit | `ff0b2f8a4e4ef105d7ff4694349ca475abf3f611` | | machine | `tank`: AMD Ryzen 7 7800X3D 8-Core Processor, 16 CPUs, 61 GiB, Linux 7.0.0-34-generic x86_64 | -| command | `conformance/run.sh --no-fetch --update-baseline` | -| rustc | rustc 1.98.1 (48a229cea 2026-09-01) | +| command | `conformance/run.sh --no-fetch` | +| rustc | | | reference | h5py 3.16.0, HDF5 2.0.0, numpy 2.5.3, hdf5plugin 7.1.0, Python 3.14.4 | | h5dump | Version 1.14.6 (CVE corpus only) | | limits | 20 s timeout (SIGKILL), 4096 MiB address space, per process; 16 files in parallel | -| runtime | 21 s probing + comparing (0 s fetch/build before it) | +| runtime | 21 s probing + comparing (18 s fetch/build before it) | ## Results @@ -29,25 +29,24 @@ A file's class is the first that applies: - **panic / hang / crash / oom** — clawhdf5 panicked (caught per object or not), hit the timeout, died on a signal, or failed an allocation. The CI gate fails on any of these. - **h5py-cannot-read** — libhdf5 could not open the file (or itself crashed or hung). Nothing to compare against; most are the deliberately malformed CVE reproducers. +- **ref-bug** — every difference is an object clawhdf5 refuses that h5py reads only through a libhdf5 bug: the values h5py returns for it change with the reading process's heap, re-checked in every run (see *Reference bugs*). - **our-error** — clawhdf5 returned an error for something h5py reads. - **mismatch** — both read it, but the shapes, values, object set or attribute set differ. - **ok** — every object h5py reads, clawhdf5 reads identically. -| corpus | files | ok | our-error | mismatch | h5py-cannot-read | panic | hang | crash | oom | -|---|---|---|---|---|---|---|---|---|---| -| NCAS-CMS_pyfive | 33 | 32 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | -| cve_hdf5 | 147 | 113 | 2 | 0 | 32 | 0 | 0 | 0 | 0 | -| h5py_data | 4 | 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -| hdf5 | 466 | 404 | 1 | 1 | 60 | 0 | 0 | 0 | 0 | -| netcdf-c | 20 | 20 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -| netcdf4-python | 18 | 18 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -| usnistgov_h5wasm | 5 | 5 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -| xarray-data | 4 | 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -| **all** | **697** | **600** | **3** | **2** | **92** | **0** | **0** | **0** | **0** | +| corpus | files | ok | our-error | mismatch | h5py-cannot-read | ref-bug | panic | hang | crash | oom | +|---|---|---|---|---|---|---|---|---|---|---| +| NCAS-CMS_pyfive | 33 | 33 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +| cve_hdf5 | 147 | 113 | 0 | 0 | 32 | 2 | 0 | 0 | 0 | 0 | +| h5py_data | 4 | 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +| hdf5 | 466 | 405 | 0 | 0 | 60 | 1 | 0 | 0 | 0 | 0 | +| netcdf-c | 20 | 20 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +| netcdf4-python | 18 | 18 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +| usnistgov_h5wasm | 5 | 5 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +| xarray-data | 4 | 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | +| **all** | **697** | **602** | **0** | **0** | **92** | **3** | **0** | **0** | **0** | **0** | -2 of the 2 mismatches are a known h5py bug, not ours (see *Known not-our-bug*). - -3 of the 3 our-errors are corrupt data that HDF5 2.0 reads only through a bug and clawhdf5 refuses (see *Known not-our-bug*). +**Our errors and mismatches: 0.** Files not ok: 92 h5py-cannot-read, 3 ref-bug. 2 object(s) were compared against h5py's values corrected for a known h5py bug (2 identical to clawhdf5's; see *Reference bugs*). Corpora (fetched by `conformance/fetch-corpus.sh` into the gitignored `conformance/.cache/`): @@ -68,18 +67,11 @@ None. ## Our-error root causes -Grouped by normalised error message. *files* counts files whose class this cause affects. - -| files | objects | error | examples | -|---:|---:|---|---| -| 3 | 3 | `ChunkedReadError("…")` | `cve_hdf5/cvefiles/cve-2025-2308.h5`, `cve_hdf5/cvefiles/cve-2025-44904.h5`, `hdf5/test/testfiles/bad_nbit_parms_walk.h5` | +None. ## Mismatch root causes -| files | objects | cause | examples | -|---:|---:|---|---| -| 1 | 1 | `attr-values: ours=vlen(>u8) h5py=object layout=- filters=-` | `NCAS-CMS_pyfive/tests/data/attr_datatypes.hdf5` | -| 1 | 1 | `values: ours=vlen({r:>f4,i:>f4}8) h5py=object layout=contiguous filters=-` | `hdf5/tools/test/testfiles/tcomplex_be.h5` | +None. ## CVE corpus: clawhdf5 vs h5dump vs h5py @@ -203,7 +195,7 @@ columns are. | cvefiles/cve-2024-33876.h5 | ok | read 3 obj, 1 errors | read 3 obj, 1 errors | ok | | cvefiles/cve-2024-33877.h5 | error exit | read 8 obj, 1 errors | read 8 obj, 1 errors | ok | | cvefiles/cve-2025-2153.h5 | error exit | open error | read 1 obj, 1 errors | h5py-cannot-read | -| cvefiles/cve-2025-2308.h5 | error exit | read 25 obj, 1 errors | read 25 obj, 2 errors | our-error | +| cvefiles/cve-2025-2308.h5 | error exit | read 25 obj, 1 errors | read 25 obj, 2 errors | ref-bug | | cvefiles/cve-2025-2309.h5 | ok | read 6 obj, 1 errors | read 6 obj | ok | | cvefiles/cve-2025-2310.h5 | error exit | read 24 obj, 8 errors | read 24 obj, 8 errors | ok | | cvefiles/cve-2025-2912.h5 | error exit | open error | open error | h5py-cannot-read | @@ -214,7 +206,7 @@ columns are. | cvefiles/cve-2025-2924.h5 | error exit | read 1 obj, 1 errors | read 1 obj, 1 errors | ok | | cvefiles/cve-2025-2925.h5 | error exit | read 1 obj, 1 errors | read 1 obj, 1 errors | ok | | cvefiles/cve-2025-2926.h5 | error exit | open error | open error | h5py-cannot-read | -| cvefiles/cve-2025-44904.h5 | error exit | read 25 obj, 1 errors | read 25 obj, 2 errors | our-error | +| cvefiles/cve-2025-44904.h5 | error exit | read 25 obj, 1 errors | read 25 obj, 2 errors | ref-bug | | cvefiles/cve-2025-44905.h5 | error exit | read 25 obj, 3 errors | read 25 obj, 3 errors | ok | | cvefiles/cve-2025-6269-1.h5 | error exit | read 1 obj, 1 errors | read 1 obj, 1 errors | ok | | cvefiles/cve-2025-6269-2.h5 | error exit | read 1 obj, 1 errors | read 1 obj, 1 errors | ok | @@ -249,13 +241,36 @@ columns are. -## Known not-our-bug +## Reference bugs + +### Objects h5py reads only through a libhdf5 bug (*ref-bug*) + +clawhdf5 refuses these objects; h5py 3.16 / HDF5 2.0 returns values for them. `conformance/ref_bugs.py` +re-reads each with h5py in six fresh processes whose heaps differ (h5py imported before numpy, three +times and twice more with `MALLOC_PERTURB_`, and numpy imported first). Values the file determines +come out the same every time; these do not, so they are memory libhdf5 over-reads, not the file's +data. A file is *ref-bug* only while every one of its differences is such an object confirmed in +the same run; an object that reads the same every time goes back to *our-error*. Reproducer: +`python conformance/ref_bugs.py conformance/.cache/corpus` (prints every read's outcome). + +| file | object | distinct results in 6 reads | confirmed | what goes wrong | +|---|---|---:|---|---| +| `cve_hdf5/cvefiles/cve-2025-2308.h5` | `/Scale_offset_long_long_data_le` | 6 | yes | the first chunk records minbits 11: its 12 values need 17 bytes of codes, and the 26-byte chunk holds 5 after its 21-byte header; libhdf5's scale-offset decoder reads past its buffer, and develop refuses the chunk ("Buffer too short") | +| `cve_hdf5/cvefiles/cve-2025-44904.h5` | `/Scale_offset_float_data_le` | 6 | yes | unfiltered chunks stored as 38 and 37 bytes for 48-byte chunks: 1.14/2.0 read the stored bytes into a buffer of that size and use it as the whole chunk (H5D__chunk_lock), so the rest is heap memory; develop refuses them ("incorrect chunk size returned from index for unfiltered chunk") | +| `hdf5/test/testfiles/bad_nbit_parms_walk.h5` | `/Nbit_int_data_le` | 3 | yes | the N-Bit parameter list holds 7 values (cd_values[0] = 7) where an integer needs 8: the decoder takes the bit offset from cd_values[7], past the list; libhdf5's own test (`test_filter_bad_params`, test/dsets.c on develop) requires the read to fail | + +### Values corrected for a known h5py bug - **h5py big-endian variable-length sequences.** h5py returns the elements of a VL sequence whose base type is big-endian with the file's big-endian bytes but a native (little-endian) - numpy dtype, so the values it reports are byte-swapped garbage; `h5dump` prints the values - clawhdf5 reads. Reproducer: `h5py.vlen_dtype(np.dtype('>f4'))` dataset holding `[1.0, 2.0]` - reads back in h5py as `[4.6e-41, 9.0e-44]`. Affected here: `NCAS-CMS_pyfive/tests/data/attr_datatypes.hdf5`, `hdf5/tools/test/testfiles/tcomplex_be.h5`. + numpy dtype: a `h5py.vlen_dtype(np.dtype('>f4'))` dataset holding `[1.0, 2.0]` reads back as + `[4.6e-41, 9.0e-44]`; `h5dump` prints the file's values. `ref.py` checks that the installed + h5py still does this (by writing and reading exactly that dataset in memory) and, if so, + relabels such elements with the file's byte order before hashing, so the values are still + compared. Corrected objects: `NCAS-CMS_pyfive/tests/data/attr_datatypes.hdf5` `/@vlen_uint64` (same as clawhdf5), `hdf5/tools/test/testfiles/tcomplex_be.h5` `/VariableLengthDatasetFloatComplex` (same as clawhdf5). + +## Other comparison rules + - **Non-IEEE floats and partial-precision integers (N-Bit).** libhdf5 converts a float whose bit layout is not IEEE (e.g. `H5Tset_precision` for the N-Bit filter) or an integer with a bit offset / reduced precision into the plain numpy type of the same size. The probe @@ -264,11 +279,6 @@ columns are. - **Types h5py widens.** Where h5py reads a type into a numpy type of a different size (FP8 -> float16, bfloat16 -> float32, x87 long double -> float128) the values are not compared (shape and presence still are): dataset file type size 1 -> numpy float16 (2) (15x), attr file type size 1 -> numpy float16 (2) (15x), dataset file type size 2 -> numpy float32 (4) (2x), dataset file type size 8 -> numpy float128 (16) (1x), dataset file type size 12 -> numpy float128 (16) (1x), attr file type size 2 -> numpy float32 (4) (1x), dataset file type size 2 -> numpy >f4 (4) (1x), attr file type size 2 -> numpy >f4 (4) (1x). -- **Corrupt data HDF5 2.0 reads through a bug.** clawhdf5 refuses these objects; h5py 3.16 / - HDF5 2.0 returns values for them that the file does not hold: - - `cve_hdf5/cvefiles/cve-2025-2308.h5` `/Scale_offset_long_long_data_le`: scale-offset codes run past the end of the chunk: HDF5 2.0 reads past its buffer; libhdf5's develop branch refuses the chunk ("Buffer too short"). - - `cve_hdf5/cvefiles/cve-2025-44904.h5` `/Scale_offset_float_data_le`: unfiltered chunks of 38 and 37 bytes for 48-byte chunks: HDF5 2.0 fills the rest with whatever its buffer held; libhdf5's develop branch refuses them ("incorrect chunk size returned from index for unfiltered chunk"). - - `hdf5/test/testfiles/bad_nbit_parms_walk.h5` `/Nbit_int_data_le`: an N-Bit parameter list one value short: HDF5 2.0 reads past the list; libhdf5's own test (`test_filter_bad_params`, test/dsets.c) now requires the read to fail. - **References** are compared by presence only (`R`), not by target. ## Objects h5py fails on but clawhdf5 reads diff --git a/docs/known-issues.md b/docs/known-issues.md index 9e572a6..d87d371 100644 --- a/docs/known-issues.md +++ b/docs/known-issues.md @@ -66,7 +66,8 @@ as "not ours" but still counted against us. Re-checked on tank conformance run (six reads per object in differently set-up processes) and such a file is classified *ref-bug* only while its values keep changing. -Result (`conformance/run.sh --no-fetch`, 2026-09-27): see CHANGELOG. +Result (`conformance/run.sh --no-fetch`, tank, 2026-09-27): 602 of 697 ok +(was 600), 0 our-error, 0 mismatch, 3 ref-bug, 92 h5py-cannot-read. ## Files a SWMR writer had open could not be read past a stale end of file