quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs(conformance): file classes, ref-bug and ref_fix, latest result

Browse Source 
Explains each class compare.py assigns, how ref_bugs.py confirms a
ref-bug in every run and how ref.py corrects h5py's big-endian VL values
(ref_fix), and records the latest result (602 of 697 ok, 3 ref-bug,
2026-09-27) with a link to CONFORMANCE.md. Mentions --no-fetch.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-28 11:13:30 -05:00
co-authored by Claude Opus 5.5
parent 999cb86071
commit cab952bb00
1 changed files with 27 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conformance/README.md
+27
View File
@@ -6,9 +6,36 @@ h5py/libhdf5, compares the two readings object by object, and writes
```sh
CLAWHDF5_PYTHON=/path/to/venv/bin/python conformance/run.sh # ~30 s once the corpus is cached
conformance/run.sh --no-fetch # use the cached corpus as is
conformance/run.sh --update-baseline # after an intended change in results
```
Latest result (tank, 2026-09-27, `conformance/run.sh --no-fetch`): 602 of
697 files ok, 0 our-error, 0 mismatch, 3 ref-bug, 92 h5py-cannot-read, and
no panic, hang, crash or out-of-memory. The report with every file is
[`CONFORMANCE.md`](../CONFORMANCE.md).
## Classes
`compare.py` puts each file in one class:
| class | meaning |
|---|---|
| **ok** | clawhdf5 and h5py read the same objects with the same values |
| **our-error** | h5py reads something clawhdf5 refuses |
| **mismatch** | both read it, with different values or structure |
| **h5py-cannot-read** | h5py (libhdf5) cannot read the file; not compared |
| **ref-bug** | h5py reads an object clawhdf5 refuses, but only through a libhdf5 over-read: `ref_bugs.py` re-reads it in six processes with different heaps (import order, `MALLOC_PERTURB_`) and its values change. The file is ref-bug only while that is confirmed in the same run; if the values become stable it counts as our-error again |
| **panic / hang / crash / oom** | a clawhdf5 failure under the timeout and address-space limit; the gate fails on any |
Where h5py itself returns wrong values through a known h5py bug (the
big-endian variable-length bug: elements returned with the file's bytes
under a little-endian dtype), `ref.py` checks that the installed h5py has
the bug, corrects the values before hashing and marks them `ref_fix`, so
those objects are still compared. The evidence for the three current
ref-bug files is under "Conformance: the last non-ok files" in
[`docs/known-issues.md`](../docs/known-issues.md).
Needs Rust, `git`, `h5dump` (Debian/Ubuntu `hdf5-tools`), `libaec` (for the
probe's `szip` feature; `libaec-dev`), and a Python with the packages in
`requirements.txt`. The first run downloads about 450 MB of sparse checkouts.