quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs: refresh README, crate READMEs and reference docs; fact-check every claim #22

Merged
osobh merged 21 commits from docs/readme-refresh into main 2026-09-28 17:24:26 +00:00
Conversation 0 Commits 21 Files Changed 47
+27
1 changed files with 27 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit cab952bb00 - Show all commits
conformance/README.md
+27
View File
@@ -6,9 +6,36 @@ h5py/libhdf5, compares the two readings object by object, and writes
```sh ```sh
CLAWHDF5_PYTHON=/path/to/venv/bin/python conformance/run.sh # ~30 s once the corpus is cached CLAWHDF5_PYTHON=/path/to/venv/bin/python conformance/run.sh # ~30 s once the corpus is cached
conformance/run.sh --no-fetch # use the cached corpus as is
conformance/run.sh --update-baseline # after an intended change in results conformance/run.sh --update-baseline # after an intended change in results
``` ```
Latest result (tank, 2026-09-27, `conformance/run.sh --no-fetch`): 602 of
697 files ok, 0 our-error, 0 mismatch, 3 ref-bug, 92 h5py-cannot-read, and
no panic, hang, crash or out-of-memory. The report with every file is
[`CONFORMANCE.md`](../CONFORMANCE.md).
## Classes
`compare.py` puts each file in one class:
| class | meaning |
|---|---|
| **ok** | clawhdf5 and h5py read the same objects with the same values |
| **our-error** | h5py reads something clawhdf5 refuses |
| **mismatch** | both read it, with different values or structure |
| **h5py-cannot-read** | h5py (libhdf5) cannot read the file; not compared |
| **ref-bug** | h5py reads an object clawhdf5 refuses, but only through a libhdf5 over-read: `ref_bugs.py` re-reads it in six processes with different heaps (import order, `MALLOC_PERTURB_`) and its values change. The file is ref-bug only while that is confirmed in the same run; if the values become stable it counts as our-error again |
| **panic / hang / crash / oom** | a clawhdf5 failure under the timeout and address-space limit; the gate fails on any |
Where h5py itself returns wrong values through a known h5py bug (the
big-endian variable-length bug: elements returned with the file's bytes
under a little-endian dtype), `ref.py` checks that the installed h5py has
the bug, corrects the values before hashing and marks them `ref_fix`, so
those objects are still compared. The evidence for the three current
ref-bug files is under "Conformance: the last non-ok files" in
[`docs/known-issues.md`](../docs/known-issues.md).
Needs Rust, `git`, `h5dump` (Debian/Ubuntu `hdf5-tools`), `libaec` (for the Needs Rust, `git`, `h5dump` (Debian/Ubuntu `hdf5-tools`), `libaec` (for the
probe's `szip` feature; `libaec-dev`), and a Python with the packages in probe's `szip` feature; `libaec-dev`), and a Python with the packages in
`requirements.txt`. The first run downloads about 450 MB of sparse checkouts. `requirements.txt`. The first run downloads about 450 MB of sparse checkouts.