osobhandClaude Opus 5.5
c27a478e44
Numbers, API names, feature defaults and PR references checked against CONFORMANCE.md, BENCHMARKS.md, CHANGELOG.md, the code and git history. - int8 index figures (1.74x memory, 1.63x QPS) carry the dates git gives them (2026-09-19/20, machine not recorded, not re-run) instead of none; the Pi 5 1.18x carries 2026-09-21. - BENCHMARKS headline: the libhdf5 chunked-write figure is the newest measurement (35x, 2026-09-23), not 45.3x (2026-08-03). - Conformance counts follow the 2026-09-28 run (1 our-error, 2 ref-bug) in conformance/README.md, ROADMAP.md and CLAUDE.md, with a pointer to the bad_nbit_parms_walk.h5 flip. - README: LZ4 is opt-in; the browser refuses reference/opaque/bitfield/ time datasets too; zlib-rs byte-identity scoped to what was measured; macOS default links the system libz for inflate. - Crate READMEs: system-zlib-decompress does something (macOS), SweepDetector lives in prefetch, checkpoint after more than 500 WAL entries, NetCDF-4 unlimited-dimension size warning. - agent-memory.md: string-dataset compression threshold, agents-md prints Markdown, float16 file sizes linked to their study. - known-issues.md: contiguous selection reads, 1.21x vs h5py threads. - docs/README.md, USE_CASES.md, ROADMAP.md, CLAUDE.md: range-read milestones M0-M5 and PRs #17-#19, missing README rows, CLI keygen/verify, dated figures, fast-math is not BLAS. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Conformance sweep
Reads every HDF5 file of eight public corpora with clawhdf5 and with
h5py/libhdf5, compares the two readings object by object, and writes
CONFORMANCE.md.
CLAWHDF5_PYTHON=/path/to/venv/bin/python conformance/run.sh # ~30 s once the corpus is cached
conformance/run.sh --no-fetch # use the cached corpus as is
conformance/run.sh --update-baseline # after an intended change in results
Latest result (tank, 2026-09-28 04:29 UTC, conformance/run.sh --no-fetch --update-baseline): 602 of 697 files ok, 1 our-error, 0 mismatch, 2
ref-bug, 92 h5py-cannot-read, and no panic, hang, crash or out-of-memory.
The our-error file is bad_nbit_parms_walk.h5, which flips between ref-bug
and our-error from run to run (see docs/known-issues.md). The report with every file is
CONFORMANCE.md.
Classes
compare.py puts each file in one class:
| class | meaning |
|---|---|
| ok | clawhdf5 and h5py read the same objects with the same values |
| our-error | h5py reads something clawhdf5 refuses |
| mismatch | both read it, with different values or structure |
| h5py-cannot-read | h5py (libhdf5) cannot read the file; not compared |
| ref-bug | h5py reads an object clawhdf5 refuses, but only through a libhdf5 over-read: ref_bugs.py re-reads it in six processes with different heaps (import order, MALLOC_PERTURB_) and its values change. The file is ref-bug only while that is confirmed in the same run; if the values become stable it counts as our-error again |
| panic / hang / crash / oom | a clawhdf5 failure under the timeout and address-space limit; the gate fails on any |
Where h5py itself returns wrong values through a known h5py bug (the
big-endian variable-length bug: elements returned with the file's bytes
under a little-endian dtype), ref.py checks that the installed h5py has
the bug, corrects the values before hashing and marks them ref_fix, so
those objects are still compared. The evidence for the three remaining
non-ok files (ref-bug or, for one, our-error) is under "Conformance: the last non-ok files" in
docs/known-issues.md.
Needs Rust, git, h5dump (Debian/Ubuntu hdf5-tools), libaec (for the
probe's szip feature; libaec-dev), and a Python with the packages in
requirements.txt. The first run downloads about 450 MB of sparse checkouts.
| file | role |
|---|---|
corpus.txt |
the corpora: git URL, pinned commit, swept root, sparse-checkout patterns |
fetch-corpus.sh |
shallow, sparse, blob-filtered checkout of each pinned commit into .cache/src/ (gitignored); no-op when already there |
list_files.py |
which files are probed (HDF5/netCDF-4 extensions minus netCDF classic, plus the CVE reproducers) |
probe/ |
the clawhdf5 side: a standalone crate (outside the workspace, so cargo test --workspace never builds it) that walks a file with clawhdf5-format and prints canonical JSON |
ref.py |
the h5py side: the same JSON from h5py (values corrected for a known h5py bug are marked ref_fix) |
ref_bugs.py |
re-reads the objects h5py reads only through a libhdf5 bug in six differently-set-up processes; an object whose values change is confirmed as a libhdf5 over-read |
test_ref.py |
tests of ref.py's correction and ref_bugs.py's confirmation (python conformance/test_ref.py) |
run_one.sh |
runs both sides on one file (and h5dump on the CVE corpus) under a timeout and an address-space limit |
compare.py |
classifies each file (ok / our-error / mismatch / h5py-cannot-read / ref-bug / panic / hang / crash / oom) and groups root causes |
report.py |
writes CONFORMANCE.md |
check.py |
the gate: fails on any panic/hang/crash/oom, on an ok count below baseline.json, or on a baseline-ok file that is no longer ok |
baseline.json |
the ok files the gate holds the line on |
requirements.txt |
pinned h5py / numpy / hdf5plugin / netCDF4 |
Results for every file (both sides' JSON and stderr, results.csv,
results.json, summary.md) are left in .cache/results/.
The nightly job is .gitea/workflows/conformance.yml; it prints the report
into the job log.
The canonical value encoding both sides hash is documented at the top of
probe/src/main.rs. Values are compared as libhdf5 presents them: a float
with a non-IEEE bit layout (N-Bit) or an integer with a bit offset is compared
as the converted number, not as raw file bytes.