osobhandClaude Opus 5.5
5c44630ea2
The last 3 our-errors and 2 mismatches were documented as not ours but
still counted against us, on a heuristic (any big-endian VL mismatch) and
a fixed list.
- ref.py checks that the installed h5py returns big-endian VL elements
with the file's bytes under a little-endian dtype (writing and reading
a vlen('>f4') in memory) and, if so, relabels them with the file's byte
order before hashing, marking the object `ref_fix`. The values are now
compared: attr_datatypes.hdf5 /@vlen_uint64 and tcomplex_be.h5
/VariableLengthDatasetFloatComplex are identical to ours (h5dump 1.14.6
prints the same (1, 2), (3, 4, 5), (42)).
- ref_bugs.py re-reads each object h5py reads only through a libhdf5 bug
in six processes with different heaps (import order, MALLOC_PERTURB_).
Values the file determines are the same every time; these three change
(6, 6 and 3 distinct results), so they are over-read memory, not data
clawhdf5 could match. compare.py classifies a file `ref-bug` only when
every difference is such an object confirmed in the same run.
- report.py: the ref-bug class, the evidence table, the corrected
objects; test_ref.py covers both (run in the nightly job).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
59 lines
2.7 KiB
YAML
59 lines
2.7 KiB
YAML
name: Conformance
|
|
# Nightly: read every file of the pinned public HDF5 corpora with clawhdf5 and
|
|
# with h5py/libhdf5 and compare (conformance/run.sh; CONFORMANCE.md explains
|
|
# the method). Fails on any panic, hang, crash or out-of-memory in clawhdf5,
|
|
# and when the ok count drops below conformance/baseline.json or a file the
|
|
# baseline lists as ok stops being ok. The report is printed into the job log;
|
|
# nothing is uploaded (artifact actions are JavaScript, which rust:latest
|
|
# cannot run — see CLAUDE.md).
|
|
on:
|
|
schedule:
|
|
- cron: "17 3 * * *"
|
|
workflow_dispatch:
|
|
jobs:
|
|
conformance:
|
|
runs-on: ubuntu-latest
|
|
container: rust:latest
|
|
timeout-minutes: 60
|
|
env:
|
|
CARGO_NET_RETRY: "10"
|
|
steps:
|
|
# Plain git, not actions/checkout (a JavaScript action; see ci.yml).
|
|
- name: Check out
|
|
run: |
|
|
git init -q .
|
|
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
|
for i in 1 2 3; do git fetch -q --depth 1 origin "${GITHUB_SHA}" && break; sleep 5; done
|
|
git checkout -q FETCH_HEAD
|
|
- name: Install h5py, h5dump and the probe's codec libraries
|
|
# hdf5-tools: h5dump for the CVE-corpus comparison. libaec-dev and
|
|
# pkg-config: the probe builds clawhdf5-format with `szip` (the core
|
|
# crates' default build needs neither).
|
|
run: |
|
|
apt-get update
|
|
apt-get install -y --no-install-recommends python3 python3-venv hdf5-tools libaec-dev pkg-config
|
|
python3 -m venv /opt/conformance
|
|
/opt/conformance/bin/pip install --no-cache-dir -r conformance/requirements.txt
|
|
/opt/conformance/bin/python -c "import h5py, hdf5plugin; print('h5py', h5py.__version__, 'HDF5', h5py.version.hdf5_version, 'hdf5plugin', hdf5plugin.version)"
|
|
h5dump --version
|
|
- name: Probe unit tests
|
|
run: cargo test --release --manifest-path conformance/probe/Cargo.toml
|
|
env:
|
|
CARGO_TARGET_DIR: conformance/.cache/target
|
|
- name: Reference-side tests
|
|
run: /opt/conformance/bin/python conformance/test_ref.py
|
|
- name: Sweep
|
|
# The corpora come from GitHub (pinned commits, conformance/corpus.txt),
|
|
# so this job needs a runner that reaches github.com.
|
|
env:
|
|
CLAWHDF5_PYTHON: /opt/conformance/bin/python
|
|
run: bash conformance/run.sh
|
|
- name: Report
|
|
if: always()
|
|
run: |
|
|
if [ -f CONFORMANCE.md ]; then cat CONFORMANCE.md; else echo "no report was generated"; fi
|
|
if [ -f conformance/.cache/results/summary.md ]; then
|
|
echo; echo "---- per-file detail (conformance/.cache/results/summary.md) ----"
|
|
cat conformance/.cache/results/summary.md
|
|
fi
|