quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
14790487a75bb64fa13980247a066f54ec7c2581
clawhdf5/CONFORMANCE.md
T
osobhandClaude Opus 5.5 694ee0a090
CI / test-arm64 (pull_request) Successful in 1m22s
Details
CI / test (pull_request) Successful in 15m34s
Details
docs: conformance report after the last non-ok files were classified (602 of 697 ok) 
Regenerated on tank: ok 600 -> 602 (attr_datatypes.hdf5 and
tcomplex_be.h5, compared against h5py's big-endian VL values corrected),
mismatch 2 -> 0; cve-2025-2308.h5 and cve-2025-44904.h5 are ref-bug
(h5py's values varied across six heaps in this run);
bad_nbit_parms_walk.h5 read the same in all six this run, so it stays
our-error, as the classification rule requires. Baseline raised.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-27 23:39:41 -05:00

23 KiB
Raw Blame History

clawhdf5 conformance report

Every HDF5 file of eight public corpora (pinned by commit) is read twice — by clawhdf5 (conformance/probe, the same clawhdf5-format calls the facade makes) and by h5py/libhdf5 (conformance/ref.py) — and the two readings are compared object by object: the set of hard-linked objects, each dataset's and attribute's shape, and a SHA-256 of its values in a canonical encoding. The CVE corpus is also run through h5dump. Each side runs under a timeout and an address-space limit, so a hang, crash or runaway allocation is recorded, not fatal. This file is generated by conformance/run.sh; do not edit it by hand.

Run
date 2026-09-28 04:29 UTC
clawhdf5 commit bf5a163dcf7fe28d651ada6545d8136ffeffc825
machine tank: AMD Ryzen 7 7800X3D 8-Core Processor, 16 CPUs, 61 GiB, Linux 7.0.0-34-generic x86_64
command conformance/run.sh --no-fetch --update-baseline
rustc rustc 1.98.1 (48a229cea 2026-09-01)
reference h5py 3.16.0, HDF5 2.0.0, numpy 2.5.3, hdf5plugin 7.1.0, Python 3.14.4
h5dump Version 1.14.6 (CVE corpus only)
limits 20 s timeout (SIGKILL), 4096 MiB address space, per process; 16 files in parallel
runtime 20 s probing + comparing (5 s fetch/build before it)

Results

A file's class is the first that applies:

  • panic / hang / crash / oom — clawhdf5 panicked (caught per object or not), hit the timeout, died on a signal, or failed an allocation. The CI gate fails on any of these.
  • h5py-cannot-read — libhdf5 could not open the file (or itself crashed or hung). Nothing to compare against; most are the deliberately malformed CVE reproducers.
  • ref-bug — every difference is an object clawhdf5 refuses that h5py reads only through a libhdf5 bug: the values h5py returns for it change with the reading process's heap, re-checked in every run (see Reference bugs).
  • our-error — clawhdf5 returned an error for something h5py reads.
  • mismatch — both read it, but the shapes, values, object set or attribute set differ.
  • ok — every object h5py reads, clawhdf5 reads identically.
corpus files ok our-error mismatch h5py-cannot-read ref-bug panic hang crash oom
NCAS-CMS_pyfive 33 33 0 0 0 0 0 0 0 0
cve_hdf5 147 113 0 0 32 2 0 0 0 0
h5py_data 4 4 0 0 0 0 0 0 0 0
hdf5 466 405 1 0 60 0 0 0 0 0
netcdf-c 20 20 0 0 0 0 0 0 0 0
netcdf4-python 18 18 0 0 0 0 0 0 0 0
usnistgov_h5wasm 5 5 0 0 0 0 0 0 0 0
xarray-data 4 4 0 0 0 0 0 0 0 0
all 697 602 1 0 92 2 0 0 0 0

Our errors and mismatches: 1. Files not ok: 1 our-error, 92 h5py-cannot-read, 2 ref-bug. 2 object(s) were compared against h5py's values corrected for a known h5py bug (2 identical to clawhdf5's; see Reference bugs).

Corpora (fetched by conformance/fetch-corpus.sh into the gitignored conformance/.cache/):

corpus source commit
hdf5 https://github.com/HDFGroup/hdf5 a3cf1ea82cc7
cve_hdf5 https://github.com/HDFGroup/cve_hdf5 3fd1f5ae3869
netcdf-c https://github.com/Unidata/netcdf-c beb7b9585273
NCAS-CMS_pyfive https://github.com/NCAS-CMS/pyfive 8cf07b874913
usnistgov_h5wasm https://github.com/usnistgov/h5wasm 02f6336527d2
netcdf4-python https://github.com/Unidata/netcdf4-python 6e67576d39ae
xarray-data https://github.com/pydata/xarray-data a35297e9da2c
h5py_data https://github.com/h5py/h5py (h5py/tests/data_files) b2f0347c4200

Panics, hangs, crashes, out-of-memory

None.

Our-error root causes

Grouped by normalised error message. files counts files whose class this cause affects.

files objects error examples
1 1 ChunkedReadError("…") hdf5/test/testfiles/bad_nbit_parms_walk.h5

Mismatch root causes

None.

CVE corpus: clawhdf5 vs h5dump vs h5py

The 147 files of HDFGroup/cve_hdf5 — reproducers for published libhdf5 CVEs and fuzzer finds. read = produced output (possibly with per-object errors), error = refused cleanly. h5dump exits non-zero on any error anywhere in a file, so its read/error split is not comparable with the other two rows; the panic, crash, hang and oom columns are.

tool read error panic crash hang oom
clawhdf5 121 26 0 0 0 0
h5dump 1.14.6 16 129 0 2 0 0
h5py 3.16.0 / HDF5 2.0.0 115 31 0 1 0 0
Per-file outcomes
file h5dump h5py clawhdf5 class
cvefiles/cve-2016-4330.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2016-4331.h5 error exit read 25 obj, 1 errors read 25 obj, 1 errors ok
cvefiles/cve-2016-4332-mtime-new.h5 error exit read 25 obj, 1 errors read 25 obj, 1 errors ok
cvefiles/cve-2016-4332-mtime.h5 error exit read 4 obj, 3 errors read 4 obj, 3 errors ok
cvefiles/cve-2016-4332-stab.h5 error exit open error read 1 obj, 1 errors h5py-cannot-read
cvefiles/cve-2016-4333.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2017-17505.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2017-17506.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2017-17507.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2017-17508.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2017-17509.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2018-11202.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2018-11203.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2018-11204.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2018-11205.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2018-11206-new.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-11206-old.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-11207.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2018-13866.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2018-13867.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2018-13868.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-13869.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2018-13870.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2018-13871.h5 error exit read 2 obj read 2 obj ok
cvefiles/cve-2018-13872.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2018-13873.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2018-13874.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2018-13875.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-13876.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2018-14031.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-14033.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-14034.h5 error exit read 1 obj, 2 errors read 1 obj ok
cvefiles/cve-2018-14035.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-14460.h5 error exit read 3 obj, 2 errors read 3 obj, 2 errors ok
cvefiles/cve-2018-15671.h5 ok read 1 obj read 1 obj ok
cvefiles/cve-2018-15672.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2018-16438.h5 error exit read 1 obj, 1 errors read 1 obj ok
cvefiles/cve-2018-17233.h5 error exit read 6 obj, 1 errors read 6 obj, 1 errors ok
cvefiles/cve-2018-17234.h5 error exit read 6 obj, 1 errors read 6 obj, 1 errors ok
cvefiles/cve-2018-17237.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2018-17432.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-17433 error exit open error open error h5py-cannot-read
cvefiles/cve-2018-17434.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-17435.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-17436 error exit open error open error h5py-cannot-read
cvefiles/cve-2018-17437.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-17438 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2018-17439 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2019-8396.h5 error exit read 3 obj, 2 errors read 3 obj, 2 errors ok
cvefiles/cve-2019-8397.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2019-8398.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2019-9151.h5 error exit read 3 obj, 1 errors read 3 obj, 2 errors ok
cvefiles/cve-2019-9152.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2020-10809 error exit open error open error h5py-cannot-read
cvefiles/cve-2020-10810.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2020-10811.h5 error exit read 25 obj, 1 errors read 25 obj, 1 errors ok
cvefiles/cve-2020-10812.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2020-18232.h5 error exit read 3 obj, 2 errors read 3 obj, 2 errors ok
cvefiles/cve-2020-18494.h5 ok read 2 obj read 2 obj ok
cvefiles/cve-2021-36977.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2021-37501.h5 error exit read 18 obj, 1 errors read 18 obj, 1 errors ok
cvefiles/cve-2021-45829.h5 error exit read 1 obj, 2 errors read 1 obj, 1 errors ok
cvefiles/cve-2021-45830.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2021-45833.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2021-46242.h5 error exit open error read 1 obj, 1 errors h5py-cannot-read
cvefiles/cve-2021-46243.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2021-46244.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2024-29157.h5 error exit read 4 obj, 7 errors read 4 obj, 7 errors ok
cvefiles/cve-2024-29158.h5 ok read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2024-29159.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2024-29160.h5 error exit read 4 obj, 1 errors read 4 obj, 1 errors ok
cvefiles/cve-2024-29161.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2024-29162.h5 error exit read 17 obj, 4 errors read 17 obj, 4 errors ok
cvefiles/cve-2024-29163.h5 error exit read 7 obj, 1 errors read 7 obj, 1 errors ok
cvefiles/cve-2024-29164.h5 ok read 3 obj read 3 obj ok
cvefiles/cve-2024-29165.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2024-29166.h5 error exit read 17 obj, 2 errors read 17 obj ok
cvefiles/cve-2024-32605.h5 ok read 6 obj, 1 errors read 6 obj, 1 errors ok
cvefiles/cve-2024-32606.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2024-32607-1.h5 ok read 10 obj read 10 obj ok
cvefiles/cve-2024-32607-2.h5 error exit read 9 obj, 1 errors read 9 obj, 1 errors ok
cvefiles/cve-2024-32608.h5 error exit read 6 obj, 1 errors read 6 obj, 1 errors ok
cvefiles/cve-2024-32609.h5 error exit SIGSEGV read 3 obj, 1 errors h5py-cannot-read
cvefiles/cve-2024-32610.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2024-32611.h5 ok read 6 obj read 6 obj ok
cvefiles/cve-2024-32612.h5 ok read 3 obj read 3 obj ok
cvefiles/cve-2024-32613.h5 error exit read 7 obj, 1 errors read 7 obj, 1 errors ok
cvefiles/cve-2024-32614.h5 error exit read 25 obj, 2 errors read 25 obj, 2 errors ok
cvefiles/cve-2024-32615.h5 error exit read 4 obj, 1 errors read 4 obj, 1 errors ok
cvefiles/cve-2024-32616.h5 error exit read 10 obj, 7 errors read 10 obj, 6 errors ok
cvefiles/cve-2024-32617.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2024-32618.h5 error exit read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2024-32619.h5 error exit read 3 obj, 2 errors read 3 obj, 2 errors ok
cvefiles/cve-2024-32620.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2024-32621.h5 ok read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2024-32622.h5 ok read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2024-32623.h5 ok read 6 obj read 6 obj ok
cvefiles/cve-2024-32624.h5 error exit read 6 obj, 1 errors read 6 obj, 1 errors ok
cvefiles/cve-2024-33873.h5 error exit read 4 obj, 1 errors read 4 obj, 1 errors ok
cvefiles/cve-2024-33874.h5 ok read 6 obj, 1 errors read 6 obj, 1 errors ok
cvefiles/cve-2024-33875.h5 ok read 2 obj read 2 obj ok
cvefiles/cve-2024-33876.h5 ok read 3 obj, 1 errors read 3 obj, 1 errors ok
cvefiles/cve-2024-33877.h5 error exit read 8 obj, 1 errors read 8 obj, 1 errors ok
cvefiles/cve-2025-2153.h5 error exit open error read 1 obj, 1 errors h5py-cannot-read
cvefiles/cve-2025-2308.h5 error exit read 25 obj, 1 errors read 25 obj, 2 errors ref-bug
cvefiles/cve-2025-2309.h5 ok read 6 obj, 1 errors read 6 obj ok
cvefiles/cve-2025-2310.h5 error exit read 24 obj, 8 errors read 24 obj, 8 errors ok
cvefiles/cve-2025-2912.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-2913.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-2914.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-2915.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-2923.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-2924.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-2925.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-2926.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-44904.h5 error exit read 25 obj, 1 errors read 25 obj, 2 errors ref-bug
cvefiles/cve-2025-44905.h5 error exit read 25 obj, 3 errors read 25 obj, 3 errors ok
cvefiles/cve-2025-6269-1.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-6269-2.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-6269-3.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-6269-4.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-6270-1.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-6270-2.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-6270-3.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-6516.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-6750.h5 error exit open error read 1 obj, 1 errors h5py-cannot-read
cvefiles/cve-2025-6816.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-6817.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-6818.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-6856.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-6857.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-6858.h5 SIGSEGV open error open error h5py-cannot-read
cvefiles/cve-2025-7067.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2025-7068.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2025-7069.h5 error exit open error open error h5py-cannot-read
cvefiles/cve-2026-26200.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
cvefiles/cve-2026-34734.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/cve-2026-92627.h5 error exit read 2 obj, 1 errors read 2 obj, 1 errors ok
cvefiles/unknown-1.h5 error exit read 11 obj, 1 errors read 11 obj, 1 errors ok
fuzzerfiles/gh-4431-poc-03.h5 error exit read 1 obj read 1 obj ok
fuzzerfiles/gh-4432-poc-05.h5 SIGSEGV read 1 obj, 1 errors read 1 obj, 1 errors ok
fuzzerfiles/gh-4433-poc-08.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
fuzzerfiles/gh-4434-poc-09.h5 error exit open error read 1 obj, 1 errors h5py-cannot-read
fuzzerfiles/gh-4435-poc-10.h5 error exit read 1 obj, 1 errors read 1 obj, 1 errors ok
fuzzerfiles/gh-4585.h5 error exit open error open error h5py-cannot-read
fuzzerfiles/gh_2649_flawed.h5 error exit read 9 obj, 1 errors read 9 obj, 1 errors ok
fuzzerfiles/gh_2649_plain_model.h5 ok read 10 obj read 10 obj ok

Reference bugs

Objects h5py reads only through a libhdf5 bug (ref-bug)

clawhdf5 refuses these objects; h5py 3.16 / HDF5 2.0 returns values for them. conformance/ref_bugs.py re-reads each with h5py in six fresh processes whose heaps differ (h5py imported before numpy, three times and twice more with MALLOC_PERTURB_, and numpy imported first). Values the file determines come out the same every time; these do not, so they are memory libhdf5 over-reads, not the file's data. A file is ref-bug only while every one of its differences is such an object confirmed in the same run; an object that reads the same every time goes back to our-error. Reproducer: python conformance/ref_bugs.py conformance/.cache/corpus (prints every read's outcome).

file object distinct results in 6 reads confirmed what goes wrong
cve_hdf5/cvefiles/cve-2025-2308.h5 /Scale_offset_long_long_data_le 6 yes the first chunk records minbits 11: its 12 values need 17 bytes of codes, and the 26-byte chunk holds 5 after its 21-byte header; libhdf5's scale-offset decoder reads past its buffer, and develop refuses the chunk ("Buffer too short")
cve_hdf5/cvefiles/cve-2025-44904.h5 /Scale_offset_float_data_le 6 yes unfiltered chunks stored as 38 and 37 bytes for 48-byte chunks: 1.14/2.0 read the stored bytes into a buffer of that size and use it as the whole chunk (H5D__chunk_lock), so the rest is heap memory; develop refuses them ("incorrect chunk size returned from index for unfiltered chunk")
hdf5/test/testfiles/bad_nbit_parms_walk.h5 /Nbit_int_data_le 1 no the N-Bit parameter list holds 7 values (cd_values[0] = 7) where an integer needs 8: the decoder takes the bit offset from cd_values[7], past the list; libhdf5's own test (test_filter_bad_params, test/dsets.c on develop) requires the read to fail

Values corrected for a known h5py bug

  • h5py big-endian variable-length sequences. h5py returns the elements of a VL sequence whose base type is big-endian with the file's big-endian bytes but a native (little-endian) numpy dtype: a h5py.vlen_dtype(np.dtype('>f4')) dataset holding [1.0, 2.0] reads back as [4.6e-41, 9.0e-44]; h5dump prints the file's values. ref.py checks that the installed h5py still does this (by writing and reading exactly that dataset in memory) and, if so, relabels such elements with the file's byte order before hashing, so the values are still compared. Corrected objects: NCAS-CMS_pyfive/tests/data/attr_datatypes.hdf5 /@vlen_uint64 (same as clawhdf5), hdf5/tools/test/testfiles/tcomplex_be.h5 /VariableLengthDatasetFloatComplex (same as clawhdf5).

Other comparison rules

  • Non-IEEE floats and partial-precision integers (N-Bit). libhdf5 converts a float whose bit layout is not IEEE (e.g. H5Tset_precision for the N-Bit filter) or an integer with a bit offset / reduced precision into the plain numpy type of the same size. The probe compares such values as converted numbers, not raw file bytes (before 2026-09-25 it compared raw bytes, which reported every N-Bit float dataset as a mismatch).
  • Types h5py widens. Where h5py reads a type into a numpy type of a different size (FP8 -> float16, bfloat16 -> float32, x87 long double -> float128) the values are not compared (shape and presence still are): dataset file type size 1 -> numpy float16 (2) (15x), attr file type size 1 -> numpy float16 (2) (15x), dataset file type size 2 -> numpy float32 (4) (2x), dataset file type size 8 -> numpy float128 (16) (1x), dataset file type size 12 -> numpy float128 (16) (1x), attr file type size 2 -> numpy float32 (4) (1x), dataset file type size 2 -> numpy >f4 (4) (1x), attr file type size 2 -> numpy >f4 (4) (1x).
  • References are compared by presence only (R), not by target.

Objects h5py fails on but clawhdf5 reads

  • 19 x OSError: Can't synchronously read data (no appropriate function for conversion path)
  • 1 x TypeError: unhandled dtype kind M (dtype('…'))
  • 1 x TypeError: No NumPy equivalent for TypeTimeID exists
  • 1 x ValueError: Insufficient precision in available types to represent (N, N, N, N, N)

Reproduce

# needs: Rust, python3 with h5py numpy hdf5plugin (conformance/requirements.txt), h5dump (hdf5-tools), git
CLAWHDF5_PYTHON=/path/to/venv/bin/python conformance/run.sh

The corpus (about 450 MB of sparse checkouts) is cached in conformance/.cache/; results for every file, both sides' raw JSON and stderr, are in conformance/.cache/results/. conformance/baseline.json holds the ok files the nightly CI job (.gitea/workflows/conformance.yml) must keep; conformance/run.sh --update-baseline rewrites it.

Reference in New Issue View Git Blame Copy Permalink