quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases 1 Wiki Activity

feat: check HDF5 2.x small floats against libhdf5 2.2.0
CI / test-arm64 (pull_request) Successful in 1m38s
Details
CI / test (pull_request) Successful in 31m22s
Details

Browse Source 
Fixture written by libhdf5 2.2.0 (built from tag 2.2.0) through ctypes:
every bit pattern of FP4 E2M1, FP6 E2M3/E3M2, FP8 E4M3/E5M2 and a
bfloat16 LE/BE set, as datasets and attributes, with what H5Dread/H5Aread
return into double and float and the conversion exceptions libhdf5
raises. clawhdf5 already decoded every value as libhdf5 does, including
an all-ones exponent as inf/NaN in the OCP formats that have none
(documented as a deliberate match in known-issues).

- data_read: NaNs of non-native float layouts get libhdf5's bits (sign
  kept, every mantissa bit set) in f64 and f32.
- h5rs dump/ls name these types as h5dump/h5ls 2.x do
  (H5T_FLOAT_F4E2M1, "FP4 E2M1 4-bit float", float4-e2m1 ...), checked
  against h5dump 2.2.0's output of the fixture.
- Python bindings read them as h5py 3.16 does (float32 for bfloat16,
  float16 for the 1-byte formats, file byte order, same bytes as h5py);
  writing them in 'r+' is refused.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-28 21:25:03 -05:00
co-authored by Claude Opus 5.5
parent 31efac1ae1
commit bce07e9cb9
16 changed files with 1207 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/clawhdf5-tools/README.md
+10 -1
View File
@@ -109,7 +109,16 @@ virtual datasets. Known differences from
h5dump:
- Floats print at their own precision (a `float32` 0.1 prints as `0.1`),
which for some values is more digits than h5dump's `%g`.
which for some values is more digits than h5dump's `%g`; non-finite
values print as `Inf`, `-Inf` and `NaN` (h5dump: `inf`, `-inf`, `nan`,
`-nan`).
- HDF5 2.x's predefined small floats print under h5dump 2.x's names
(`H5T_FLOAT_BFLOAT16LE`, `H5T_FLOAT_F8E4M3`, ..., `H5T_FLOAT_F4E2M1`;
`ls -v` as h5ls 2.x does: `FP4 E2M1 4-bit float`).
`tests/mx_floats_dump.rs` checks this, and the values, against the output
of h5dump 2.2.0 stored with the fixture. h5dump 1.14 describes these types
instead (`8-bit floating-point 4-bit precision`); other non-IEEE floats
print as an `H5T_FLOAT { ... }` block.
- A compound nested in a compound prints inline (`{ 1, 2.5 }`) where
h5dump prints it as an indented block, one member per line; only the
outer compound is a block.