quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(format): read a v1 chunk B-tree where libhdf5's lookup finds chunks

Browse Source 
libhdf5 does not walk the chunk B-tree to read a dataset: it looks each
chunk up (H5B_find with H5D__btree_cmp3 and H5D__btree_found), asking for
the element-size coordinate as 0. collect_chunk_info_checked now parses
the tree with its keys and returns each stored chunk only when that
lookup, replayed over the scaled keys, finds it.

A key with a non-zero element-size coordinate is therefore found in a
1-D dataset (cmp3 compares only the first coordinate there, and found
compares with <=) and missed in a dataset of rank 2 or more, which reads
fill values. The previous commit refused every such key, which refused
1-D files libhdf5 reads correctly; before that, the rank-2 case read the
chunk's data where h5py reads fill values (cve-2025-44905
/Shuffle_float_data_le, now identical to h5py, so it leaves the
conformance report's list of libhdf5 bugs).

Test: chunk_keys_with_an_element_offset_read_as_libhdf5_reads_them
compares 1-D and 2-D files against h5py's values. It fails on the
previous commit (the 1-D file is refused) and with the refusal removed
(the 2-D file reads 0..23 where h5py reads fill values).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 11:35:13 -05:00
co-authored by Claude Opus 5.5
parent 378afa1584
commit 742ed4dfb8
4 changed files with 322 additions and 125 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conformance/report.py
+2 -5
View File
@@ -100,8 +100,8 @@ def is_h5py_be_vlen(i):
# Objects the reference (h5py 3.16 / HDF5 2.0) reads only because of an
# HDF5 2.0 bug, and that clawhdf5 refuses: each one reads past a buffer or
# returns bytes the file does not hold, and libhdf5's develop branch refuses the
# first three. (file, object) -> why. Checked 2026-09-26 against HDF5 2.0.0
# returns bytes the file does not hold, and libhdf5's develop branch refuses all
# three. (file, object) -> why. Checked 2026-09-26 against HDF5 2.0.0
# and HDFGroup/hdf5 develop sources; see docs/known-issues.md.
LIBHDF5_BUGS = {
("cve_hdf5/cvefiles/cve-2025-2308.h5", "/Scale_offset_long_long_data_le"):
@@ -114,9 +114,6 @@ LIBHDF5_BUGS = {
("hdf5/test/testfiles/bad_nbit_parms_walk.h5", "/Nbit_int_data_le"):
"an N-Bit parameter list one value short: HDF5 2.0 reads past the list; libhdf5's own "
"test (`test_filter_bad_params`, test/dsets.c) now requires the read to fail",
("cve_hdf5/cvefiles/cve-2025-44905.h5", "/Shuffle_float_data_le"):
"a chunk B-tree key with element offset 4096: libhdf5's lookup misses the chunk and "
"returns fill values for data the file holds; clawhdf5 refuses the key",
}