fix(format): read a v1 chunk B-tree where libhdf5's lookup finds chunks

libhdf5 does not walk the chunk B-tree to read a dataset: it looks each
chunk up (H5B_find with H5D__btree_cmp3 and H5D__btree_found), asking for
the element-size coordinate as 0. collect_chunk_info_checked now parses
the tree with its keys and returns each stored chunk only when that
lookup, replayed over the scaled keys, finds it.

A key with a non-zero element-size coordinate is therefore found in a
1-D dataset (cmp3 compares only the first coordinate there, and found
compares with <=) and missed in a dataset of rank 2 or more, which reads
fill values. The previous commit refused every such key, which refused
1-D files libhdf5 reads correctly; before that, the rank-2 case read the
chunk's data where h5py reads fill values (cve-2025-44905
/Shuffle_float_data_le, now identical to h5py, so it leaves the
conformance report's list of libhdf5 bugs).

Test: chunk_keys_with_an_element_offset_read_as_libhdf5_reads_them
compares 1-D and 2-D files against h5py's values. It fails on the
previous commit (the 1-D file is refused) and with the refusal removed
(the 2-D file reads 0..23 where h5py reads fill values).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 11:35:13 -05:00
co-authored by Claude Opus 5.5
parent 378afa1584
commit 742ed4dfb8
4 changed files with 322 additions and 125 deletions
+9 -2
View File
@@ -40,8 +40,15 @@ ZFP filters; the 2 mismatches are the known h5py big-endian VL bug.
- shuffle uses its own parameter as the element size, as libhdf5 does
(`cve-2025-44905`);
- an unfiltered chunk the index records at other than the chunk's size is
refused (it read with zeros for the missing bytes; `cve-2025-44904`),
as is a chunk B-tree key with a non-zero element offset.
refused (it read with zeros for the missing bytes; `cve-2025-44904`);
- a v1 B-tree chunk index is read as libhdf5 reads it: each chunk is
looked up the way `H5B_find` / `H5D__btree_cmp3` / `H5D__btree_found`
look it up, and a chunk that lookup does not find reads as fill values.
A key with a non-zero element-size coordinate is found in a 1-D dataset
and not in one of rank 2 or more (`cve-2025-44905`
`/Shuffle_float_data_le`, which read the chunk's data where h5py reads
fill values); an interim fix refused every such key, including 1-D
files libhdf5 reads correctly.
- **Refused as libhdf5 refuses them:** a v1 group with an empty link name
fails its listing (`FormatError::InvalidLinkName`; lookups still work,
`cve-2021-46244`); dataspaces with more than 32 dimensions, a rank on a