feat(format): read chunked datasets indexed by a version-2 B-tree

With libver='latest', a chunked dataset with two or more unlimited dimensions
indexes its chunks with a v2 B-tree (layout v4, index type 5). Reading one
failed with "unsupported chunked layout version=4, index_type=Some(5)".

read_btree_v2_chunks decodes record types 10 (address + scaled offsets) and 11
(address, stored size, filter mask, scaled offsets). The width of the
stored-size field is taken from the record size the tree header declares
rather than re-deriving the library's formula. Scaled offsets are multiplied
back by the chunk dimensions with overflow checks.

The chunk-index dispatch existed four times (uncached, cached, sweep and
indexed readers). The three copies outside list_chunks now call it, so every
read path — and fill-value handling and partial reads — supports every index
type from one place.

h5py interop test: plain, gzip+shuffle, a 2500-chunk tree with internal nodes,
a sparse dataset with a fill value, and a strided hyperslab.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
osobh
2026-09-19 13:59:18 -07:00
co-authored by Claude Fable 5.1
parent c6a7bbfc67
commit d668e45ab5
4 changed files with 229 additions and 256 deletions
+4
View File
@@ -11,6 +11,10 @@
2.7 ms, one column 5.2 ms. Results are identical to the full-read path
(equivalence-tested over random hyperslabs and point lists, ranks 1-3,
contiguous / chunked / deflate). New `read_harness` bench binary.
- **Datasets indexed by a version-2 B-tree now read** (layout v4, chunk index
type 5 — what `libver='latest'` uses for two or more unlimited dimensions;
previously "unsupported chunked layout"). The four copies of the chunk-index
dispatch are now one shared function, so every read path gets it.
- **Out-of-range selections are errors.** They used to return data: a hyperslab
past an edge came back padded with zeros, and a point whose column was out of
range wrapped into the next row and returned that element. Now