feat(format): read chunked datasets indexed by a version-2 B-tree
With libver='latest', a chunked dataset with two or more unlimited dimensions indexes its chunks with a v2 B-tree (layout v4, index type 5). Reading one failed with "unsupported chunked layout version=4, index_type=Some(5)". read_btree_v2_chunks decodes record types 10 (address + scaled offsets) and 11 (address, stored size, filter mask, scaled offsets). The width of the stored-size field is taken from the record size the tree header declares rather than re-deriving the library's formula. Scaled offsets are multiplied back by the chunk dimensions with overflow checks. The chunk-index dispatch existed four times (uncached, cached, sweep and indexed readers). The three copies outside list_chunks now call it, so every read path — and fill-value handling and partial reads — supports every index type from one place. h5py interop test: plain, gzip+shuffle, a 2500-chunk tree with internal nodes, a sparse dataset with a fill value, and a strided hyperslab. Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
c6a7bbfc67
commit
d668e45ab5
@@ -11,6 +11,10 @@
|
||||
2.7 ms, one column 5.2 ms. Results are identical to the full-read path
|
||||
(equivalence-tested over random hyperslabs and point lists, ranks 1-3,
|
||||
contiguous / chunked / deflate). New `read_harness` bench binary.
|
||||
- **Datasets indexed by a version-2 B-tree now read** (layout v4, chunk index
|
||||
type 5 — what `libver='latest'` uses for two or more unlimited dimensions;
|
||||
previously "unsupported chunked layout"). The four copies of the chunk-index
|
||||
dispatch are now one shared function, so every read path gets it.
|
||||
- **Out-of-range selections are errors.** They used to return data: a hyperslab
|
||||
past an edge came back padded with zeros, and a point whose column was out of
|
||||
range wrapped into the next row and returned that element. Now
|
||||
|
||||
Reference in New Issue
Block a user