writer: v2 B-trees with internal nodes (no 65 535-record limit)
Dense link and attribute indexes and the chunk index for several unlimited dimensions were single leaves, capping them at 65 535 records. btree_v2_write builds trees of any depth, with node capacities and pointer widths from libhdf5's H5B2__hdr_init arithmetic (now shared with the reader as btree_v2::node_info) and libhdf5's node sizes (512 dense, 2048 chunks). Indexes that fit the old one-leaf layout are written byte for byte as before (compared for 10..65 535 links, attrs and chunks, tracked and filtered). Tests: 100 000 links (short names; long names with creation order), 70 000 attributes, 200 000 chunks (and 80 000 deflated), read by h5py, h5dump and clawhdf5 and edited by h5py r+; h5rs check on the same shapes, asserting depths 2-3. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -559,20 +559,6 @@ fn we_write_btree_v2_for_several_unlimited_dims() {
|
||||
check_we_write(&cases);
|
||||
}
|
||||
|
||||
/// A single-leaf B-tree has a 16-bit record count; beyond it the writer
|
||||
/// refuses rather than writing a tree libhdf5 would misread.
|
||||
#[test]
|
||||
fn btree_v2_index_past_one_leaf_is_refused() {
|
||||
let mut b = FileBuilder::new();
|
||||
b.create_dataset("d")
|
||||
.with_i32_data(&vec![0i32; 70_000])
|
||||
.with_shape(&[70_000, 1])
|
||||
.with_chunks(&[1, 1])
|
||||
.with_maxshape(&[u64::MAX, u64::MAX]);
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
assert!(b.write(dir.path().join("too_many.h5")).is_err());
|
||||
}
|
||||
|
||||
/// A maxshape equal to the shape cannot grow, so it needs no chunks: the
|
||||
/// dataset stays contiguous (as h5py makes it) unless chunks are requested.
|
||||
#[test]
|
||||
|
||||
Reference in New Issue
Block a user