writer: v2 B-trees with internal nodes (no 65 535-record limit)

Dense link and attribute indexes and the chunk index for several
unlimited dimensions were single leaves, capping them at 65 535
records. btree_v2_write builds trees of any depth, with node capacities
and pointer widths from libhdf5's H5B2__hdr_init arithmetic (now shared
with the reader as btree_v2::node_info) and libhdf5's node sizes (512
dense, 2048 chunks). Indexes that fit the old one-leaf layout are
written byte for byte as before (compared for 10..65 535 links, attrs
and chunks, tracked and filtered).

Tests: 100 000 links (short names; long names with creation order),
70 000 attributes, 200 000 chunks (and 80 000 deflated), read by h5py,
h5dump and clawhdf5 and edited by h5py r+; h5rs check on the same
shapes, asserting depths 2-3.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

This commit is contained in:

osobh

2026-09-26 10:12:07 -05:00

co-authored by

Claude Opus 5.5

parent 7acfb79584

commit d63c76e7ab

11 changed files with 1005 additions and 185 deletions

									
										crates/clawhdf5-format/src/lib.rs
									
		+1
		
												View File
												
				@@ -61,6 +61,7 @@ pub mod attribute;

				pub mod attribute_info;

				pub mod btree_v1;

				pub mod btree_v2;

				mod btree_v2_write;

				mod bulk_alloc;

				pub mod checksum;

				pub mod chunk_cache;