edit: version-2 B-tree chunk indexes, shrinking, early allocation

FileEditor can now:

- add, move and resize chunks of datasets with two or more unlimited
  dimensions (version-2 B-tree chunk index, record types 10/11). The new
  edit/btree2.rs follows libhdf5's H5B2 code: H5B2_update (modify, or
  insert into a leaf with room, or fall back to H5B2__insert), the
  preemptive split/redistribute loop with its two retries, split1,
  split_root (depth growth, node geometry per depth), redistribute2/3,
  cumulative record counts and the pointer widths H5B2__hdr_init derives,
  a checksum per node. Removal (H5B2_remove: merge2/3, redistribution,
  root collapse, the internal-record swap with a leaf) is there too. A
  dataset without an index yet gets one from the layout message's node
  size and split/merge percentages.
- shrink a chunked dataset along any dimension (resize to a smaller
  shape), as H5D__set_extent / H5D__chunk_prune_by_extent do: the same
  chunks visited in the same order; chunks wholly outside the new extent
  are removed from the index (version-1 B-tree: H5B_remove with its
  sibling key and link fix-ups and the empty-root case; version-2
  B-tree; Fixed/Extensible Array elements reset to the fill element;
  an implicit index keeps its chunks, as libhdf5 does) and their space
  noted as free; the part of each partial edge chunk outside the extent
  is overwritten with the fill value, so elements that come back after
  a later growth read as fill.
- under early allocation, allocate and fill the chunks a growth brings in
  (H5D__chunk_allocate), which an implicit index needs: libhdf5 refills
  them, and they may hold the data of chunks pruned earlier.

Tests (crates/clawhdf5-tools/tests/edit_coverage_interop.rs): growth in
both dimensions of v110/latest files, unfiltered and deflated, gives
node-for-node the version-2 B-tree libhdf5 builds (h5py with its chunk
cache off, so chunks enter the index in the editor's order), through a
depth increase; random chunk order; 60 random shrink/grow/write steps on
Extensible Array, version-2 B-tree, Fixed Array, 1-D and implicit
datasets (earliest/v110/latest, with and without gzip+shuffle) give the
values h5py gets doing the same and the same index shape (version-1 and
version-2 B-tree node shapes, Extensible Array statistics); h5py r+
continues on every result; h5dump and h5rs check accept them.
edit_interop's version-2 B-tree case now appends instead of expecting a
refusal; shrinking is no longer an error in edit_tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

crates/clawhdf5-tools/tests/edit_interop.rs

@@ -752,19 +752,20 @@ fn overwrite_every_layout() {
             }
             check_tools(&path, *dump);
         }
-        // A version-2 B-tree index can take new chunks only from libhdf5 for
-        // now: growing works, writing the new chunks is refused and changes
-        // nothing.
+        // A version-2 B-tree index takes new chunks too.
         if *lv != "'earliest'" {
             let mut ed = FileEditor::open(&path).unwrap();
-            ed.resize("bt2", &[8, 6]).unwrap();
-            let before = std::fs::read(&path).unwrap();
-            unsupported(ed.write_values("bt2", &block(&[6, 0], &[2, 6]), &[5; 12]));
-            assert!(
-                std::fs::read(&path).unwrap() == before,
-                "a refused edit changed the file"
-            );
-            models[12].resize(&[8, 6], 0);
+            ed.resize("bt2", &[8, 7]).unwrap();
+            models[12].resize(&[8, 7], 0);
+            let vals: Vec<i32> = (0..23).collect();
+            ed.write_values("bt2", &block(&[6, 0], &[2, 7]), &vals[..14])
+                .unwrap();
+            models[12].write_block(&[6, 0], &[2, 7], &vals[..14]);
+            ed.write_values("bt2", &block(&[0, 6], &[6, 1]), &vals[14..20])
+                .unwrap();
+            models[12].write_block(&[0, 6], &[6, 1], &vals[14..20]);
+            drop(ed);
+            verify(&path, "bt2", &models[12]);
         }
         // libhdf5 goes on modifying what we wrote.
         py(&format!(
@@ -1027,7 +1028,7 @@ fn refused_edits_change_nothing() {
     let before = std::fs::read(&path).unwrap();
     let mut ed = FileEditor::open(&path).unwrap();
     unsupported(ed.write_all("s", &[0u8; 32]));
-    unsupported(ed.resize("x", &[4]));
+    unsupported(ed.resize("c", &[4]));
     unsupported(
         ed.resize("c", &[6, 1])
             .map_err(|_| Error::Unsupported(String::new())),