fix(format): index every Extensible Array chunk, not just the first 244

The Extensible Array writer only filled the index block's 4 inline
elements and the 6 data blocks it addresses directly (240 elements); its
super block addresses were always undefined. Chunks from index 244 on were
written to the file but never indexed, so they read back as fill values in
our reader and in libhdf5, without an error.

The writer now lays out data blocks and super blocks for any element
count as H5EA__hdr_init sizes them, pages data blocks larger than 1024
elements (page-init bits in the owning super block), leaves blocks with no
defined element unallocated, and records real header statistics
(max_idx_set is one past the highest defined index).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-25 21:09:29 -05:00
co-authored by Claude Opus 5.5
parent 4a1876faf2
commit 44f5f8b5c5
3 changed files with 271 additions and 271 deletions
@@ -390,3 +390,24 @@ fn we_write_paged_fixed_array() {
cases.push(wcase("fa_2d_1100", &[110, 40], &[1, 4], None));
check_we_write(&cases);
}
/// An Extensible Array holds 4 elements in its index block and 240 in the
/// data blocks the index block addresses; everything after that lives under
/// super blocks, and from ~131K elements on in paged data blocks. Chunks past
/// index 243 used to be written but never indexed (read back as fill by us
/// and by libhdf5).
#[test]
fn we_write_extensible_array_past_index_block() {
let unl: &[u64] = &[u64::MAX];
let mut cases: Vec<WriteCase> = [1u64, 4, 5, 243, 244, 245, 300, 1000, 5000]
.iter()
.map(|&n| wcase(&format!("ea_{n}"), &[n * 4], &[4], Some(unl)))
.collect();
let mut filtered = wcase("ea_300_deflate", &[300 * 4], &[4], Some(unl));
filtered.deflate = true;
cases.push(filtered);
// Several super blocks and paged data blocks (level 13, the first with
// data blocks over 1024 elements, starts at element 4 + 131056).
cases.push(wcase("ea_140000", &[140_000], &[1], Some(unl)));
check_we_write(&cases);
}