fix(format): limit chunks to 4 GiB only under a v1 B-tree index

libhdf5 refuses a chunk of 4 GiB or more only when a version-1 B-tree indexes it (H5D__chunk_init: "chunk size must be < 4GB with v1 b-tree index"). HDF5 2.0 writes larger chunks with layout version 5, and h5py reads them; these were refused. chunk_geometry now takes the layout version and applies the limit to layout version 3 and earlier only. The interop test is ignored by default: h5py writes a 4 GiB chunk and both libraries hold it in memory. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-26 01:17:36 -05:00
parent 6a8ee3ec7f
commit a14ccc36bf
5 changed files with 104 additions and 42 deletions
@@ -360,7 +360,7 @@ pub fn read_raw_data_selection(
            chunk_index_type,
            ..
        } => {
-            crate::chunked_read::chunk_geometry(chunk_dimensions, dataspace, elem_size)?;
+            crate::chunked_read::chunk_geometry(chunk_dimensions, *version, dataspace, elem_size)?;
            // For chunked data, only read chunks that intersect the selection
            let chunk_dims: Vec<u64> = chunk_dimensions.iter().map(|&d| d as u64).collect();
            let rank = dims.len();