fix(format): limit chunks to 4 GiB only under a v1 B-tree index

libhdf5 refuses a chunk of 4 GiB or more only when a version-1 B-tree indexes it (H5D__chunk_init: "chunk size must be < 4GB with v1 b-tree index"). HDF5 2.0 writes larger chunks with layout version 5, and h5py reads them; these were refused. chunk_geometry now takes the layout version and applies the limit to layout version 3 and earlier only. The interop test is ignored by default: h5py writes a 4 GiB chunk and both libraries hold it in memory. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-26 01:17:36 -05:00
parent 6a8ee3ec7f
commit a14ccc36bf
5 changed files with 104 additions and 42 deletions
@@ -212,7 +212,7 @@ pub enum FormatError {
    InvalidDatatype(String),
    /// A chunked layout whose chunk dimensions libhdf5 refuses: a zero
    /// dimension, a rank that does not match the dataspace, or a chunk of
-    /// 4 GiB or more.
+    /// 4 GiB or more indexed by a version-1 B-tree.
    InvalidChunkDimensions(String),
    /// The superblock's end-of-file address lies past the end of the file:
    /// the file was truncated (libhdf5 refuses to open it).