fix(format): limit chunks to 4 GiB only under a v1 B-tree index
libhdf5 refuses a chunk of 4 GiB or more only when a version-1 B-tree indexes it (H5D__chunk_init: "chunk size must be < 4GB with v1 b-tree index"). HDF5 2.0 writes larger chunks with layout version 5, and h5py reads them; these were refused. chunk_geometry now takes the layout version and applies the limit to layout version 3 and earlier only. The interop test is ignored by default: h5py writes a 4 GiB chunk and both libraries hold it in memory. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
+3
-1
@@ -323,7 +323,9 @@
|
||||
VAX byte order; libhdf5 ignores it before version 3, and so does this.
|
||||
- chunked layouts (`FormatError::InvalidChunkDimensions`): a zero chunk
|
||||
dimension, a chunk rank that does not match the dataspace, a chunk of
|
||||
4 GiB or more (0x80000000-sized chunks hung the reader), and v1 B-tree
|
||||
4 GiB or more indexed by a v1 B-tree (layout version 3 or earlier;
|
||||
0x80000000-sized chunks hung the reader — layout versions 4 and 5 allow
|
||||
larger chunks, and HDF5 2.0 writes them), and v1 B-tree
|
||||
chunk keys whose offsets are not multiples of the chunk dimensions,
|
||||
including the keys that only bound a node
|
||||
(`chunked_read::collect_chunk_info_checked`).
|
||||
|
||||
Reference in New Issue
Block a user