Chunk dimensions of 2^32 or more; copy-free unfiltered 4 GiB writes #32

Open
osobh wants to merge 7 commits from feat/huge-chunk-dims into main
Showing only changes of commit 4cc3111151 - Show all commits
+4 -7
View File
@@ -629,7 +629,7 @@ given.
## NetCDF-4: files without dimension scales, types and order differed from netCDF-C
**Status:** fixed 2026-09-29 (branch `fix/netcdf-phony-dims`). Affected
**Status:** fixed 2026-09-29 (#30). Affected
every release (v2.1.0 to v2.7.0) and `main` after #27. Wrong metadata only
(names, order and types of dimensions and variables); values were read
right. Users: the dimensions of a file without dimension scales are now
@@ -684,8 +684,7 @@ netCDF4-python-written files with netCDF-C itself, and
## HDF5 2.0 native complex numbers read as a `{r, i}` compound
**Status:** fixed 2026-09-29 (branch `feat/complex-first-class`; PR not
yet opened). Affected v2.2.0 to v2.7.0 (class 11 has parsed since v2.2.0)
**Status:** fixed 2026-09-29 (#31). Affected v2.2.0 to v2.7.0 (class 11 has parsed since v2.2.0)
and `main` until then. Listed until now under
[HDF5 features still unsupported](#hdf5-features-still-unsupported).
@@ -710,8 +709,7 @@ browser reads `[re, im]` pairs.
## Chunk dimensions of 2^32 or more were refused
**Status:** fixed 2026-09-29 (branch `feat/huge-chunk-dims`, PR not yet
opened); affected every release (v2.1.0 to v2.7.0). An error, never wrong
**Status:** fixed 2026-09-29 (#32); affected every release (v2.1.0 to v2.7.0). An error, never wrong
data. Nothing for users to do but upgrade; code that matches on
`DataLayout::Chunked::chunk_dimensions` gets `u64`s now.
@@ -735,8 +733,7 @@ test (the file lists; reading such a chunk on wasm32 is
## Writing a 4 GiB unfiltered chunk held several copies of it
**Status:** fixed 2026-09-29 (branch `feat/huge-chunk-dims`, PR not yet
opened); affected every release (v2.1.0 to v2.7.0). Memory only. To write
**Status:** fixed 2026-09-29 (#32); affected every release (v2.1.0 to v2.7.0). Memory only. To write
large data with the least memory, hand the builder the data
(`DatasetBuilder::with_u8_data_owned`) and write with
`FileBuilder::write` (or `FileWriter::finish_with`).