clawhdf5-netcdf4: an unlimited dimension reports its current length
Dimension::size of an unlimited dimension was its dimension scale's extent, which netCDF-C leaves at 0, so it read 0 for a dimension holding records. It is now what netCDF-C reports (nc4_find_dim_len): the largest current extent of the variables using it in any group, found through the scale's REFERENCE_LIST, a coordinate variable's own extent included; 0 when nothing has been written. interop_tests::unlimited_dimension_lengths_match_netcdf4_python compares with netCDF4-python (variables of different lengths, one in a subgroup, an unwritten dimension, a coordinate variable shorter than another variable on its dimension, a subgroup's own dimension); before the fix it got time 0/6, rec 3/5, srec 0/1. The known-issues entry moves to Fixed; the crate README's warning goes. A new open entry records a related bug found meanwhile: variables' dimensions are matched by size, not DIMENSION_LIST. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
+45
-14
@@ -15,7 +15,7 @@ Checked against `main` at `9b5803f` on 2026-09-28.
|
||||
|
||||
| Issue | Kind | Since |
|
||||
|---|---|---|
|
||||
| [NetCDF-4: an unlimited dimension reports size 0](#netcdf-4-an-unlimited-dimension-reports-size-0) | **wrong metadata** (dimension size; variable shapes and values are right) | 2026-09-28 |
|
||||
| [NetCDF-4: variables' dimensions are guessed from sizes](#netcdf-4-variables-dimensions-are-guessed-from-sizes) | **wrong metadata** (`Variable::dimensions`, pure dimension scales listed as variables; values and dimension sizes are right) | 2026-09-28 |
|
||||
| [In-place modification (`FileEditor`) limits](#in-place-modification-fileeditor-limits) | refused edits (`Error::Unsupported`), space reuse per editor, no journal | 2026-09-26 |
|
||||
| [Python in-place editing limits](#python-in-place-editing-clawhdf5filepath-r-limits) | refused writes (`NotImplementedError`), deliberate conversion differences | 2026-09-27 |
|
||||
| [Selection reads that decode more than the selection](#selection-reads-that-decode-more-than-the-selection) | speed only | 2026-09-26 |
|
||||
@@ -401,20 +401,24 @@ always expose it. A fix belongs in `conformance/ref_bugs.py` (more or more
|
||||
varied reads for this object) or in documenting the file as a known
|
||||
refusal; neither is done.
|
||||
|
||||
## NetCDF-4: an unlimited dimension reports size 0
|
||||
## NetCDF-4: variables' dimensions are guessed from sizes
|
||||
|
||||
**Status:** open (found 2026-09-28 while verifying the README refresh).
|
||||
`clawhdf5-netcdf4`'s `NetCDF4File::dimensions()` reports an unlimited
|
||||
dimension's `size` as 0 when variables along it hold records. Reproducer: with
|
||||
netCDF4-python, create dimension `time` (unlimited) and `x` (3), a variable
|
||||
`t(time, x)`, and write 2 records; netCDF4 reports `time` = 2 and `t` shape
|
||||
(2, 3). clawhdf5-netcdf4 reports `dim time size 0 unlimited true`, while
|
||||
`variable("t").shape()` correctly gives `[2, 3]`. In NetCDF-4 an unlimited
|
||||
dimension's length is the largest extent of the variables that use it (its
|
||||
dimension-scale dataset is not extended by netCDF-C), and the size is read
|
||||
from the dimension scale instead. Variable shapes and values are correct;
|
||||
only `Dimension::size` of unlimited dimensions is wrong. Workaround: use the
|
||||
variables' shapes.
|
||||
**Status:** open (found 2026-09-28 while fixing unlimited dimension sizes).
|
||||
`clawhdf5-netcdf4` gives each variable the dimensions it finds by size
|
||||
(`match_dimensions_to_variable`: the first unused dimension of equal size,
|
||||
else an anonymous `dim_<n>`), not the ones its `DIMENSION_LIST` names, and
|
||||
`variables()` also lists the dimension scales that are only dimensions
|
||||
(netCDF-C hides them). With netCDF4-python: unlimited `time` and `empty`,
|
||||
`x` (3), `a(time)` with 2 records, `b(time, x)` with 5, `e(empty)` — netCDF4
|
||||
reports `time` = 5, `a` on `time`, and variables `a`, `b`, `e` and a `c(x)`;
|
||||
clawhdf5-netcdf4 reports `time` = 5 (right) but `a` on `dim_2`, and also
|
||||
variables `time` and `empty` (the scales, put on `empty`). Two dimensions of
|
||||
one size can be swapped the same way. Related: netCDF4 gives `a` the shape
|
||||
(5,) (a variable along an unlimited dimension has the dimension's length,
|
||||
unwritten records read as fill); `Variable::shape` is the HDF5 extent,
|
||||
`[2]`, and reads return those 2 values. `dimensions()` is right.
|
||||
Workaround: read the variable's `_Netcdf4Coordinates` attribute (dimension
|
||||
ids, matching each scale's `_Netcdf4Dimid`).
|
||||
|
||||
## The Node.js package (`packages/clawhdf5-node`) does not work
|
||||
|
||||
@@ -477,6 +481,33 @@ helped: it is inherited across `fork` the same way, and on Linux it does
|
||||
not conflict with `flock`, so libhdf5 (h5py), which locks with `flock`,
|
||||
would no longer be refused while an editor holds the file.
|
||||
|
||||
## NetCDF-4: an unlimited dimension reports size 0
|
||||
|
||||
**Status:** fixed 2026-09-28 (`fix/netcdf-unlimited-dim-size`). Affected
|
||||
every release (v2.1.0 to v2.7.0). Wrong metadata only: variable values and
|
||||
HDF5 extents were always right. Users who worked around it with the variables'
|
||||
shapes can use `Dimension::size` again.
|
||||
|
||||
Found 2026-09-28 while verifying the README refresh.
|
||||
`clawhdf5-netcdf4`'s `NetCDF4File::dimensions()` reported an unlimited
|
||||
dimension's `size` as 0 when variables along it hold records. Reproducer: with
|
||||
netCDF4-python, create dimension `time` (unlimited) and `x` (3), a variable
|
||||
`t(time, x)`, and write 2 records; netCDF4 reports `time` = 2 and `t` shape
|
||||
(2, 3). clawhdf5-netcdf4 reported `dim time size 0 unlimited true`, while
|
||||
`variable("t").shape()` correctly gave `[2, 3]`. The size was read from the
|
||||
dimension scale's extent, which netCDF-C does not extend (it stays 0; a
|
||||
coordinate variable is extended only by its own writes).
|
||||
|
||||
The size is now what netCDF-C reports (`NC4_inq_dim` → `nc4_find_dim_len`):
|
||||
the largest current extent, along the dimension, of the variables that use
|
||||
it in any group, found through the scale's `REFERENCE_LIST`, plus a
|
||||
coordinate variable's own extent; 0 when nothing has been written.
|
||||
`interop_tests::unlimited_dimension_lengths_match_netcdf4_python` compares
|
||||
with netCDF4-python for variables of different lengths (one in a subgroup),
|
||||
an unwritten dimension, a coordinate variable shorter than a variable on its
|
||||
dimension, and a subgroup's own unlimited dimension; before the fix it got
|
||||
`time` 0 (want 6), `rec` 3 (want 5), `srec` 0 (want 1).
|
||||
|
||||
## `ObjectHeader::parse` 4% slower after range-read M2/M3
|
||||
|
||||
**Status:** fixed 2026-09-27 (`96086ad`, PR #21), before any release.
|
||||
|
||||
Reference in New Issue
Block a user