clawhdf5-netcdf4: unlimited dimensions report their length #24

2026-09-29T02:38:31Z

osobh commented

2026-09-29 02:38:31 +00:00

Merge order: 2 of 4. This PR is stacked on #23.

clawhdf5-netcdf4 now reports an unlimited dimension's size the way netCDF-C does (nc4_find_dim_len):

the largest current extent of the variables that use the dimension, in any group (found through the scale's REFERENCE_LIST);
a coordinate variable counts with its own extent;
0 when nothing has been written.

Before, it returned the dimension scale's own extent, which netCDF-C leaves at 0, so a dimension holding records reported size 0. Every release (v2.1.0–v2.7.0) is affected.

Test: interop_tests::unlimited_dimension_lengths_match_netcdf4_python compares against netCDF4-python. It covers variables of different lengths (one in a subgroup), a dimension nothing has written to, a coordinate variable shorter than another variable on its dimension, and a subgroup's own dimension. It fails without the fix.

New open known issue, found while fixing this: Variable::dimensions() matches dimensions by size instead of reading DIMENSION_LIST, and variables() also lists dimension scales that have no data of their own. Not fixed here.

🤖 Generated with Claude Code

**Merge order: 2 of 4.** This PR is stacked on #23. `clawhdf5-netcdf4` now reports an unlimited dimension's size the way netCDF-C does (`nc4_find_dim_len`): - the largest current extent of the variables that use the dimension, in any group (found through the scale's `REFERENCE_LIST`); - a coordinate variable counts with its own extent; - 0 when nothing has been written. Before, it returned the dimension scale's own extent, which netCDF-C leaves at 0, so a dimension holding records reported size 0. Every release (v2.1.0–v2.7.0) is affected. **Test:** `interop_tests::unlimited_dimension_lengths_match_netcdf4_python` compares against netCDF4-python. It covers variables of different lengths (one in a subgroup), a dimension nothing has written to, a coordinate variable shorter than another variable on its dimension, and a subgroup's own dimension. It fails without the fix. **New open known issue, found while fixing this:** `Variable::dimensions()` matches dimensions by size instead of reading `DIMENSION_LIST`, and `variables()` also lists dimension scales that have no data of their own. Not fixed here. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

osobh added 2 commits 2026-09-29 02:38:31 +00:00

clawhdf5: a dropped FileEditor releases its lock at once

CI / test-arm64 (pull_request) Successful in 1m38s

Details

CI / test (pull_request) Successful in 28m11s

Details

3eca5d8334

FileEditor's flock belongs to the open file description. When another
thread forks to spawn a process, the child shares the locked descriptor
until it execs, so a reopen right after the drop could be refused with
Error::Locked (a one-off failure of edit_interop::editor_locks_the_file in
a parallel test run). Drop now unlocks before closing, which releases the
lock for every descriptor sharing it.

Reproducer edit_tests::drop_releases_the_lock_while_other_threads_spawn_processes
(4 threads running `true`, 2000 open/drop rounds): 1483 of 2000 reopens
refused before, 0 in 30 runs after (tank). An OFD lock would not help: it
is inherited across fork the same way and does not conflict with
libhdf5's flock. The agent store's lock file unlocks on drop too.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

clawhdf5-netcdf4: an unlimited dimension reports its current length

CI / test-arm64 (pull_request) Successful in 1m34s

Details

CI / test (pull_request) Successful in 30m38s

Details

31efac1ae1

Dimension::size of an unlimited dimension was its dimension scale's
extent, which netCDF-C leaves at 0, so it read 0 for a dimension holding
records. It is now what netCDF-C reports (nc4_find_dim_len): the largest
current extent of the variables using it in any group, found through the
scale's REFERENCE_LIST, a coordinate variable's own extent included; 0
when nothing has been written.

interop_tests::unlimited_dimension_lengths_match_netcdf4_python compares
with netCDF4-python (variables of different lengths, one in a subgroup,
an unwritten dimension, a coordinate variable shorter than another
variable on its dimension, a subgroup's own dimension); before the fix it
got time 0/6, rec 3/5, srec 0/1.

The known-issues entry moves to Fixed; the crate README's warning goes.
A new open entry records a related bug found meanwhile: variables'
dimensions are matched by size, not DIMENSION_LIST.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

osobh referenced this pull request

2026-09-29 02:38:32 +00:00

HDF5 2.x small floats (FP4/FP6/FP8/bfloat16) checked against libhdf5 2.2.0 #25

osobh referenced this pull request

2026-09-29 02:38:33 +00:00

Write complex numbers, incl. HDF5 2.0 native complex (class 11) #26

osobh referenced this issue from a commit

2026-09-29 02:38:39 +00:00

docs: known-issues — cite PRs #23 and #24

osobh merged commit b6bbe6b604 into main

2026-09-29 03:10:57 +00:00

osobh deleted branch fix/netcdf-unlimited-dim-size