quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases 1 Wiki Activity

clawhdf5-netcdf4: variables' dimensions come from the file
CI / test-arm64 (pull_request) Successful in 1m33s
Details
CI / test (pull_request) Successful in 18m24s
Details

Browse Source 
Variables got the first unused dimension of equal size, so a variable on
an unlimited dimension with fewer records got an anonymous dim_<n>, and
dimensions of one size could be swapped. Resolve them as netCDF-C does
(libhdf5/hdf5open.c): _Netcdf4Coordinates ids, else the scales
DIMENSION_LIST references (the last one attached to an axis), searched in
the variable's group and its parents; a coordinate variable is on its own
scale. Size matching remains only for axes the file names nothing for.

variables()/variable_names() leave out dimension scales that are only
dimensions, and _nc4_non_coord_<name> is the variable <name>.
Variable::shape is the netCDF shape (an unlimited dimension's length) and
the reads pad unwritten records with the fill value (_FillValue, else
NC_FILL_*; NaN from read_f64); Variable::stored_shape is the HDF5 extent.
New NetCDF4File::variable_names.

Tests compare with netCDF4-python variable by variable: the known-issues
reproducer, equal sizes, (p, p), scalars, inherited dimensions, unwritten
records, h5py dimension scales, h5netcdf and xarray files. CI installs
h5netcdf. known-issues entry moved to Fixed (history); stale open-table
row for the unlimited-size fix removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-28 22:52:04 -05:00
co-authored by Claude Opus 5.5
parent f9edf4d6ad
commit 00b6f76ee0
12 changed files with 1191 additions and 237 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/clawhdf5-netcdf4/README.md
+25 -7
View File
@@ -36,17 +36,35 @@ let values: Vec<f64> = temp.read_f64()?;
| Item | What |
|---|---|
| `NetCDF4File` | `open`, `from_bytes`, `dimensions`, `variables`, `variable`, `global_attrs`, `group`, `group_names`, `nc_properties`, and `hdf5_file` for the underlying `clawhdf5::File` |
| `NetCDF4Group` | the same for a sub-group (`dimensions`, `variables`, `attrs`, nested `group`) |
| `Variable` | `name`, `shape`, `dimensions`, `nc_type`, `is_coordinate`, `attrs`, `cf_attributes`; `read_f64` (CF scale/offset and fill applied), `read_raw_f32`/`_f64`/`_i32`/`_i64`/`_u64`, `read_string`, `read_raw` |
| `NetCDF4File` | `open`, `from_bytes`, `dimensions`, `variables`, `variable_names`, `variable`, `global_attrs`, `group`, `group_names`, `nc_properties`, and `hdf5_file` for the underlying `clawhdf5::File` |
| `NetCDF4Group` | the same for a sub-group (`dimensions`, `variables`, `variable_names`, `attrs`, nested `group`) |
| `Variable` | `name`, `shape`, `stored_shape`, `dimensions`, `nc_type`, `is_coordinate`, `attrs`, `cf_attributes`; `read_f64` (CF scale/offset and fill applied), `read_raw_f32`/`_f64`/`_i32`/`_i64`/`_u64`, `read_string`, `read_raw` |
| `Dimension` | `name`, `size`, `is_unlimited` (an unlimited dimension's `size` is its current length as netCDF-C reports it: the largest extent of the variables using it) |
| `CfAttributes` | CF convention attributes: `units`, `long_name`, `standard_name`, `fill_value` (`_FillValue`), `missing_value`, `scale_factor`, `add_offset`, `valid_range`, `calendar`, `axis` |
| `NcType` | the NetCDF type of a variable |
No cargo features. Tests compare against files written by netCDF4-python
(`tests/interop_tests.rs`; the CI job requires them with
`CLAWHDF5_REQUIRE_INTEROP=1`). What the HDF5 reader underneath cannot
read is listed in [`docs/known-issues.md`](../../docs/known-issues.md).
Variables and dimensions follow netCDF-C:
- A variable's dimensions are the ones the file names: the ids in its
`_Netcdf4Coordinates` attribute, else the dimension scales its
`DIMENSION_LIST` references, found in its group or a parent group. Only
an axis the file names no dimension for (an HDF5 file not written by a
netCDF library) gets the first dimension of the group of the same size,
else an anonymous `dim_<size>`.
- Dimension scales that are only dimensions are not variables; a dataset
`_nc4_non_coord_<name>` is the variable `<name>`.
- A variable along an unlimited dimension has the dimension's length:
`shape` is that length, and the reads return that many values, the
records the variable has not written as its fill value (`_FillValue`,
else netCDF's default for the type; NaN from `read_f64`).
`stored_shape` is the HDF5 dataset's extent.
No cargo features. Tests compare against files written by netCDF4-python,
h5py dimension scales, h5netcdf and xarray, variable by variable with what
netCDF4-python reads (`tests/interop_tests.rs`; the CI job requires them
with `CLAWHDF5_REQUIRE_INTEROP=1`; the h5netcdf cases skip when h5netcdf is
not installed). What the HDF5 reader underneath cannot read is listed in
[`docs/known-issues.md`](../../docs/known-issues.md).
## License