Commit Graph
4 Commits
Author SHA1 Message Date
osobhandClaude Opus 5.5 00b6f76ee0 clawhdf5-netcdf4: variables' dimensions come from the file
CI / test-arm64 (pull_request) Successful in 1m33s
CI / test (pull_request) Successful in 18m24s
Variables got the first unused dimension of equal size, so a variable on
an unlimited dimension with fewer records got an anonymous dim_<n>, and
dimensions of one size could be swapped. Resolve them as netCDF-C does
(libhdf5/hdf5open.c): _Netcdf4Coordinates ids, else the scales
DIMENSION_LIST references (the last one attached to an axis), searched in
the variable's group and its parents; a coordinate variable is on its own
scale. Size matching remains only for axes the file names nothing for.

variables()/variable_names() leave out dimension scales that are only
dimensions, and _nc4_non_coord_<name> is the variable <name>.
Variable::shape is the netCDF shape (an unlimited dimension's length) and
the reads pad unwritten records with the fill value (_FillValue, else
NC_FILL_*; NaN from read_f64); Variable::stored_shape is the HDF5 extent.
New NetCDF4File::variable_names.

Tests compare with netCDF4-python variable by variable: the known-issues
reproducer, equal sizes, (p, p), scalars, inherited dimensions, unwritten
records, h5py dimension scales, h5netcdf and xarray files. CI installs
h5netcdf. known-issues entry moved to Fixed (history); stale open-table
row for the unlimited-size fix removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-28 22:52:04 -05:00
osobhandClaude Opus 5.5 31efac1ae1 clawhdf5-netcdf4: an unlimited dimension reports its current length
CI / test-arm64 (pull_request) Successful in 1m34s
CI / test (pull_request) Successful in 30m38s
Dimension::size of an unlimited dimension was its dimension scale's
extent, which netCDF-C leaves at 0, so it read 0 for a dimension holding
records. It is now what netCDF-C reports (nc4_find_dim_len): the largest
current extent of the variables using it in any group, found through the
scale's REFERENCE_LIST, a coordinate variable's own extent included; 0
when nothing has been written.

interop_tests::unlimited_dimension_lengths_match_netcdf4_python compares
with netCDF4-python (variables of different lengths, one in a subgroup,
an unwritten dimension, a coordinate variable shorter than another
variable on its dimension, a subgroup's own dimension); before the fix it
got time 0/6, rec 3/5, srec 0/1.

The known-issues entry moves to Fixed; the crate README's warning goes.
A new open entry records a related bug found meanwhile: variables'
dimensions are matched by size, not DIMENSION_LIST.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-28 21:24:59 -05:00
osobhandClaude Fable 5.1 97ab658c11 feat: attrs() reports every attribute; unsigned arrays stay unsigned
attrs() silently omitted any attribute whose datatype had no AttrValue variant
— including every Python bool, which h5py stores as an enum — plus complex,
compound and reference attributes, and cast unsigned 64-bit arrays to
I64Array so values above i64::MAX came back negative.

- numpy/h5py-style booleans (an enum of exactly FALSE=0 / TRUE=1 over an
  integer base) decode as I64 / I64Array of 0/1.
- AttrValue::U64Array keeps unsigned arrays unsigned. Behaviour change: an
  unsigned array attribute no longer arrives as I64Array; the netCDF-4 CF
  helpers (_FillValue, valid_range) and the Python bindings handle it.
- AttrValue::Raw { datatype, shape, data } carries any other attribute
  verbatim (also used when a value fails to decode as its declared type), so
  the attribute list is always complete. Decodable with data_read against the
  datatype; Python receives {"dtype", "shape", "data"}.
- Both new variants are writable, so attributes round-trip between files.
  h5py interop tests cover reading 13 attribute kinds and h5py reading back a
  compound and a u64 attribute written by clawhdf5.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-19 07:05:34 -07:00
redclawsystems 3f222f6956 Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17) from sdlc-docs/clawhdf5-types-20260514-165210 into main 2026-05-14 23:54:48 +00:00