clawhdf5-netcdf4: phony dimensions, skipped types and order as netCDF-C
CI / test-arm64 (pull_request) Successful in 1m43s
CI / test (pull_request) Successful in 20m28s

Read a file's metadata the way netCDF-C 4.9.3 does (libhdf5/hdf5open.c),
for the whole file on first use (src/model.rs, replacing src/scope.rs):

- links in creation order when the group tracks it, else name order;
  a group's datasets before its subgroups; dimension ids file-wide;
- variables' dimensions from _Netcdf4Coordinates (file-wide ids), else
  the scales DIMENSION_LIST attaches when the first axis has one, else
  netCDF-C's phony dimensions phony_dim_<id> (create_phony_dims: shared
  by length and unlimitedness within a group, not between two axes of
  one variable, numbered subgroups first, a zero length unlimited);
- datasets of types netCDF-C cannot represent are not variables
  (references, bit fields, time, arrays, compounds/enums/VLENs over
  them), replaying netCDF-C's file-wide type list, failed types
  included;
- unlimited lengths as nc4_find_dim_len (its group and below).

NcType gains Enum, Compound, VLen, Opaque and is #[non_exhaustive];
Variable::nc_type is netCDF-C's type (1-byte strings NC_CHAR). New
clawhdf5_format::group_v2::links_in_creation_order_in.

Tests compare with netCDF-C itself (tests/netcdf_c_view.py calls the
libnetcdf netCDF4-python bundles through ctypes): new interop cases for
h5py files without dimension scales, every type class, link order; and
the gated corpus_vs_netcdf_c (CLAWHDF5_NETCDF_CORPUS): 420 of the 429
conformance-corpus files netCDF-C opens match (main: 68); the other 9
are explained in tests/corpus_known_differences.txt and known-issues.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-29 20:38:51 -05:00
co-authored by Claude Opus 5.5
parent 4260af4f70
commit e5d6f59e12
18 changed files with 2263 additions and 524 deletions
+43
View File
@@ -731,6 +731,49 @@ fn group_children<S: Storage + ?Sized>(
Ok(entries)
}
/// The names of the links of the group at `group_address` in creation
/// order — the order libhdf5 iterates them in with `H5_INDEX_CRT_ORDER`
/// (`H5Literate`) — when the group tracks the creation order of its links;
/// `None` when it does not (a version-1 group never does), in which case
/// libhdf5 can only iterate by name (`H5_INDEX_NAME`: byte order of the
/// names). Hard, soft and external links are listed (user-defined ones,
/// which cannot be followed, are not); links without a creation order
/// value, which a tracking group should not have, come last in the order
/// they are stored.
pub fn links_in_creation_order_in<S: Storage + ?Sized>(
file_data: &S,
superblock: &Superblock,
group_address: u64,
) -> Result<Option<Vec<String>>, FormatError> {
let os = superblock.offset_size;
let ls = superblock.length_size;
let header = ObjectHeader::parse_in(file_data, checked_addr(group_address)?, os, ls)?;
if !is_v2_group(&header) {
return Ok(None);
}
let link_info = find_link_info(&header, os)?;
if link_info.max_creation_order.is_none() {
return Ok(None);
}
let mut links: Vec<(u64, String)> = Vec::new();
let mut visit = |link: LinkMessage| {
links.push((link.creation_order.unwrap_or(u64::MAX), link.name));
};
if let Some(fh_addr) = link_info.fractal_heap_address {
for_each_dense_link(file_data, &link_info, fh_addr, os, ls, false, visit)?;
} else {
for msg in &header.messages {
if msg.msg_type == MessageType::Link
&& let Some(link) = parse_link(&msg.data, os)?
{
visit(link);
}
}
}
links.sort_by_key(|&(order, _)| order);
Ok(Some(links.into_iter().map(|(_, name)| name).collect()))
}
/// Soft links followed while resolving one path. Guards against link cycles
/// (`a -> b -> a`), which are legal to create.
const MAX_SOFT_LINK_DEPTH: u8 = 16;