feat(facade): read VL strings and VL sequences through File

VL-string datasets (h5py's default str dtype) failed read_string with
"type mismatch: expected String, got VariableLength". read_string now
reads fixed- and variable-length strings, with h5py's values (a string
ends at a NUL, a null element is ""). New:
- Dataset::read_string_bytes: each VL string's exact bytes;
- Dataset::read_string_selection: hyperslabs/points of either kind;
- Dataset::read_vlen::<T>() and read_vlen_selection::<T>(): VL sequences
  of numbers as Vec<Vec<T>>, T in f64/f32/i64/i32/u64, converted like the
  other typed readers;
- File::decode_strings / decode_string_bytes / decode_vlen: VL values in
  compound fields and AttrValue::Raw attributes;
- MmapDataset and LazyDataset: read_string for VL strings,
  read_string_bytes and read_vlen.

tests/vl_data_interop.rs checks every path against h5py with 8- and
4-byte offsets: scalar, 1-D and 2-D, ASCII and UTF-8, empty strings,
contiguous, compact, chunked with gzip and shuffle, unwritten and partly
written chunks, hyperslabs, compound members, attributes, a big-endian
base type, and a patched file with an embedded NUL and mis-sized heap
objects. NetCDF-4 string variables read too (netCDF4-python test).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:25:05 -05:00
co-authored by Claude Opus 5.5
parent f99587c27d
commit 8ce6eca34d
9 changed files with 818 additions and 8 deletions
@@ -350,3 +350,30 @@ ds.close()
let press_vals = press_var.read_raw_f32().unwrap();
assert_eq!(press_vals, vec![1000.0f32, 850.0, 500.0, 200.0]);
}
#[test]
fn netcdf4_python_string_variable_clawhdf5_reads() {
// NC_STRING variables are HDF5 variable-length strings, which
// `read_string` refused ("expected String, got VariableLength") until
// 2026-09-26.
skip_if_no_netcdf4!();
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("strings.nc");
let path_str = path.display().to_string();
let script = format!(
r#"
import netCDF4 as nc
import numpy as np
ds = nc.Dataset("{path_str}", "w", format="NETCDF4")
ds.createDimension("station", 4)
v = ds.createVariable("name", str, ("station",))
v[:] = np.array(["Oslo", "", "São Paulo", "x"], dtype=object)
ds.close()
"#
);
run_python(&script);
let file = NetCDF4File::open(&path).unwrap();
let names = file.variable("name").unwrap().read_string().unwrap();
assert_eq!(names, vec!["Oslo", "", "São Paulo", "x"]);
}