feat(facade): read VL strings and VL sequences through File

VL-string datasets (h5py's default str dtype) failed read_string with
"type mismatch: expected String, got VariableLength". read_string now
reads fixed- and variable-length strings, with h5py's values (a string
ends at a NUL, a null element is ""). New:
- Dataset::read_string_bytes: each VL string's exact bytes;
- Dataset::read_string_selection: hyperslabs/points of either kind;
- Dataset::read_vlen::<T>() and read_vlen_selection::<T>(): VL sequences
  of numbers as Vec<Vec<T>>, T in f64/f32/i64/i32/u64, converted like the
  other typed readers;
- File::decode_strings / decode_string_bytes / decode_vlen: VL values in
  compound fields and AttrValue::Raw attributes;
- MmapDataset and LazyDataset: read_string for VL strings,
  read_string_bytes and read_vlen.

tests/vl_data_interop.rs checks every path against h5py with 8- and
4-byte offsets: scalar, 1-D and 2-D, ASCII and UTF-8, empty strings,
contiguous, compact, chunked with gzip and shuffle, unwritten and partly
written chunks, hyperslabs, compound members, attributes, a big-endian
base type, and a patched file with an embedded NUL and mis-sized heap
objects. NetCDF-4 string variables read too (netCDF4-python test).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:25:05 -05:00
co-authored by Claude Opus 5.5
parent f99587c27d
commit 8ce6eca34d
9 changed files with 818 additions and 8 deletions
+9 -2
View File
@@ -140,7 +140,13 @@ fill-value item that did is fixed).
is left out of `attrs()` (reported by `attrs_with_errors()`) instead of
failing the others.
- **Other readers:**
- VL-string datasets are not readable through `File`.
- VL-string datasets are not readable through `File`. **Fixed
2026-09-26:** `read_string` reads them (also `read_string_bytes`,
`read_string_selection`, and on `MmapFile`/`LazyFile`), with h5py's
values: strings end at a NUL, null elements are `""`; VL sequences of
numbers read with `read_vlen::<T>()`, and VL values inside compounds or
`AttrValue::Raw` attributes decode with `File::decode_strings` /
`File::decode_vlen` (`crates/clawhdf5/tests/vl_data_interop.rs`).
- Variable-length values inside a compound (and VL-string attributes) in
a file with 4-byte offsets (`sizeof_addr = 4`) fail with
`GlobalHeapObjectNotFound` or come back as `Raw`: these paths assume
@@ -505,7 +511,8 @@ which is what libhdf5 itself writes.
followed (no file system).
- Variable-length string datasets are read by decoding `read_selection`'s
bytes with `clawhdf5_format::vl_data` in the wasm crate; `File` itself still
cannot (see the audit gaps above).
cannot (see the audit gaps above). (`File` can since 2026-09-26; the wasm
crate still decodes them itself.)
## The Node.js package (`packages/clawhdf5-node`) does not work