feat(facade): read VL strings and VL sequences through File
VL-string datasets (h5py's default str dtype) failed read_string with "type mismatch: expected String, got VariableLength". read_string now reads fixed- and variable-length strings, with h5py's values (a string ends at a NUL, a null element is ""). New: - Dataset::read_string_bytes: each VL string's exact bytes; - Dataset::read_string_selection: hyperslabs/points of either kind; - Dataset::read_vlen::<T>() and read_vlen_selection::<T>(): VL sequences of numbers as Vec<Vec<T>>, T in f64/f32/i64/i32/u64, converted like the other typed readers; - File::decode_strings / decode_string_bytes / decode_vlen: VL values in compound fields and AttrValue::Raw attributes; - MmapDataset and LazyDataset: read_string for VL strings, read_string_bytes and read_vlen. tests/vl_data_interop.rs checks every path against h5py with 8- and 4-byte offsets: scalar, 1-D and 2-D, ASCII and UTF-8, empty strings, contiguous, compact, chunked with gzip and shuffle, unwritten and partly written chunks, hyperslabs, compound members, attributes, a big-endian base type, and a patched file with an embedded NUL and mis-sized heap objects. NetCDF-4 string variables read too (netCDF4-python test). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -140,7 +140,13 @@ fill-value item that did is fixed).
|
||||
is left out of `attrs()` (reported by `attrs_with_errors()`) instead of
|
||||
failing the others.
|
||||
- **Other readers:**
|
||||
- VL-string datasets are not readable through `File`.
|
||||
- VL-string datasets are not readable through `File`. **Fixed
|
||||
2026-09-26:** `read_string` reads them (also `read_string_bytes`,
|
||||
`read_string_selection`, and on `MmapFile`/`LazyFile`), with h5py's
|
||||
values: strings end at a NUL, null elements are `""`; VL sequences of
|
||||
numbers read with `read_vlen::<T>()`, and VL values inside compounds or
|
||||
`AttrValue::Raw` attributes decode with `File::decode_strings` /
|
||||
`File::decode_vlen` (`crates/clawhdf5/tests/vl_data_interop.rs`).
|
||||
- Variable-length values inside a compound (and VL-string attributes) in
|
||||
a file with 4-byte offsets (`sizeof_addr = 4`) fail with
|
||||
`GlobalHeapObjectNotFound` or come back as `Raw`: these paths assume
|
||||
@@ -505,7 +511,8 @@ which is what libhdf5 itself writes.
|
||||
followed (no file system).
|
||||
- Variable-length string datasets are read by decoding `read_selection`'s
|
||||
bytes with `clawhdf5_format::vl_data` in the wasm crate; `File` itself still
|
||||
cannot (see the audit gaps above).
|
||||
cannot (see the audit gaps above). (`File` can since 2026-09-26; the wasm
|
||||
crate still decodes them itself.)
|
||||
|
||||
## The Node.js package (`packages/clawhdf5-node`) does not work
|
||||
|
||||
|
||||
Reference in New Issue
Block a user