feat(facade): read VL strings and VL sequences through File

VL-string datasets (h5py's default str dtype) failed read_string with
"type mismatch: expected String, got VariableLength". read_string now
reads fixed- and variable-length strings, with h5py's values (a string
ends at a NUL, a null element is ""). New:
- Dataset::read_string_bytes: each VL string's exact bytes;
- Dataset::read_string_selection: hyperslabs/points of either kind;
- Dataset::read_vlen::<T>() and read_vlen_selection::<T>(): VL sequences
  of numbers as Vec<Vec<T>>, T in f64/f32/i64/i32/u64, converted like the
  other typed readers;
- File::decode_strings / decode_string_bytes / decode_vlen: VL values in
  compound fields and AttrValue::Raw attributes;
- MmapDataset and LazyDataset: read_string for VL strings,
  read_string_bytes and read_vlen.

tests/vl_data_interop.rs checks every path against h5py with 8- and
4-byte offsets: scalar, 1-D and 2-D, ASCII and UTF-8, empty strings,
contiguous, compact, chunked with gzip and shuffle, unwritten and partly
written chunks, hyperslabs, compound members, attributes, a big-endian
base type, and a patched file with an embedded NUL and mis-sized heap
objects. NetCDF-4 string variables read too (netCDF4-python test).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:25:05 -05:00
co-authored by Claude Opus 5.5
parent f99587c27d
commit 8ce6eca34d
9 changed files with 818 additions and 8 deletions
+18
View File
@@ -30,6 +30,24 @@
a VL datatype whose stored size is not 4 + offset size + 4 (libhdf5
ignores the stored size). The conformance probe resolves VL elements
with `VlResolver` too; conformance unchanged at 575 of 697.
- **VL data through the facade.** VL-string datasets (h5py's default `str`
dtype) failed `read_string` with "type mismatch: expected String, got
VariableLength". `Dataset::read_string` now reads fixed- and
variable-length strings; new `read_string_bytes` (a VL string's exact
bytes, as h5py's `Dataset[()]` returns them), `read_string_selection`,
`read_vlen::<T>()` / `read_vlen_selection::<T>()` for VL sequences of
numbers (`T` = `f64`, `f32`, `i64`, `i32`, `u64`; converted like the
other typed readers), and `File::decode_strings` / `decode_string_bytes`
/ `decode_vlen` for VL values in compound fields and `AttrValue::Raw`
attributes. `MmapDataset` and `LazyDataset` gain `read_string` for VL
strings, `read_string_bytes` and `read_vlen`. Checked against h5py with
8- and 4-byte offsets: scalar and 1-/2-D, ASCII and UTF-8, empty strings,
contiguous, compact, chunked with gzip/shuffle, never-written and
partly written chunks, hyperslab selections, VL members of compound
datasets and attributes (`crates/clawhdf5/tests/vl_data_interop.rs`).
NetCDF-4 `string` variables now read through
`clawhdf5_netcdf4::Variable::read_string` (checked against netCDF4-python
in `crates/clawhdf5-netcdf4/tests/interop_tests.rs`).
### Plugin filters (2026-09-26)
- **LZF, bitshuffle, bzip2 and Blosc read and write, in pure Rust.** Files