feat(facade): read VL strings and VL sequences through File
VL-string datasets (h5py's default str dtype) failed read_string with "type mismatch: expected String, got VariableLength". read_string now reads fixed- and variable-length strings, with h5py's values (a string ends at a NUL, a null element is ""). New: - Dataset::read_string_bytes: each VL string's exact bytes; - Dataset::read_string_selection: hyperslabs/points of either kind; - Dataset::read_vlen::<T>() and read_vlen_selection::<T>(): VL sequences of numbers as Vec<Vec<T>>, T in f64/f32/i64/i32/u64, converted like the other typed readers; - File::decode_strings / decode_string_bytes / decode_vlen: VL values in compound fields and AttrValue::Raw attributes; - MmapDataset and LazyDataset: read_string for VL strings, read_string_bytes and read_vlen. tests/vl_data_interop.rs checks every path against h5py with 8- and 4-byte offsets: scalar, 1-D and 2-D, ASCII and UTF-8, empty strings, contiguous, compact, chunked with gzip and shuffle, unwritten and partly written chunks, hyperslabs, compound members, attributes, a big-endian base type, and a patched file with an embedded NUL and mis-sized heap objects. NetCDF-4 string variables read too (netCDF4-python test). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -30,6 +30,24 @@
|
||||
a VL datatype whose stored size is not 4 + offset size + 4 (libhdf5
|
||||
ignores the stored size). The conformance probe resolves VL elements
|
||||
with `VlResolver` too; conformance unchanged at 575 of 697.
|
||||
- **VL data through the facade.** VL-string datasets (h5py's default `str`
|
||||
dtype) failed `read_string` with "type mismatch: expected String, got
|
||||
VariableLength". `Dataset::read_string` now reads fixed- and
|
||||
variable-length strings; new `read_string_bytes` (a VL string's exact
|
||||
bytes, as h5py's `Dataset[()]` returns them), `read_string_selection`,
|
||||
`read_vlen::<T>()` / `read_vlen_selection::<T>()` for VL sequences of
|
||||
numbers (`T` = `f64`, `f32`, `i64`, `i32`, `u64`; converted like the
|
||||
other typed readers), and `File::decode_strings` / `decode_string_bytes`
|
||||
/ `decode_vlen` for VL values in compound fields and `AttrValue::Raw`
|
||||
attributes. `MmapDataset` and `LazyDataset` gain `read_string` for VL
|
||||
strings, `read_string_bytes` and `read_vlen`. Checked against h5py with
|
||||
8- and 4-byte offsets: scalar and 1-/2-D, ASCII and UTF-8, empty strings,
|
||||
contiguous, compact, chunked with gzip/shuffle, never-written and
|
||||
partly written chunks, hyperslab selections, VL members of compound
|
||||
datasets and attributes (`crates/clawhdf5/tests/vl_data_interop.rs`).
|
||||
NetCDF-4 `string` variables now read through
|
||||
`clawhdf5_netcdf4::Variable::read_string` (checked against netCDF4-python
|
||||
in `crates/clawhdf5-netcdf4/tests/interop_tests.rs`).
|
||||
|
||||
### Plugin filters (2026-09-26)
|
||||
- **LZF, bitshuffle, bzip2 and Blosc read and write, in pure Rust.** Files
|
||||
|
||||
Reference in New Issue
Block a user