quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(format): read VL values in files with 4-byte offsets

Browse Source 
In a file with sizeof_addr = 4, a VL string attribute came back as
AttrValue::Raw, a compound's VL member failed with
GlobalHeapObjectNotFound and VL datasets failed with a size mismatch.

Two bugs: Datatype::type_size() said 16 for every VL type, while the
element is 4 + offset size + 4 bytes (12 here); and the global heap was
parsed without the padding libhdf5 puts after its collection and object
headers (both round up to 8), so with 4-byte lengths every object was
looked up 4 bytes early. Datatype::VariableLength now carries the size
its datatype message stores, and writes it back.

Checked against h5py in tests/vl_offset4_interop.rs (fails with either
fix reverted). Conformance unchanged at 575 of 697; in cve-2024-32608 a
VL attribute whose datatype claims 524304-byte elements is now an error
(h5py cannot iterate those attributes at all).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:19:13 -05:00
co-authored by Claude Opus 5.5
parent 63648c7000
commit 10da8f0d09
5 changed files with 184 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+15
View File
@@ -2,6 +2,21 @@
## Unreleased
### Variable-length data (2026-09-26)
- **VL values in files with 4-byte offsets** (`sizeof_addr = 4`). A VL
string attribute came back as `AttrValue::Raw`, a VL member of a compound
failed with `GlobalHeapObjectNotFound`, and VL datasets failed with a
size mismatch. Two causes: `Datatype::type_size()` reported 16 for every
VL type (the element is 4 + offset size + 4 bytes: 12 here), and the
global heap was parsed without the padding libhdf5 puts after its
collection and object headers (`H5HG_SIZEOF_HDR`/`H5HG_SIZEOF_OBJHDR`
round up to 8), so with 4-byte lengths every object was looked up 4
bytes early. `Datatype::VariableLength` now carries the element `size`
stored in the datatype message (**breaking** for code that builds or
exhaustively destructures that variant; patterns with `..` are
unaffected), and it is written back as stored. Tested against h5py
(`crates/clawhdf5/tests/vl_offset4_interop.rs`).
### Plugin filters (2026-09-26)
- **LZF, bitshuffle, bzip2 and Blosc read and write, in pure Rust.** Files
written by h5py with `compression="lzf"`, or with hdf5plugin's