osobhandClaude Opus 5.5
7d7a7e75d4
The superblock records where the file's data ends. libhdf5 refuses to
open a file shorter than that ("truncated file", H5F__super_read) and
fails any read past it ("addr overflow" / "address plus size exceeds
file eoa"). clawhdf5 read whatever was left of a truncated file, and read
bytes after the recorded end as if they belonged to the file.
New Superblock::data_end: FormatError::TruncatedFile for a file shorter
than its recorded end, otherwise where the HDF5 data ends. As in libhdf5,
the recorded end moves with the superblock when its recorded base address
is not where it is (a user block added afterwards; cve-2021-36977, which
h5py reads, depends on it), and the check is skipped for a v3 superblock
still being written in SWMR mode. File, LazyFile, MmapFile and the
conformance probe refuse a truncated file and parse only up to the end.
Files clawhdf5 writes record their true length, and the v2.5.0 agent-store
fixture and files written by v2.7.0 (plain, paged, user-block free) pass
the check.
Interop test: h5py writes a file; a copy missing its last 8 bytes must be
refused by both, a copy with bytes appended and one moved behind a new
512-byte user block must read in both.
Conformance (cached corpus, tank): 570 -> 571 ok
(h5clear_fsm_persist_less.h5, whose data past the recorded end was being
read); ten files h5py refuses as truncated (cve-2018-13874,
cve-2018-13876, the family/multi/subfiling members, h5clear_fsm_persist_
greater/user_greater) are now refused at open instead of read.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT