osobhandClaude Opus 5.5
3c89a31df0
New clawhdf5::FileEditor opens an HDF5 file (h5py-written at any libver, HDF5 2.0 format included, or clawhdf5-written) under an exclusive flock and changes only what an edit touches: - write_selection/write_all/write_values: compact, contiguous (also late-allocated) and chunked datasets, any selection. Chunks are decoded, updated and re-encoded; a filtered chunk that no longer fits moves to the end of the file unless it is the file's last structure, which grows in place. New chunks go into v1 B-tree, Extensible Array (paged data blocks included), Fixed Array and single-chunk indexes, created on first use. - resize: grow chunked datasets up to maxshape. - set_attr: add/replace compact attributes, in a NIL slot or a new continuation chunk. Each edit is planned in an in-memory image and refused whole (Error::Unsupported) when any part is unsupported (v2 B-tree / implicit new chunks, shrinking, vlen/reference data, dense or order-tracked attributes, cache images, paged/persistent free space). Commit writes and syncs new space before patching existing bytes. Layout v5 (HDF5 2.0) array indexes use 8-byte filtered chunk sizes, as libhdf5 does. Error gains Unsupported/InvalidArgument/Locked and is #[non_exhaustive]; the Python bindings map them. build_attr_message is public. Tests (h5py, h5dump, h5rs check --data after every round; h5py r+ afterwards): appends crossing EA super/data blocks and B-tree splits, the same B-tree node counts and EA statistics as libhdf5 for the same writes (in order, reversed and shuffled; paged blocks), every layout and chunk index overwritten under random selections, attributes to continuation chunks, random operations against a model, refused edits leave the file byte-identical, locking. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT