osobhandClaude Opus 5.5
1c85986079
A heap ID's type is in bits 4-5 of its first byte (H5HF_ID_TYPE_MASK 0x30); bits 6-7 are the ID version. The reader took the type from bits 6-7, so every huge object ID (0x10) was decoded as a managed one and failed — and since dense attributes are read all at once, one attribute over the heap's 4 KiB managed limit made every attribute on its object unreadable (netcdf4-python's issue671.nc / issue672.nc). - Huge objects (type 1): located directly from the ID when address and length fit in it, otherwise through the huge-object v2 B-tree (record types 1 and 2); filtered huge objects are decoded with the heap's pipeline and their filter mask. - Tiny objects (type 2): read from the ID itself. - Filtered heaps: the header's pipeline is parsed (it was skipped short, so the header checksum was read from the wrong place), indirect-block entries for direct blocks carry their filtered size and mask, and direct blocks are decoded before objects are read from them. - An unknown ID version is an error. FractalHeapHeader gains huge_btree_address, filter_pipeline, root_direct_block_filtered_size, root_direct_block_filter_mask, offset_size and length_size; read_managed_object now accepts any ID type. Regression tests (h5py-written, compared with h5py): dense_attribute_stored_as_a_huge_heap_object, dense_group_with_a_huge_link, dense_group_with_a_filtered_link_heap; unit tests tiny_object_is_read_from_the_id, huge_object_with_a_direct_id, unknown_heap_id_version_is_refused. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
clawhdf5-format
Pure-Rust HDF5 binary format parsing and writing — no C dependencies.
Features
- Zero-copy superblock, object header, and B-tree parsing
- Chunked dataset read/write with filter pipelines
no_stdsupport (disablestdfeature)- Optional parallel reads via Rayon
- SHA-256 provenance tracking
Usage
use clawhdf5_format::Superblock;
let data = std::fs::read("data.h5").unwrap();
let sb = Superblock::from_bytes(&data).unwrap();
println!("HDF5 version {}.{}", sb.version_major(), sb.version_minor());
License
MIT