feat(format): parse H5T_STD_REF references and decode object references
HDF5 1.12 revised the reference datatype (class 7) in datatype message version 4: reference types 2-4 are the new H5T_STD_REF object / dataset-region / attribute references. Datatype::parse rejected them with InvalidReferenceType, so any dataset of that type was unreadable. h5py cannot write this type, which is why it had never been tested. A real file was produced by calling the libhdf5 bundled in the h5py wheel through ctypes (H5T_STD_REF_g, H5Rcreate_object, H5Dwrite); the 2 KB result is committed as tests/fixtures/std_ref_hdf5_2_0.h5 with its generator, gen_std_ref.py. - ReferenceType gains Object2, DatasetRegion2 and Attribute, accepted only from datatype version 4. - read_object_references decodes Object2 elements: type(1) flags(1) token_size(1) token, zero-padded to the element size; the token is the target's object header address. A null reference decodes to the undefined address; an external reference, a wrong type byte or a token that doesn't fit is an error. The fixture test follows both references and checks they resolve to the objects they were created from. Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
05c665a898
commit
52cfcf20b2
@@ -22,6 +22,11 @@
|
||||
type 5 — what `libver='latest'` uses for two or more unlimited dimensions;
|
||||
previously "unsupported chunked layout"). The four copies of the chunk-index
|
||||
dispatch are now one shared function, so every read path gets it.
|
||||
- **`H5T_STD_REF` references** (HDF5 1.12+, datatype message version 4) parse:
|
||||
`ReferenceType` gains `Object2`, `DatasetRegion2` and `Attribute`, and
|
||||
`read_object_references` decodes the new object references. Previously any
|
||||
dataset of this type failed with `InvalidReferenceType(2)`. Tested against a
|
||||
file written by HDF5 2.0 itself (fixture + generator script committed).
|
||||
- **Automatic chunk sizes.** Asking for compression (or any filter) without
|
||||
`with_chunks` used to store the whole dataset as one chunk, so any read had
|
||||
to decompress everything and nothing could be decoded in parallel. Datasets up
|
||||
|
||||
Reference in New Issue
Block a user