feat(format): parse H5T_STD_REF references and decode object references
HDF5 1.12 revised the reference datatype (class 7) in datatype message version 4: reference types 2-4 are the new H5T_STD_REF object / dataset-region / attribute references. Datatype::parse rejected them with InvalidReferenceType, so any dataset of that type was unreadable. h5py cannot write this type, which is why it had never been tested. A real file was produced by calling the libhdf5 bundled in the h5py wheel through ctypes (H5T_STD_REF_g, H5Rcreate_object, H5Dwrite); the 2 KB result is committed as tests/fixtures/std_ref_hdf5_2_0.h5 with its generator, gen_std_ref.py. - ReferenceType gains Object2, DatasetRegion2 and Attribute, accepted only from datatype version 4. - read_object_references decodes Object2 elements: type(1) flags(1) token_size(1) token, zero-padded to the element size; the token is the target's object header address. A null reference decodes to the undefined address; an external reference, a wrong type byte or a token that doesn't fit is an error. The fixture test follows both references and checks they resolve to the objects they were created from. Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
05c665a898
commit
52cfcf20b2
+14
-6
@@ -62,13 +62,21 @@ only files using the native type through the C API / h5py low-level API hit this
|
||||
|
||||
## Revised reference datatype (class 7, version 4) is not parsed
|
||||
|
||||
**Status:** open, unconfirmed against a real file.
|
||||
**Status:** fixed 2026-09-19 for object references; region and attribute
|
||||
references are recognised but not decoded.
|
||||
|
||||
**Summary:** HDF5 1.12+ `H5T_STD_REF` references use datatype version 4 with
|
||||
reference types 2–4 (object2 / region2 / attribute), which `Datatype::parse`
|
||||
rejects with `InvalidReferenceType`. h5py still writes the legacy v1
|
||||
object/region references, which read correctly, so no reproducing file has been
|
||||
generated yet; one written with the C API (`H5T_STD_REF`) is needed.
|
||||
**Summary:** HDF5 1.12+ `H5T_STD_REF` references use datatype message version 4
|
||||
with reference types 2-4 (object / region / attribute), which `Datatype::parse`
|
||||
rejected with `InvalidReferenceType`. h5py still writes the legacy references,
|
||||
so no file had been available to test against.
|
||||
|
||||
**Fix:** a real file was produced by driving the libhdf5 bundled in the h5py
|
||||
wheel through ctypes (`tests/fixtures/gen_std_ref.py` ->
|
||||
`std_ref_hdf5_2_0.h5`). The three new types parse as
|
||||
`ReferenceType::{Object2, DatasetRegion2, Attribute}`, and
|
||||
`read_object_references` decodes `Object2` elements (type, flags, token size,
|
||||
token = target object header address). External references (flag bit 0) and
|
||||
the region/attribute payloads are errors rather than misreads.
|
||||
|
||||
## `clawhdf5-gpu` `gpu_tests` can hang under the default parallel test runner
|
||||
|
||||
|
||||
Reference in New Issue
Block a user