A heap ID's type is in bits 4-5 of its first byte (H5HF_ID_TYPE_MASK
0x30); bits 6-7 are the ID version. The reader took the type from bits
6-7, so every huge object ID (0x10) was decoded as a managed one and
failed — and since dense attributes are read all at once, one attribute
over the heap's 4 KiB managed limit made every attribute on its object
unreadable (netcdf4-python's issue671.nc / issue672.nc).
- Huge objects (type 1): located directly from the ID when address and
length fit in it, otherwise through the huge-object v2 B-tree (record
types 1 and 2); filtered huge objects are decoded with the heap's
pipeline and their filter mask.
- Tiny objects (type 2): read from the ID itself.
- Filtered heaps: the header's pipeline is parsed (it was skipped short,
so the header checksum was read from the wrong place), indirect-block
entries for direct blocks carry their filtered size and mask, and
direct blocks are decoded before objects are read from them.
- An unknown ID version is an error.
FractalHeapHeader gains huge_btree_address, filter_pipeline,
root_direct_block_filtered_size, root_direct_block_filter_mask,
offset_size and length_size; read_managed_object now accepts any ID type.
Regression tests (h5py-written, compared with h5py):
dense_attribute_stored_as_a_huge_heap_object, dense_group_with_a_huge_link,
dense_group_with_a_filtered_link_heap; unit tests
tiny_object_is_read_from_the_id, huge_object_with_a_direct_id,
unknown_heap_id_version_is_refused.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A child indirect block in row r of a fractal heap's doubling table spans
that row's block size of heap space, so it has
log2(size) - log2(start_block_size * width) + 1 rows (libhdf5's
H5HF__dtable_size_to_rows). The reader used row - first_indirect_row + 1,
which undercounts, so every object stored past the root block's direct
rows (512 KiB with libhdf5's defaults) was unreachable: dense groups with
a few thousand long link names, or ~20 000 short ones, could not be listed.
Regression test: dense_group_whose_heap_outgrows_the_root_direct_rows
(h5py writes 2 500 links with 248-byte names; listing compared with h5py).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The fractal-heap reader split direct vs indirect block rows using the FRHP
"Starting # of Rows in Root Indirect Block" field (a constant, typically 1),
mislabeled as starting_row_of_indirect_blocks. For any heap whose data spans
more than one direct block — common in libhdf5 files with a large group or
many dense attributes — this treated direct blocks as indirect and walked into
garbage, failing with InvalidFractalHeapSignature.
Derive the split from the heap geometry instead: max_direct_rows =
log2(max_direct_block_size / starting_block_size) + 2. Rows below it hold
direct blocks; rows at/above hold child indirect blocks.
Validated against an h5py-written group with 400 dense attributes (root
indirect block, 4 rows, 13 direct blocks): all values now read correctly.
Regression fixture covers an 80-attribute multi-block heap.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
