quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

writer: track attribute creation order with track_order

Browse Source 
h5py's track_order=True orders attributes as well as links; the writer
tracked links only. A tracking object's header now sets the attribute
creation order tracked/indexed flags and carries per-message creation
orders, an Attribute Info message holds the next order (inline too),
and dense storage gets a type-9 creation-order index. The file default
applies to datasets, with DatasetBuilder::track_order per dataset; more
than 65 535 attributes on a tracking object is an error (libhdf5's
counter is 2 bytes). The reader lists such attributes in creation
order.

h5py lists them in order (inline, dense, 20 000 on one dataset) and
keeps numbering in r+ mode, including its inline-to-dense move.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 10:17:37 -05:00
co-authored by Claude Opus 5.5
parent d63c76e7ab
commit 193a5f8a82
9 changed files with 515 additions and 101 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+16
View File
@@ -3,6 +3,22 @@
## Unreleased
### Writer: large dense indexes (2026-09-26)
- **`track_order` orders attributes too, as h5py's `track_order=True`
does.** It tracked link creation order only, so h5py listed a tracked
object's attributes by name. A tracking object's header now has the
attribute creation order tracked and indexed flags, an Attribute Info
message with the next creation order (also for inline attributes), and a
creation order on each inline attribute message; dense attribute storage
gets a creation-order index (B-tree type 9). `FileWriter::track_order` /
`FileBuilder::track_order` now apply to datasets' attributes as well, and
`DatasetBuilder::track_order` sets it per dataset. Groups and datasets
that track order are written differently from before; others are
unchanged. More than 65 535 attributes on a tracking object is an error
(libhdf5's creation order counter is 2 bytes). The reader
(`attribute::extract_attributes*`) lists a tracking object's attributes
in creation order. Test `track_order_lists_attributes_in_creation_order`
(h5py lists, reads and extends them in "r+" mode, including libhdf5's
move from inline to dense storage).
- **No more 65 535-record limit on the writer's v2 B-trees.** Dense link
storage (name index and creation-order index), dense attribute storage
and the chunk index of datasets with more than one unlimited dimension