feat(format): nested groups, soft/hard/external links and creation order in the writer
FileWriter wrote the root group plus one level of groups, and refused
path-like names. The writer now flattens its builders into a group tree
(writer_tree.rs) before layout:
- A name may be a path ("a/b/x", "/a/b/x" at the root); missing
intermediate groups are created as h5py does, and GroupBuilder gains
create_group/add_group so builders nest to any depth. A group added at a
path that already holds a group is merged into it (require_group);
any other repeated name, an empty or "." component, or an absolute path
below the root is an error.
- add_soft_link, add_hard_link and add_external_link on FileWriter,
FileBuilder and GroupBuilder. Hard-link targets are resolved to objects
at finish (through other hard links; a missing target, a soft link on the
way or a cycle of paths is an error). Objects with several hard links get
an Object Reference Count message so libhdf5 can delete one link without
freeing the object.
- track_order(true) per group, or as the file default, tracks and indexes
link creation order: Link Info flags and max order, the order in each
Link message, and a type-6 creation-order B-tree for dense groups.
- A group's link index is one B-tree leaf; more than 65535 links is an
error.
Groups are laid out depth-first from the root, datasets group by group,
and untracked groups keep writing datasets, then groups, then other links:
files with one level of groups are byte-identical to before.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -3,6 +3,44 @@
|
||||
## Unreleased
|
||||
|
||||
### Writer: groups and links (2026-09-26)
|
||||
- **Nested groups, to any depth.** `FileWriter`/`FileBuilder` wrote the root
|
||||
group plus one level, and refused path-like names. Now a name may be a path
|
||||
(`create_dataset("a/b/x")`, `create_group("a/b")`, a leading `/` at the
|
||||
root) and missing intermediate groups are created, as h5py does; groups
|
||||
also nest through the new `GroupBuilder::create_group`/`add_group`. A group
|
||||
added at a path that already holds a group is merged into it (h5py's
|
||||
`require_group`); a name used twice otherwise, an empty or `"."`
|
||||
component (`"a//b"`, `"a/"`) or an absolute path below the root is an
|
||||
error. Datasets, attributes, dense attribute storage and dense link
|
||||
storage work at every level.
|
||||
- **Soft, hard and external links at any depth:** `add_soft_link(name,
|
||||
target)` (h5py's `SoftLink`; the target may dangle),
|
||||
`add_hard_link(name, target)` (h5py's `f[name] = f[target]`; the target
|
||||
path is resolved when the file is written, may go through other hard
|
||||
links, and a missing target, a soft link on the way or a cycle of
|
||||
hard-link paths is an error) and `add_external_link`, on `FileWriter`,
|
||||
`FileBuilder` and `GroupBuilder`. An object with several hard links gets
|
||||
an Object Reference Count message, so libhdf5 can delete one of the links
|
||||
without freeing the object.
|
||||
- **Link creation order:** `track_order(true)` on a `GroupBuilder`, or on
|
||||
`FileWriter`/`FileBuilder` for every group that does not set its own,
|
||||
tracks and indexes link creation order (h5py's `track_order=True`): the
|
||||
Link Info message carries the flags, each link its order, and a dense
|
||||
group a creation-order B-tree (type 6). h5py then lists members in
|
||||
insertion order. Attribute creation order is not tracked.
|
||||
- A group holds at most 65 535 links (its link index is one B-tree leaf);
|
||||
more is an error. `GroupBuilder`'s fields changed (they were
|
||||
crate-private); `FinishedGroup` is unchanged for callers.
|
||||
- Files that use one level of groups and no new link kinds are laid out as
|
||||
before: byte-identical to the writer with the Group Info fix below
|
||||
(compared on simple, mixed dense/chunked/compact/external-link and paged
|
||||
files). Tests: h5py and clawhdf5 read the same
|
||||
tree (every path, attribute and value) from a 5-level file; soft, hard,
|
||||
external and cyclic hard links; 10 000 links in one group, with and
|
||||
without creation order; libhdf5 adding and deleting links in our groups;
|
||||
`h5rs check` passes and `h5rs dump` equals h5dump
|
||||
(`crates/clawhdf5/tests/writer_groups_interop.rs`,
|
||||
`crates/clawhdf5-tools/tests/h5rs_interop.rs`).
|
||||
- **libhdf5 could not add links to groups we wrote.** h5py in `"r+"` mode
|
||||
failed with "Unable to create link (message type not found)" on every
|
||||
group `FileWriter` wrote: libhdf5 reads a group's Group Info message before
|
||||
|
||||
Reference in New Issue
Block a user