feat(format): nested groups, soft/hard/external links and creation order in the writer

FileWriter wrote the root group plus one level of groups, and refused
path-like names. The writer now flattens its builders into a group tree
(writer_tree.rs) before layout:

- A name may be a path ("a/b/x", "/a/b/x" at the root); missing
  intermediate groups are created as h5py does, and GroupBuilder gains
  create_group/add_group so builders nest to any depth. A group added at a
  path that already holds a group is merged into it (require_group);
  any other repeated name, an empty or "." component, or an absolute path
  below the root is an error.
- add_soft_link, add_hard_link and add_external_link on FileWriter,
  FileBuilder and GroupBuilder. Hard-link targets are resolved to objects
  at finish (through other hard links; a missing target, a soft link on the
  way or a cycle of paths is an error). Objects with several hard links get
  an Object Reference Count message so libhdf5 can delete one link without
  freeing the object.
- track_order(true) per group, or as the file default, tracks and indexes
  link creation order: Link Info flags and max order, the order in each
  Link message, and a type-6 creation-order B-tree for dense groups.
- A group's link index is one B-tree leaf; more than 65535 links is an
  error.

Groups are laid out depth-first from the root, datasets group by group,
and untracked groups keep writing datasets, then groups, then other links:
files with one level of groups are byte-identical to before.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:33:46 -05:00
co-authored by Claude Opus 5.5
parent 8cbbef3fae
commit d102c06306
11 changed files with 1694 additions and 433 deletions
+38
View File
@@ -3,6 +3,44 @@
## Unreleased
### Writer: groups and links (2026-09-26)
- **Nested groups, to any depth.** `FileWriter`/`FileBuilder` wrote the root
group plus one level, and refused path-like names. Now a name may be a path
(`create_dataset("a/b/x")`, `create_group("a/b")`, a leading `/` at the
root) and missing intermediate groups are created, as h5py does; groups
also nest through the new `GroupBuilder::create_group`/`add_group`. A group
added at a path that already holds a group is merged into it (h5py's
`require_group`); a name used twice otherwise, an empty or `"."`
component (`"a//b"`, `"a/"`) or an absolute path below the root is an
error. Datasets, attributes, dense attribute storage and dense link
storage work at every level.
- **Soft, hard and external links at any depth:** `add_soft_link(name,
target)` (h5py's `SoftLink`; the target may dangle),
`add_hard_link(name, target)` (h5py's `f[name] = f[target]`; the target
path is resolved when the file is written, may go through other hard
links, and a missing target, a soft link on the way or a cycle of
hard-link paths is an error) and `add_external_link`, on `FileWriter`,
`FileBuilder` and `GroupBuilder`. An object with several hard links gets
an Object Reference Count message, so libhdf5 can delete one of the links
without freeing the object.
- **Link creation order:** `track_order(true)` on a `GroupBuilder`, or on
`FileWriter`/`FileBuilder` for every group that does not set its own,
tracks and indexes link creation order (h5py's `track_order=True`): the
Link Info message carries the flags, each link its order, and a dense
group a creation-order B-tree (type 6). h5py then lists members in
insertion order. Attribute creation order is not tracked.
- A group holds at most 65 535 links (its link index is one B-tree leaf);
more is an error. `GroupBuilder`'s fields changed (they were
crate-private); `FinishedGroup` is unchanged for callers.
- Files that use one level of groups and no new link kinds are laid out as
before: byte-identical to the writer with the Group Info fix below
(compared on simple, mixed dense/chunked/compact/external-link and paged
files). Tests: h5py and clawhdf5 read the same
tree (every path, attribute and value) from a 5-level file; soft, hard,
external and cyclic hard links; 10 000 links in one group, with and
without creation order; libhdf5 adding and deleting links in our groups;
`h5rs check` passes and `h5rs dump` equals h5dump
(`crates/clawhdf5/tests/writer_groups_interop.rs`,
`crates/clawhdf5-tools/tests/h5rs_interop.rs`).
- **libhdf5 could not add links to groups we wrote.** h5py in `"r+"` mode
failed with "Unable to create link (message type not found)" on every
group `FileWriter` wrote: libhdf5 reads a group's Group Info message before