docs: dense storage fixes and the real limits of big groups

The changelog, known issues and README said a group holds up to 65 535
links while a group of about 17 000 was already unreadable. Record the
fixes (child indirect blocks, the next-block offset, the index leaf cap,
refusing oversized dense messages, hard-link memoisation, dataset
attribute overwrite) and the limits that remain true: 65 535 links or
dense attributes per object, and 65 515 bytes per dense message.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 09:02:47 -05:00
co-authored by Claude Opus 5.5
parent 8bcae3c78e
commit 546fdb84fa
3 changed files with 54 additions and 8 deletions
+12 -3
View File
@@ -209,11 +209,20 @@ fill-value item that did is fixed).
h5py, h5dump and `h5rs check --data` read them
(`crates/clawhdf5/tests/writer_groups_interop.rs`,
`crates/clawhdf5-tools/tests/h5rs_interop.rs`). Still missing: a group
with more than 65 535 links (its link index is one B-tree leaf) is an
error, and attribute creation order is not tracked.
with more than 65 535 links, or an object with more than 65 535 dense
attributes, is an error (the index is one B-tree leaf), and attribute
creation order is not tracked.
- ~~Dense link or attribute storage past 512 KiB of messages was written
unreadable (child indirect blocks of the fractal heap written as direct
blocks).~~ **Fixed 2026-09-26** (it affected 2.7.0 too): tested with
20 000 and 65 535 links and with 8 MB of dense attributes, read by
h5py, h5dump, `h5rs check` and clawhdf5, and h5py can add links to
such groups.
- ~~libhdf5 could not add a link to a group we wrote (no Group Info
message).~~ **Fixed 2026-09-26.**
- Dense attribute storage for attributes over 64 KiB.
- Huge fractal heap objects: in dense storage (more than 8 attributes on
an object, or more than 8 links in a group) one attribute or link
message over 65 515 bytes is an error.
- Output that HDF5 1.8 can read.
- A B-tree v2 chunk index larger than one leaf, so datasets with several
unlimited dimensions are limited to 65 535 chunks.