Merge branch 'feat/p2b-writer-btree-internal-nodes' into feat/p2b-scale
# Conflicts: # CHANGELOG.md
This commit is contained in:
+30
-6
@@ -275,10 +275,29 @@ fill-value item that did is fixed).
|
||||
external links at any depth and optional creation-order tracking;
|
||||
h5py, h5dump and `h5rs check --data` read them
|
||||
(`crates/clawhdf5/tests/writer_groups_interop.rs`,
|
||||
`crates/clawhdf5-tools/tests/h5rs_interop.rs`). Still missing: a group
|
||||
with more than 65 535 links, or an object with more than 65 535 dense
|
||||
attributes, is an error (the index is one B-tree leaf), and attribute
|
||||
creation order is not tracked.
|
||||
`crates/clawhdf5-tools/tests/h5rs_interop.rs`). ~~Still missing:
|
||||
attribute creation order is not tracked.~~ **Fixed 2026-09-26:**
|
||||
`track_order` (file default, `GroupBuilder`, and the new
|
||||
`DatasetBuilder::track_order`) tracks and indexes attribute creation
|
||||
order as h5py's `track_order=True` does; h5py lists the attributes in
|
||||
the order they were set, inline and dense (20 000 on one dataset), and
|
||||
keeps numbering them in "r+" mode (tank,
|
||||
`cargo test -p clawhdf5 --test writer_groups_interop
|
||||
track_order_lists_attributes_in_creation_order`). libhdf5 numbers at
|
||||
most 65 535 attributes on such an object, so more is an error.
|
||||
- ~~A group with more than 65 535 links, or an object with more than
|
||||
65 535 dense attributes, is an error (the index is one B-tree leaf).~~
|
||||
**Fixed 2026-09-26:** the dense indexes are v2 B-trees of any depth
|
||||
(libhdf5's 512-byte nodes once the records outgrow the one-leaf layout,
|
||||
which smaller indexes keep byte for byte). Tested on tank with 100 000
|
||||
links in one group (short names, and 111-byte names with creation order
|
||||
tracked) and 70 000 attributes on one object: h5py lists them in order
|
||||
and reads the values, h5dump reads spot checks, `h5rs check` reads every
|
||||
record, and h5py in "r+" mode adds and deletes thousands of links and
|
||||
attributes in those trees (`cargo test -p clawhdf5 --test
|
||||
deep_btree_interop`; `cargo test -p clawhdf5-tools --test h5rs_interop
|
||||
check_files_with_deep_btrees`). The name indexes are ordered by hash and
|
||||
then name, as libhdf5 needs for names whose hashes collide.
|
||||
- ~~Dense link or attribute storage past 512 KiB of messages was written
|
||||
unreadable (child indirect blocks of the fractal heap written as direct
|
||||
blocks).~~ **Fixed 2026-09-26** (it affected 2.7.0 too): tested with
|
||||
@@ -291,8 +310,13 @@ fill-value item that did is fixed).
|
||||
an object, or more than 8 links in a group) one attribute or link
|
||||
message over 65 515 bytes is an error.
|
||||
- Output that HDF5 1.8 can read.
|
||||
- A B-tree v2 chunk index larger than one leaf, so datasets with several
|
||||
unlimited dimensions are limited to 65 535 chunks.
|
||||
- ~~A B-tree v2 chunk index larger than one leaf, so datasets with
|
||||
several unlimited dimensions are limited to 65 535 chunks.~~ **Fixed
|
||||
2026-09-26:** more chunks get libhdf5's 2048-byte nodes with internal
|
||||
nodes above the leaves. Tested on tank with 200 000 chunks (and 80 000
|
||||
deflated): h5py and clawhdf5 read every value, and h5py resizes the
|
||||
dataset and writes 4 510 new chunks into the tree (same commands as
|
||||
above).
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user