format: read object header chunks from a queue, one buffer at a time

The version-1 chunk walk nested continuation chunks depth-first and kept
every enclosing chunk's buffer alive, up to 65 536 chunks. With storage
that hands out owned buffers (CountingStorage, the Storage trait, remote
storage) a crafted chain of chunks nested in each other read and held the
square of the file's size (a 192 KB file read 768 MB).

Chunks are now read from a FIFO queue of (address, length) pairs in the
order their continuation messages are found, as H5O_protect does and as
the editor's header walker already did, each buffer released before the
next read. In both header versions a chunk starting at an address seen
before (cycle) is refused, and so are chunks adding up to more than the
file, which bounds a header's reads by the file's size. Overlap itself is
allowed: libhdf5 reads cve-2025-7067.h5, whose continuation chunk overlaps
chunk 0 (refusing overlap cost that conformance file).

Tests: the nested chain is refused having read at most the file (it read
n^2 bytes before); a 3000-chunk chain reads each chunk once; a chunk's
messages follow the whole previous chunk (they were inserted at the
continuation message); an overlapping continuation chunk is read.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 18:50:23 -05:00
co-authored by Claude Opus 5.5
parent 930921e8cb
commit a69c5be8b2
2 changed files with 218 additions and 81 deletions
+9 -3
View File
@@ -87,9 +87,15 @@
header whose continuation chunks chain more than 32 deep (a header that
gains a chunk per attribute added when full, as libhdf5 and the editor
grow it) was refused with `NestingDepthExceeded`; version-2 headers
stopped at 256 chunks. Chunks are now followed without recursion, in the
same order; a chunk address seen twice (a cycle) or more than 65 536
chunks are refused.
stopped at 256 chunks. Chunks are now read one at a time from a queue,
in the order their continuation messages are found (libhdf5's
`H5O_protect` order, which the editor already used; a version-1
chunk's messages used to be inserted at its continuation message), each
buffer released before the next is read; a chunk address seen twice (a
cycle), chunks adding up to more than the file (a crafted chain of
chunks nested in each other made storage with owned buffers read and
hold the square of the file's size), or more than 65 536 chunks are
refused, so a header's chunks read at most the file's size.
- Tests: `crates/clawhdf5-tools/tests/edit_coverage_interop.rs` (h5py
`earliest`/`v110`/`latest` and clawhdf5-written files; structure
comparisons with libhdf5 for version-2 B-trees, shrink on every index,