format: object headers without per-header allocations for their chunk list
Reading continuation chunks from a queue (7e5e920,a69c5be) allocated a queue Vec and a BTreeSet of chunk starts for every header, and inlined the per-chunk message loop into the generic parser: ObjectHeader::parse over 401 version-1 headers went from 24.8 to 45.7 us. ChunkSpans now keeps the first 8 chunks in an inline array (cycle check by scan) and is also the read queue; only a header of more chunks allocates (a boxed spill list and start set). The message loop of one version-1 chunk is its own non-generic function. Same checks as before: any number of chunks up to 65,536, cycles refused, chunks bounded by the file size, one chunk buffer alive at a time, libhdf5 message order, overlap allowed. The cycle test now also covers spilled chunk lists. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -295,6 +295,14 @@
|
||||
chunks nested in each other made storage with owned buffers read and
|
||||
hold the square of the file's size), or more than 65 536 chunks are
|
||||
refused, so a header's chunks read at most the file's size.
|
||||
The first version allocated a queue and a set of chunk starts for every
|
||||
header and made `ObjectHeader::parse` of 401 small headers 1.8x slower
|
||||
(`local_metadata_bench`: 45.7 vs 24.8 µs on `main` `8f59b2e`). The first
|
||||
8 chunks now live in an inline array (cycle check by scan; only a longer
|
||||
header allocates), and the per-chunk message loop is its own function
|
||||
instead of being inlined into the generic parser. Provisional (tank load
|
||||
3–4; Criterion, separate binaries, 2 alternating rounds): 24.96–25.12 vs
|
||||
24.85–25.15 µs on `main`.
|
||||
- Tests: `crates/clawhdf5-tools/tests/edit_coverage_interop.rs` (h5py
|
||||
`earliest`/`v110`/`latest` and clawhdf5-written files; structure
|
||||
comparisons with libhdf5 for version-2 B-trees, shrink on every index,
|
||||
|
||||
Reference in New Issue
Block a user