fix(format): bound B-tree v2 traversal against crafted files

Traversal recursed one frame per level with the depth taken from the
file (a u16), and followed child addresses without asking whether they
were shared. Two crafted inputs, both reproduced before fixing:

- A node listing itself as its own child, under a header claiming 65 535
  levels, overflowed the stack and aborted the process — SIGABRT, not an
  error a caller can handle — from under 100 bytes.
- Levels whose children all point at one shared node below reached it
  fan-out^depth times: 29.5 million records in 8 s from ~5 KB, and one
  more level would exhaust memory.

Depth is now capped at 64, as the fractal heap already was; no real tree
approaches it, since even at the minimum fan-out of two that is over
2^64 records. And traversal stops once it has produced more records
than the file has bytes to hold them — a valid tree stores each record
once in its own bytes, so this bounds shared subtrees without trusting
the header's own `total_records`. Both inputs now fail in under a
millisecond.

Every B-tree v2 user goes through this collector: dense attributes, v2
groups, shared messages and chunk indexes. To show the budget never
refuses a real file, a new interop test has HDF5 2.0 write a depth-2
chunk index with 40 000 records and reads back all 160 000 values; it
fails when the budget is deliberately made too tight.

Also corrects `BM25Index::search`, which claimed to use Block-Max WAND.
It scores exhaustively, and pruning would not help the store:
`hybrid_search` needs every score because fusion normalises over them.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-20 16:56:05 -07:00
co-authored by Claude Opus 5
parent 5889b378e9
commit e9aeb110b7
5 changed files with 251 additions and 4 deletions
+24
View File
@@ -1,5 +1,29 @@
# Changelog
## Unreleased
### Security
- `clawhdf5-format`: **a crafted file could crash any reader through B-tree v2
traversal.** Recursion was bounded only by the depth the file claimed (a
`u16`), and child addresses were never checked for sharing. A node listing
itself as its own child under a header claiming 65 535 levels — under 100
bytes — overflowed the stack and **aborted the process** (SIGABRT, not a
catchable error). Levels whose children all point at one shared node below
reached it fan-out^depth times: 29.5 million records from ~5 KB, and one
more level would exhaust memory. Both are now errors, returned in under a
millisecond: depth is capped at 64 (as the fractal heap already was), and
traversal stops once it has produced more records than the file has bytes
to hold. Every B-tree v2 user goes through this path — dense attributes,
v2 groups, shared messages and chunk indexes. Valid files are unaffected,
including a depth-2 HDF5 2.0 chunk index with 40 000 records, now covered by
an interop test.
### Documentation
- `clawhdf5-agent`: `BM25Index::search` claimed to use Block-Max WAND for early
termination. It never did; it scores every match exhaustively. It now says
so, and why no pruning would help the store: `hybrid_search` uses `scores()`,
since fusion normalises over every match.
## v2.6.0 (2026-09-20)
### Upgrade Notes