perf(agent): open a store without copying the whole file

`read_from_disk` memory-mapped the file and then copied the entire
mapping into a `Vec` to hand to `File::from_bytes` — but `File::open`
memory-maps it itself whenever the facade's `mmap` feature is on, which
it is by default. So every open mapped the file, memcpy'd all of it, and
parsed the copy.

Store open at 100k x 384: 455 ms -> 327 ms, about 28% faster (two runs
after the change, 326.8 and 328.1 ms).

Peak memory is unchanged, which is worth saying because the opposite is
the natural assumption. The footprint harness now tracks a high-water
mark next to the retained figure, and it shows the peak falling after
the parse, during the index build — so a buffer allocated and freed
inside the parse never reaches it. Confirmed rather than assumed:
holding a deliberate extra copy of the whole file across the parse
leaves the peak exactly where it was, which is also what proved the
instrument was working before trusting its answer.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-20 17:49:47 -07:00
co-authored by Claude Opus 5
parent a91df3f1c3
commit 1d767e3b93
4 changed files with 80 additions and 18 deletions
+10
View File
@@ -2,6 +2,16 @@
## Unreleased
### Performance
- `clawhdf5-agent`: **opening a store is ~28% faster** (455 ms -> 327 ms at
100k x 384). `read_from_disk` memory-mapped the file and then copied the
entire mapping into a `Vec` for `File::from_bytes`, when `File::open`
memory-maps it directly — so every open paid a full-file memcpy for nothing.
Process peak memory is unchanged: the peak falls after the parse, during the
index build, so the transient never reached the high-water mark. The
footprint harness now reports that peak next to the retained figure, which
is how this was checked rather than assumed.
### Integrity
- `clawhdf5-format`: **Fixed and Extensible Array chunk indexes now verify
their checksums** (the `checksum` feature, on by default). Every structure