perf(agent): open a store without copying the whole file
`read_from_disk` memory-mapped the file and then copied the entire mapping into a `Vec` to hand to `File::from_bytes` — but `File::open` memory-maps it itself whenever the facade's `mmap` feature is on, which it is by default. So every open mapped the file, memcpy'd all of it, and parsed the copy. Store open at 100k x 384: 455 ms -> 327 ms, about 28% faster (two runs after the change, 326.8 and 328.1 ms). Peak memory is unchanged, which is worth saying because the opposite is the natural assumption. The footprint harness now tracks a high-water mark next to the retained figure, and it shows the peak falling after the parse, during the index build — so a buffer allocated and freed inside the parse never reaches it. Confirmed rather than assumed: holding a deliberate extra copy of the whole file across the parse leaves the peak exactly where it was, which is also what proved the instrument was working before trusting its answer. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
@@ -2,6 +2,16 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Performance
|
||||
- `clawhdf5-agent`: **opening a store is ~28% faster** (455 ms -> 327 ms at
|
||||
100k x 384). `read_from_disk` memory-mapped the file and then copied the
|
||||
entire mapping into a `Vec` for `File::from_bytes`, when `File::open`
|
||||
memory-maps it directly — so every open paid a full-file memcpy for nothing.
|
||||
Process peak memory is unchanged: the peak falls after the parse, during the
|
||||
index build, so the transient never reached the high-water mark. The
|
||||
footprint harness now reports that peak next to the retained figure, which
|
||||
is how this was checked rather than assumed.
|
||||
|
||||
### Integrity
|
||||
- `clawhdf5-format`: **Fixed and Extensible Array chunk indexes now verify
|
||||
their checksums** (the `checksum` feature, on by default). Every structure
|
||||
|
||||
Reference in New Issue
Block a user