bench: concurrent-read harness against h5py threads and processes
concurrent_read reads one shared File from 1-16 threads: every dataset in full (distinct datasets per thread) and random hyperslabs of one dataset, over a deflate and a contiguous file it generates (or reuses while manifest.json matches). It reports decoded MB/s and scaling efficiency, warm or --cold (posix_fadvise) page cache, sizes the decode pool with --decode-threads, and writes JSON. scripts/concurrent_read_h5py.py runs the same workload on the same files with h5py threads or spawned processes (same splitmix64 data and slab stream, checked at spot elements), and compare_concurrent_read.py prints one table and refuses runs with different workloads. A smoke test runs all three end to end on tiny files (h5py half honours CLAWHDF5_PYTHON / CLAWHDF5_REQUIRE_INTEROP). BENCHMARKS.md gets a "Concurrent reads" section with the commands, marked not yet measured. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -195,6 +195,13 @@
|
||||
README claimed but nothing measured.
|
||||
- `footprint_bench` reports whether it built `float16` or `f32` stores and
|
||||
takes `--f32`; it had kept printing "f32" after the default changed.
|
||||
- New `concurrent_read` harness, with an h5py counterpart
|
||||
(`crates/clawhdf5-bench/scripts/concurrent_read_h5py.py`, threads or
|
||||
processes) and `compare_concurrent_read.py`: decoded read throughput and
|
||||
scaling efficiency at 1-16 threads on one open file, full reads of distinct
|
||||
datasets and random hyperslabs of one dataset, deflate and contiguous, warm
|
||||
or `--cold` page cache, JSON output. Not yet measured — `BENCHMARKS.md`
|
||||
("Concurrent reads") has the commands and no numbers.
|
||||
|
||||
### Interop
|
||||
- **Conformance sweep in the repo** (`conformance/`, report in
|
||||
|
||||
Reference in New Issue
Block a user