From f2ff2c424fb8cee1c48b5c05f1a246bd3ca5eadd Mon Sep 17 00:00:00 2001 From: osobh Date: Sat, 26 Sep 2026 12:20:47 -0500 Subject: [PATCH] bench: chunked full reads now beat an h5py process pool Idle-start run on tank at c5334b1 (noisier than the last: compare ratios within the run). Full reads of deflate data at 16 threads: 4944 MB/s vs 3135 for 16 h5py processes (1.58x; 0.69x-0.76x before). One thread with the default pool: 6143 MB/s, 15x one h5py call. The concurrent-read known issue is closed. Co-Authored-By: Claude Opus 5.5 (1M context) --- BENCHMARKS.md | 39 +++++++++++++++++++++++++++++++++++++++ docs/known-issues.md | 8 +++++--- 2 files changed, 44 insertions(+), 3 deletions(-) diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 0bc5fe7..62beeb6 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -484,6 +484,45 @@ explain the slower windows. ## Concurrent reads +### Results after in-place chunk decoding (2026-09-26, tank, `c5334b1`) + +Same machine, files and commands, re-run after chunked reads started +decoding into reusable per-thread buffers straight into the (typed) output, +with the calling thread decoding alongside the pool. Load average 1.78 at +the start; it rose to 6-9 during the runs (the clawhdf5 runs' own threads, +and it stayed around 5-6 through the h5py runs, so something else was +active). **This run was noisier than the previous one: h5py's own contiguous +figures are about 40% lower than in the run below, and ours dropped +similarly, so compare ratios within a run rather than MB/s across runs.** +h5py was re-run in the same session. + +Each read decoding on its calling thread (`--decode-threads 1`, like h5py): + +| layout | mode | threads | clawhdf5 MB/s (eff) | h5py threads MB/s (eff) | h5py processes MB/s (eff) | vs h5py processes | +|---|---|---:|---:|---:|---:|---:| +| deflate | distinct | 1 | 670 (1.00) | 410 (1.00) | 397 (1.00) | 1.69x | +| deflate | distinct | 4 | 2434 (0.91) | 406 (0.25) | 1470 (0.93) | 1.66x | +| deflate | distinct | 8 | 3749 (0.70) | 406 (0.12) | 2398 (0.76) | 1.56x | +| deflate | distinct | 16 | 4944 (0.46) | 390 (0.06) | 3135 (0.49) | 1.58x | +| deflate | same | 1 | 211 (1.00) | 125 (1.00) | 124 (1.00) | 1.70x | +| deflate | same | 16 | 1835 (0.54) | 122 (0.06) | 961 (0.48) | 1.91x | +| contiguous | distinct | 1 | 6718 (1.00) | 5545 (1.00) | 5200 (1.00) | 1.29x | +| contiguous | distinct | 16 | 11035 (0.10) | 4950 (0.06) | 10558 (0.13) | 1.05x | +| contiguous | same | 1 | 14483 (1.00) | 2593 (1.00) | 2737 (1.00) | 5.29x | +| contiguous | same | 16 | 132175 (0.57) | 2224 (0.05) | 14809 (0.34) | 8.93x | + +With the default rayon pool, deflate `distinct` reads 6143 MB/s from a single +thread (15x h5py's 410 on one call) and 4556 MB/s at 16 threads (1.45x h5py +processes); the other rows are within the noise of the table above. + +What changed: full reads of chunked datasets were 0.69x-0.76x of h5py +processes at 16 threads in the run below, and are 1.58x here; with one +thread they were 1.44x and are 1.69x. Minor page faults for the 16-thread +run fell from about 4.6M to 0.2M (`/usr/bin/time -v`, provisional, loaded +machine). clawhdf5 now reads faster than 16 h5py processes in every row of +this benchmark except contiguous full reads at 16 threads, where both +saturate memory bandwidth (1.05x). + ### Results after the read fixes (2026-09-26, tank, `408f69e`) Same machine, files and commands as the first run below, re-run on an idle diff --git a/docs/known-issues.md b/docs/known-issues.md index a58f42e..287e501 100644 --- a/docs/known-issues.md +++ b/docs/known-issues.md @@ -26,9 +26,11 @@ performance" below. ## Concurrent and contiguous read performance (measured 2026-09-26) -**Status:** open for chunked full reads at 16 threads until re-measured -(the causes identified below are fixed as of 2026-09-26); the contiguous -item is fixed (2026-09-26). Measured on +**Status:** fixed (2026-09-26). Re-measured on tank at `c5334b1`: full +reads of chunked deflate data at 16 threads run at 4944 MB/s against 3135 +for 16 h5py processes (1.58x; 0.69x-0.76x before), and contiguous reads +are 1.29x h5py on one thread (`BENCHMARKS.md`, "Results after in-place +chunk decoding"). The history below is kept for reference. Measured on tank with `concurrent_read` against h5py 3.16 / HDF5 2.0 (`BENCHMARKS.md`, "Concurrent reads"): - **Partly fixed 2026-09-26.** Full reads of chunked datasets from several threads