docs: the rayon fix covers a one-thread pool, not the h5py-process gap

The review measured the default pool unchanged (about 2900 MB/s at 16
threads before and after) and still short of 16 h5py processes; small
pools still make outside readers wait. Say so instead of marking the
scaling issue fixed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:48:05 -05:00
co-authored by Claude Opus 5.5
parent a5e41c1a53
commit 37770f594a
3 changed files with 14 additions and 9 deletions
+5 -5
View File
@@ -528,11 +528,11 @@ What this shows:
(about 880 MB/s) while h5py processes reach 4424 MB/s. Hyperslab
reads, which bypass the `File`'s chunk cache, keep scaling, so the
cache (one mutex and one 16 MiB budget per `File`, thrashed by 64 MiB
datasets) is the suspect. **Fixed after these measurements
(2026-09-26); the table above predates the fix and has not been
re-measured.** The cause was not the cache: with `--decode-threads 1`
every full read queued its chunks for the pool's single rayon worker;
see `docs/known-issues.md`.
datasets) is the suspect. The cause of the `--decode-threads 1`
ceiling was not the cache: every full read queued its chunks for the
pool's single rayon worker. That case was fixed after these
measurements (2026-09-26, not yet re-measured here). With the default
pool the gap to h5py processes remains (see `docs/known-issues.md`).
- *Contiguous reads are slow*: 2.5 GB/s for a single-threaded full read
against h5py's 9.8 GB/s (0.25x), and 0.12x for 256 x 256 hyperslabs.
Threads close the gap (about 1.0x h5py at 16), but single-thread