docs: say when a selection read decodes more than the selection

The READMEs said ds[...] reads only the selected elements, and the
facade's read_selection docs that only intersecting chunks are
decompressed. The bounding-box path runs only when the box covers at
most half the dataset; larger boxes (any strided slice across the
dataset), compact, virtual and unwritten datasets and chunked ones with
a non-default fill value decode the whole dataset. The READMEs, the
facade and format docs, the bindings' docstrings and known-issues now
say so, and how index lists are read.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 09:04:48 -05:00
co-authored by Claude Opus 5.5
parent 17edfe2cf0
commit 45d617c39e
8 changed files with 72 additions and 16 deletions
+17
View File
@@ -7,6 +7,23 @@ deleting it.
---
## Selection reads that decode more than the selection
**Status:** open (documented 2026-09-26). `Dataset::read_selection` (and so
the Python `ds[...]`) materialises only the selection's bounding box when
that box covers at most half the dataset (`partial_read`). It decodes the
whole dataset and extracts the selection instead when:
- the bounding box covers more than half the dataset — which a strided
selection across a chunked dataset (`ds[::100]`) always does, although
it may touch few chunks;
- the dataset is compact or virtual, or has no storage;
- it is chunked with a non-default fill value (the box path does not fill
unallocated chunks, so the fill-aware full read is used).
Values are correct in every case; this is cost only. Selections other than
`Selection::All` also bypass the file's chunk cache. The bounding-box
heuristic's other cost is measured under "Concurrent and contiguous read
performance" below.
## Concurrent and contiguous read performance (measured 2026-09-26)
**Status:** open. Measured on tank with `concurrent_read` against h5py