docs: say when a selection read decodes more than the selection

The READMEs said ds[...] reads only the selected elements, and the
facade's read_selection docs that only intersecting chunks are
decompressed. The bounding-box path runs only when the box covers at
most half the dataset; larger boxes (any strided slice across the
dataset), compact, virtual and unwritten datasets and chunked ones with
a non-default fill value decode the whole dataset. The READMEs, the
facade and format docs, the bindings' docstrings and known-issues now
say so, and how index lists are read.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 09:04:48 -05:00
co-authored by Claude Opus 5.5
parent 17edfe2cf0
commit 45d617c39e
8 changed files with 72 additions and 16 deletions
+12 -2
View File
@@ -29,7 +29,7 @@ with clawhdf5.File("data.h5", "r") as f:
f.keys(), f["group"].items(), "group/data" in f
ds = f["group/data"] # or f["/group/data"], f["group"]["data"]
ds.shape, ds.dtype, ds.attrs["units"]
ds[10:20, ::2] # only the selected elements are read
ds[10:20, ::2] # a small selection reads only its chunks
ds[-1], ds[..., 0], ds[[1, 4, 7]]
np.asarray(ds)
f["table"]["id"] # a compound field
@@ -45,9 +45,19 @@ with clawhdf5.File("data.h5", "r") as f:
increasing list of integers, compound field names. Each maps onto a
hyperslab selection. `None`, negative steps and boolean masks are refused
with h5py's errors.
- What is read from the file: a selection whose bounding box covers at
most half the dataset decodes only the chunks (or contiguous rows) the box
overlaps. The library decodes the whole dataset for a larger box
(including a strided slice such as `ds[::100]` across a chunked dataset),
and for compact, virtual and unwritten datasets and chunked ones with a
non-default fill value. An index list is read one group of neighbouring
chunks at a time (a new group only past a chunk with no selected index),
so each chunk is decoded once. `ds[()]`, `ds[...]` and `np.asarray(ds)`
use the file's chunk cache; other selections do not.
- The bytes the library reads become the numpy array's buffer without a
copy, and the read runs with the GIL released, so threads read in
parallel.
parallel. A bug in the library (a Rust panic) raises
`clawhdf5.InternalError`, a `RuntimeError`.
- Attributes return what h5py returns; `clawhdf5.Empty` stands for a null
dataspace (h5py's `Empty`).