docs: say when a selection read decodes more than the selection
The READMEs said ds[...] reads only the selected elements, and the facade's read_selection docs that only intersecting chunks are decompressed. The bounding-box path runs only when the box covers at most half the dataset; larger boxes (any strided slice across the dataset), compact, virtual and unwritten datasets and chunked ones with a non-default fill value decode the whole dataset. The READMEs, the facade and format docs, the bindings' docstrings and known-issues now say so, and how index lists are read. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -29,7 +29,7 @@ with clawhdf5.File("data.h5", "r") as f:
|
||||
f.keys(), f["group"].items(), "group/data" in f
|
||||
ds = f["group/data"] # or f["/group/data"], f["group"]["data"]
|
||||
ds.shape, ds.dtype, ds.attrs["units"]
|
||||
ds[10:20, ::2] # only the selected elements are read
|
||||
ds[10:20, ::2] # a small selection reads only its chunks
|
||||
ds[-1], ds[..., 0], ds[[1, 4, 7]]
|
||||
np.asarray(ds)
|
||||
f["table"]["id"] # a compound field
|
||||
@@ -45,9 +45,19 @@ with clawhdf5.File("data.h5", "r") as f:
|
||||
increasing list of integers, compound field names. Each maps onto a
|
||||
hyperslab selection. `None`, negative steps and boolean masks are refused
|
||||
with h5py's errors.
|
||||
- What is read from the file: a selection whose bounding box covers at
|
||||
most half the dataset decodes only the chunks (or contiguous rows) the box
|
||||
overlaps. The library decodes the whole dataset for a larger box
|
||||
(including a strided slice such as `ds[::100]` across a chunked dataset),
|
||||
and for compact, virtual and unwritten datasets and chunked ones with a
|
||||
non-default fill value. An index list is read one group of neighbouring
|
||||
chunks at a time (a new group only past a chunk with no selected index),
|
||||
so each chunk is decoded once. `ds[()]`, `ds[...]` and `np.asarray(ds)`
|
||||
use the file's chunk cache; other selections do not.
|
||||
- The bytes the library reads become the numpy array's buffer without a
|
||||
copy, and the read runs with the GIL released, so threads read in
|
||||
parallel.
|
||||
parallel. A bug in the library (a Rust panic) raises
|
||||
`clawhdf5.InternalError`, a `RuntimeError`.
|
||||
- Attributes return what h5py returns; `clawhdf5.Empty` stands for a null
|
||||
dataspace (h5py's `Empty`).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user