feat(py): h5py-style reads of only the selected elements, GIL released
ds[key] read the whole dataset and sliced it in numpy, and knew six dtypes. Keys (ints, positive-step slices, Ellipsis, one increasing index list, compound field names) now map onto hyperslab selections, and the facade's read_selection bytes become the numpy buffer without a copy (PyArray::from_vec viewed as the dtype). dtype mapping follows h5py for all integer/IEEE float widths and byte orders, bool, enum, complex, fixed and variable-length strings, vlen sequences, opaque, array types and (nested, padded) compounds; anything it cannot describe exactly is a TypeError. Attributes return what h5py returns; groups and files gain the rest of the h5py mapping interface. Reads run under py.detach. tests/test_read_vs_h5py.py compares >500 reads with h5py 3.16 on an h5py-written file, checks errors match, that a damaged chunk outside the selection is never touched, and 8 threads reading at once. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -0,0 +1,18 @@
|
||||
"""Shared fixtures for the clawhdf5 Python binding tests."""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def h5py():
|
||||
"""h5py, or a skip — unless CLAWHDF5_REQUIRE_INTEROP=1, which makes a
|
||||
missing h5py a failure (as for the Rust interop suites)."""
|
||||
try:
|
||||
import h5py as mod
|
||||
except ImportError:
|
||||
if os.environ.get("CLAWHDF5_REQUIRE_INTEROP") == "1":
|
||||
pytest.fail("h5py is required (CLAWHDF5_REQUIRE_INTEROP=1) but not importable")
|
||||
pytest.skip("h5py not installed")
|
||||
return mod
|
||||
Reference in New Issue
Block a user