py: boolean-mask keys raise NotImplementedError, not TypeError

h5py supports boolean masks for reads and writes; clawhdf5 supports
neither, so a mask is an unsupported operation (NotImplementedError, as
for every other edit the bindings cannot do), not an invalid key.

Tests: test_unsupported_edits_are_clear_errors (1-D, N-D and per-axis
mask writes, file unchanged) and test_boolean_masks_are_refused (reads).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-27 07:48:04 -05:00
co-authored by Claude Opus 5.5
parent 173de3e0d2
commit b0930708b6
6 changed files with 24 additions and 10 deletions
+2
View File
@@ -98,6 +98,8 @@
compound fields by name, variable-length data, ...;
`docs/known-issues.md`).
- `File.mode`, `File.flush()` (a no-op), `Dataset.chunks`.
- Boolean-mask keys (`ds[mask]`, `ds[mask] = v`), which h5py supports,
raise `NotImplementedError` (they raised `TypeError`).
- Tests (`tests/test_edit.py`): each edit applied to two copies of a file,
by h5py and by clawhdf5, and both read back through h5py after every
edit, on files h5py writes with `libver` earliest, v114 and latest and on
+3 -2
View File
@@ -43,8 +43,9 @@ with clawhdf5.File("data.h5", "r") as f:
Other types raise `TypeError`.
- Keys are h5py's: integers, slices with a positive step, `...`, one
increasing list of integers, compound field names. Each maps onto a
hyperslab selection. `None`, negative steps and boolean masks are refused
with h5py's errors.
hyperslab selection. `None` and negative steps are refused
with h5py's errors; boolean masks (which h5py supports) raise
`NotImplementedError`, for reads and writes.
- What is read from the file: a selection whose bounding box covers at
most half the dataset decodes only the chunks (or contiguous rows) the box
overlaps. The library decodes the whole dataset for a larger box
+8 -5
View File
@@ -7,11 +7,12 @@
//! (negative from the end) drop their axis, slices must have a positive
//! step, one `Ellipsis` fills the unmentioned axes, a single increasing list
//! of integers may index one axis, and strings name compound fields.
//! Everything else (`None`/`np.newaxis`, boolean masks, several index lists)
//! is refused with the error h5py gives.
//! Everything else (`None`/`np.newaxis`, several index lists) is refused
//! with the error h5py gives; boolean masks, which h5py supports, raise
//! `NotImplementedError`.
use clawhdf5_format::selection::Selection;
use pyo3::exceptions::{PyIndexError, PyTypeError, PyValueError};
use pyo3::exceptions::{PyIndexError, PyNotImplementedError, PyTypeError, PyValueError};
use pyo3::prelude::*;
use pyo3::types::{PyEllipsis, PySlice, PyString, PyTuple};
@@ -379,8 +380,10 @@ fn parse_axis(py: Python<'_>, a: &Bound<'_, PyAny>, n: u64) -> PyResult<Axis> {
let arr = np.call_method1("asarray", (a,))?;
let kind: String = arr.getattr("dtype")?.getattr("kind")?.extract()?;
if kind == "b" {
return Err(PyTypeError::new_err(
"Boolean mask indexing is not supported by clawhdf5",
// h5py supports masks; clawhdf5 does not (yet), for reads or
// writes: an unsupported operation, not a wrong key.
return Err(PyNotImplementedError::new_err(
"boolean mask indexing is not supported by clawhdf5",
));
}
let ndim: usize = arr.getattr("ndim")?.extract()?;
+7
View File
@@ -643,6 +643,13 @@ def test_unsupported_edits_are_clear_errors(h5py, tmp_path):
f["chunk_ext"].resize(3, axis=2)
with pytest.raises(TypeError):
f["i4"][0] = np.array(["a"] * 10)
# h5py writes through boolean masks; clawhdf5 does not.
with pytest.raises(NotImplementedError, match="mask"):
f["u1"][np.arange(16) % 2 == 0] = 5
with pytest.raises(NotImplementedError, match="mask"):
f["i4"][f["i4"][()] > 30] = 0
with pytest.raises(NotImplementedError, match="mask"):
f["i4"][np.ones(6, dtype=bool), 2] = 0
assert snapshot(h5py, path) == before
h5dump_reads(path)
@@ -440,7 +440,7 @@ def test_unsupported_types_are_errors_not_data(pair):
def test_boolean_masks_are_refused(pair):
ours, _, _ = pair
with pytest.raises(TypeError):
with pytest.raises(NotImplementedError, match="mask"):
ours["num/le_i4_1d"][np.ones(37, dtype=bool)]
+3 -2
View File
@@ -164,8 +164,9 @@ before anything is written. On top of them:
elements, variable-length data, strings padded with spaces or
NUL-terminated (libhdf5 converts those differently from numpy; NUL-padded
ones, h5py's, are writable), compounds containing such strings, null
dataspaces, and index-list writes of more than 2²² elements (write them
in slices).
dataspaces, index-list writes of more than 2²² elements (write them
in slices), and boolean-mask keys (`ds[mask] = v`, and mask reads;
h5py supports both).
- **`str` attributes are fixed-length UTF-8**, where h5py writes
variable-length strings: h5py reads them back as `bytes`
(`numpy.bytes_`), not `str`.