py: boolean-mask keys raise NotImplementedError, not TypeError
h5py supports boolean masks for reads and writes; clawhdf5 supports neither, so a mask is an unsupported operation (NotImplementedError, as for every other edit the bindings cannot do), not an invalid key. Tests: test_unsupported_edits_are_clear_errors (1-D, N-D and per-axis mask writes, file unchanged) and test_boolean_masks_are_refused (reads). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -98,6 +98,8 @@
|
||||
compound fields by name, variable-length data, ...;
|
||||
`docs/known-issues.md`).
|
||||
- `File.mode`, `File.flush()` (a no-op), `Dataset.chunks`.
|
||||
- Boolean-mask keys (`ds[mask]`, `ds[mask] = v`), which h5py supports,
|
||||
raise `NotImplementedError` (they raised `TypeError`).
|
||||
- Tests (`tests/test_edit.py`): each edit applied to two copies of a file,
|
||||
by h5py and by clawhdf5, and both read back through h5py after every
|
||||
edit, on files h5py writes with `libver` earliest, v114 and latest and on
|
||||
|
||||
@@ -43,8 +43,9 @@ with clawhdf5.File("data.h5", "r") as f:
|
||||
Other types raise `TypeError`.
|
||||
- Keys are h5py's: integers, slices with a positive step, `...`, one
|
||||
increasing list of integers, compound field names. Each maps onto a
|
||||
hyperslab selection. `None`, negative steps and boolean masks are refused
|
||||
with h5py's errors.
|
||||
hyperslab selection. `None` and negative steps are refused
|
||||
with h5py's errors; boolean masks (which h5py supports) raise
|
||||
`NotImplementedError`, for reads and writes.
|
||||
- What is read from the file: a selection whose bounding box covers at
|
||||
most half the dataset decodes only the chunks (or contiguous rows) the box
|
||||
overlaps. The library decodes the whole dataset for a larger box
|
||||
|
||||
@@ -7,11 +7,12 @@
|
||||
//! (negative from the end) drop their axis, slices must have a positive
|
||||
//! step, one `Ellipsis` fills the unmentioned axes, a single increasing list
|
||||
//! of integers may index one axis, and strings name compound fields.
|
||||
//! Everything else (`None`/`np.newaxis`, boolean masks, several index lists)
|
||||
//! is refused with the error h5py gives.
|
||||
//! Everything else (`None`/`np.newaxis`, several index lists) is refused
|
||||
//! with the error h5py gives; boolean masks, which h5py supports, raise
|
||||
//! `NotImplementedError`.
|
||||
|
||||
use clawhdf5_format::selection::Selection;
|
||||
use pyo3::exceptions::{PyIndexError, PyTypeError, PyValueError};
|
||||
use pyo3::exceptions::{PyIndexError, PyNotImplementedError, PyTypeError, PyValueError};
|
||||
use pyo3::prelude::*;
|
||||
use pyo3::types::{PyEllipsis, PySlice, PyString, PyTuple};
|
||||
|
||||
@@ -379,8 +380,10 @@ fn parse_axis(py: Python<'_>, a: &Bound<'_, PyAny>, n: u64) -> PyResult<Axis> {
|
||||
let arr = np.call_method1("asarray", (a,))?;
|
||||
let kind: String = arr.getattr("dtype")?.getattr("kind")?.extract()?;
|
||||
if kind == "b" {
|
||||
return Err(PyTypeError::new_err(
|
||||
"Boolean mask indexing is not supported by clawhdf5",
|
||||
// h5py supports masks; clawhdf5 does not (yet), for reads or
|
||||
// writes: an unsupported operation, not a wrong key.
|
||||
return Err(PyNotImplementedError::new_err(
|
||||
"boolean mask indexing is not supported by clawhdf5",
|
||||
));
|
||||
}
|
||||
let ndim: usize = arr.getattr("ndim")?.extract()?;
|
||||
|
||||
@@ -643,6 +643,13 @@ def test_unsupported_edits_are_clear_errors(h5py, tmp_path):
|
||||
f["chunk_ext"].resize(3, axis=2)
|
||||
with pytest.raises(TypeError):
|
||||
f["i4"][0] = np.array(["a"] * 10)
|
||||
# h5py writes through boolean masks; clawhdf5 does not.
|
||||
with pytest.raises(NotImplementedError, match="mask"):
|
||||
f["u1"][np.arange(16) % 2 == 0] = 5
|
||||
with pytest.raises(NotImplementedError, match="mask"):
|
||||
f["i4"][f["i4"][()] > 30] = 0
|
||||
with pytest.raises(NotImplementedError, match="mask"):
|
||||
f["i4"][np.ones(6, dtype=bool), 2] = 0
|
||||
assert snapshot(h5py, path) == before
|
||||
h5dump_reads(path)
|
||||
|
||||
|
||||
@@ -440,7 +440,7 @@ def test_unsupported_types_are_errors_not_data(pair):
|
||||
|
||||
def test_boolean_masks_are_refused(pair):
|
||||
ours, _, _ = pair
|
||||
with pytest.raises(TypeError):
|
||||
with pytest.raises(NotImplementedError, match="mask"):
|
||||
ours["num/le_i4_1d"][np.ones(37, dtype=bool)]
|
||||
|
||||
|
||||
|
||||
@@ -164,8 +164,9 @@ before anything is written. On top of them:
|
||||
elements, variable-length data, strings padded with spaces or
|
||||
NUL-terminated (libhdf5 converts those differently from numpy; NUL-padded
|
||||
ones, h5py's, are writable), compounds containing such strings, null
|
||||
dataspaces, and index-list writes of more than 2²² elements (write them
|
||||
in slices).
|
||||
dataspaces, index-list writes of more than 2²² elements (write them
|
||||
in slices), and boolean-mask keys (`ds[mask] = v`, and mask reads;
|
||||
h5py supports both).
|
||||
- **`str` attributes are fixed-length UTF-8**, where h5py writes
|
||||
variable-length strings: h5py reads them back as `bytes`
|
||||
(`numpy.bytes_`), not `str`.
|
||||
|
||||
Reference in New Issue
Block a user