fix(py): index lists of padded compounds no longer return uninitialised padding

np.concatenate copies structured dtypes field by field into np.empty, so
the padding of ds[[0, 3, 6]] held process memory. The runs' bytes are
joined in Rust, whole elements at a time, before anything becomes numpy:
the padding is the file's bytes (h5py's) and the result is still a view
of the Rust buffer. The h5py comparisons now compare every byte of
structured values; the new test failed on the padding before.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 08:54:04 -05:00
co-authored by Claude Opus 5.5
parent 24412a0e59
commit 8c51b05b9c
4 changed files with 116 additions and 52 deletions
+9
View File
@@ -60,6 +60,15 @@
is now guarded and a panic becomes `clawhdf5.InternalError` (a
`RuntimeError`) naming the object; with the implicit-index panic above
restored, `ds[0:30]` raises it.
- **Wrong data: uninitialised padding in compound results of index lists.**
`ds[[0, 3, 6]]` joined one read per run with `np.concatenate`, which
copies structured dtypes field by field into an `np.empty` result, so the
padding bytes held whatever was in memory (pointers were seen) and leaked
through `tobytes()`, hashes and write-backs. The runs' bytes are now joined
in Rust, whole elements at a time, so the result carries the bytes read
from the file (h5py's, zero for files it wrote) and stays zero-copy.
The h5py comparisons now also compare every byte of structured values
(`test_compound_padding_bytes_match_h5py` and `assert_same`).
- **CI builds and tests the Python package.** It was excluded from CI.
`scripts/ci-test.sh` now lints `clawhdf5-py`, builds the wheel with
maturin, unpacks it under `target/` and runs the pytest suite; skipped