docs: the Python package — install with maturin, h5py-style reading

README gains a Python section (maturin develop into a venv, a reading example that was run against an h5py-written file, the supported types and keys, what writing covers). The crate README says the same in more detail. QUICKSTART showed clawhdf5.open()/read_f64(), which never existed; it now shows File(...)[...]. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-26 08:21:48 -05:00
parent f7d88bb4fb
commit c3850a0b66
3 changed files with 107 additions and 14 deletions
@@ -407,6 +407,48 @@ let values = ds.read_f64()?;
 assert_eq!(values, vec![22.5, 23.1, 21.8]);
 ```

+### Python
+
+`crates/clawhdf5-py` is a Python package (PyO3 + numpy) that reads HDF5 with
+an h5py-shaped API and no libhdf5. It is not on PyPI; build it with
+[maturin](https://www.maturin.rs) into a virtualenv:
+
+```bash
+python -m venv .venv && . .venv/bin/activate
+pip install maturin numpy
+maturin develop --release -m crates/clawhdf5-py/Cargo.toml
+python -c "import clawhdf5; print(clawhdf5.__version__)"
+```
+
+```python
+import numpy as np
+import clawhdf5
+
+with clawhdf5.File("data.h5", "r") as f:
+    print(list(f.keys()))              # sorted member names, like h5py
+    ds = f["group/temperatures"]       # relative or absolute ("/group/...") paths
+    print(ds.shape, ds.dtype)          # dtype is the numpy dtype h5py reports
+    block = ds[100:200, ::4]           # reads only the selected elements
+    row = ds[-1]                       # integers drop the axis
+    picked = ds[[1, 5, 9], :]          # one increasing index list per key
+    units = ds.attrs["units"]          # attributes come back as h5py returns them
+    everything = np.asarray(ds)
+
+    records = f["table"]               # compound -> numpy structured array
+    ids = records["id"]                # one field
+```
+
+Reads cover integers and IEEE floats of every width in either byte order,
+`bool`, enums, complex, fixed and variable-length strings, variable-length
+sequences, opaque, HDF5 array types and compounds; other types (references,
+bitfields, ...) raise `TypeError` instead of returning guessed data. Keys
+follow h5py (negative steps, `None` and boolean masks are refused). The
+read itself runs with the GIL released, so Python threads read in parallel.
+Writing (`File(path, "w")`, `create_dataset`, `create_group`, `attrs[...] =`)
+covers `float64`, `float32`, `int64`, `int32` and `uint8` arrays. The tests
+in `crates/clawhdf5-py/tests` compare every read with h5py; run them with
+`pip install pytest h5py && pytest crates/clawhdf5-py/tests`.
+
 ### Agent Memory

 ```rust
@@ -3,23 +3,71 @@
 [![crates.io](https://img.shields.io/crates/v/clawhdf5-py.svg)](https://crates.io/crates/clawhdf5-py)
 [![docs.rs](https://docs.rs/clawhdf5-py/badge.svg)](https://docs.rs/clawhdf5-py)

-Python bindings for clawhdf5 — a pure-Rust HDF5 library.
+Python bindings for clawhdf5 — a pure-Rust HDF5 library. The package is
+`clawhdf5` (`import clawhdf5`); it needs numpy and no libhdf5.

-## Features
+## Install

- h5py-compatible API (`File`, `Group`, `Dataset`)
- NumPy array integration
- Read and write HDF5 files from Python with no C dependencies
+Not on PyPI yet. Build it into a virtualenv with [maturin](https://www.maturin.rs):

-## Usage
+```bash
+pip install maturin numpy
+cd crates/clawhdf5-py
+maturin develop --release
+python -c "import clawhdf5; print(clawhdf5.__version__)"
+```
+
+## Reading
+
+The read API follows h5py:

 ```python
+import numpy as np
 import clawhdf5

-with clawhdf5.File('data.h5', 'r') as f:
-    data = f['/dataset'][:]
+with clawhdf5.File("data.h5", "r") as f:
+    f.keys(), f["group"].items(), "group/data" in f
+    ds = f["group/data"]           # or f["/group/data"], f["group"]["data"]
+    ds.shape, ds.dtype, ds.attrs["units"]
+    ds[10:20, ::2]                 # only the selected elements are read
+    ds[-1], ds[..., 0], ds[[1, 4, 7]]
+    np.asarray(ds)
+    f["table"]["id"]               # a compound field
 ```

+- `Dataset.dtype` is the numpy dtype h5py reports: integers and IEEE floats
+  of every width in either byte order, `bool`, enums (with
+  `dtype.metadata['enum']`), complex, `S<n>` fixed strings, `object` for
+  variable-length strings (`bytes` values) and sequences (array values),
+  `V<n>` opaque, array types, and compounds as structured dtypes.
+  Other types raise `TypeError`.
+- Keys are h5py's: integers, slices with a positive step, `...`, one
+  increasing list of integers, compound field names. Each maps onto a
+  hyperslab selection. `None`, negative steps and boolean masks are refused
+  with h5py's errors.
+- The bytes the library reads become the numpy array's buffer without a
+  copy, and the read runs with the GIL released, so threads read in
+  parallel.
+- Attributes return what h5py returns; `clawhdf5.Empty` stands for a null
+  dataspace (h5py's `Empty`).
+
+## Writing
+
+`clawhdf5.File(path, "w")` with `create_dataset(name, data=array,
+chunks=..., compression="gzip")`, `create_group` and `attrs[...] = ...`
+writes `float64`, `float32`, `int64`, `int32` and `uint8` arrays; the file is
+written on `close()`.
+
+## Tests
+
+```bash
+pip install pytest h5py
+pytest crates/clawhdf5-py/tests
+```
+
+`tests/test_read_vs_h5py.py` compares every read with h5py on a file h5py
+writes. `scripts/ci-test.sh` builds the wheel and runs these in CI.
+
 ## License

 MIT
@@ -395,19 +395,22 @@ clawhdf5 --path agent.h5 snapshot backup_2026-03-19.h5
 Read HDF5 files from Python without libhdf5:

 ```bash
-pip install clawhdf5  # coming soon — build from source for now
-cd crates/clawhdf5-py && maturin develop
+# Not on PyPI yet: build from source into a virtualenv
+pip install maturin numpy
+cd crates/clawhdf5-py && maturin develop --release
 ```

 ```python
 import clawhdf5

-# Read
-f = clawhdf5.open("data.h5")
-temps = f.read_f64("temperatures")
-print(temps)  # [22.5, 23.1, 21.8]
+# Read (h5py-style)
+with clawhdf5.File("data.h5", "r") as f:
+    temps = f["temperatures"][:]
+    print(temps)  # [22.5 23.1 21.8]
 ```

+See `crates/clawhdf5-py/README.md` for the supported types and indexing.
+
 ---

 ## Common Patterns