docs: the Python package — install with maturin, h5py-style reading
README gains a Python section (maturin develop into a venv, a reading example that was run against an h5py-written file, the supported types and keys, what writing covers). The crate README says the same in more detail. QUICKSTART showed clawhdf5.open()/read_f64(), which never existed; it now shows File(...)[...]. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -407,6 +407,48 @@ let values = ds.read_f64()?;
|
|||||||
assert_eq!(values, vec![22.5, 23.1, 21.8]);
|
assert_eq!(values, vec![22.5, 23.1, 21.8]);
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Python
|
||||||
|
|
||||||
|
`crates/clawhdf5-py` is a Python package (PyO3 + numpy) that reads HDF5 with
|
||||||
|
an h5py-shaped API and no libhdf5. It is not on PyPI; build it with
|
||||||
|
[maturin](https://www.maturin.rs) into a virtualenv:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m venv .venv && . .venv/bin/activate
|
||||||
|
pip install maturin numpy
|
||||||
|
maturin develop --release -m crates/clawhdf5-py/Cargo.toml
|
||||||
|
python -c "import clawhdf5; print(clawhdf5.__version__)"
|
||||||
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
import numpy as np
|
||||||
|
import clawhdf5
|
||||||
|
|
||||||
|
with clawhdf5.File("data.h5", "r") as f:
|
||||||
|
print(list(f.keys())) # sorted member names, like h5py
|
||||||
|
ds = f["group/temperatures"] # relative or absolute ("/group/...") paths
|
||||||
|
print(ds.shape, ds.dtype) # dtype is the numpy dtype h5py reports
|
||||||
|
block = ds[100:200, ::4] # reads only the selected elements
|
||||||
|
row = ds[-1] # integers drop the axis
|
||||||
|
picked = ds[[1, 5, 9], :] # one increasing index list per key
|
||||||
|
units = ds.attrs["units"] # attributes come back as h5py returns them
|
||||||
|
everything = np.asarray(ds)
|
||||||
|
|
||||||
|
records = f["table"] # compound -> numpy structured array
|
||||||
|
ids = records["id"] # one field
|
||||||
|
```
|
||||||
|
|
||||||
|
Reads cover integers and IEEE floats of every width in either byte order,
|
||||||
|
`bool`, enums, complex, fixed and variable-length strings, variable-length
|
||||||
|
sequences, opaque, HDF5 array types and compounds; other types (references,
|
||||||
|
bitfields, ...) raise `TypeError` instead of returning guessed data. Keys
|
||||||
|
follow h5py (negative steps, `None` and boolean masks are refused). The
|
||||||
|
read itself runs with the GIL released, so Python threads read in parallel.
|
||||||
|
Writing (`File(path, "w")`, `create_dataset`, `create_group`, `attrs[...] =`)
|
||||||
|
covers `float64`, `float32`, `int64`, `int32` and `uint8` arrays. The tests
|
||||||
|
in `crates/clawhdf5-py/tests` compare every read with h5py; run them with
|
||||||
|
`pip install pytest h5py && pytest crates/clawhdf5-py/tests`.
|
||||||
|
|
||||||
### Agent Memory
|
### Agent Memory
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
|
|||||||
@@ -3,23 +3,71 @@
|
|||||||
[](https://crates.io/crates/clawhdf5-py)
|
[](https://crates.io/crates/clawhdf5-py)
|
||||||
[](https://docs.rs/clawhdf5-py)
|
[](https://docs.rs/clawhdf5-py)
|
||||||
|
|
||||||
Python bindings for clawhdf5 — a pure-Rust HDF5 library.
|
Python bindings for clawhdf5 — a pure-Rust HDF5 library. The package is
|
||||||
|
`clawhdf5` (`import clawhdf5`); it needs numpy and no libhdf5.
|
||||||
|
|
||||||
## Features
|
## Install
|
||||||
|
|
||||||
- h5py-compatible API (`File`, `Group`, `Dataset`)
|
Not on PyPI yet. Build it into a virtualenv with [maturin](https://www.maturin.rs):
|
||||||
- NumPy array integration
|
|
||||||
- Read and write HDF5 files from Python with no C dependencies
|
|
||||||
|
|
||||||
## Usage
|
```bash
|
||||||
|
pip install maturin numpy
|
||||||
|
cd crates/clawhdf5-py
|
||||||
|
maturin develop --release
|
||||||
|
python -c "import clawhdf5; print(clawhdf5.__version__)"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Reading
|
||||||
|
|
||||||
|
The read API follows h5py:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
import numpy as np
|
||||||
import clawhdf5
|
import clawhdf5
|
||||||
|
|
||||||
with clawhdf5.File('data.h5', 'r') as f:
|
with clawhdf5.File("data.h5", "r") as f:
|
||||||
data = f['/dataset'][:]
|
f.keys(), f["group"].items(), "group/data" in f
|
||||||
|
ds = f["group/data"] # or f["/group/data"], f["group"]["data"]
|
||||||
|
ds.shape, ds.dtype, ds.attrs["units"]
|
||||||
|
ds[10:20, ::2] # only the selected elements are read
|
||||||
|
ds[-1], ds[..., 0], ds[[1, 4, 7]]
|
||||||
|
np.asarray(ds)
|
||||||
|
f["table"]["id"] # a compound field
|
||||||
```
|
```
|
||||||
|
|
||||||
|
- `Dataset.dtype` is the numpy dtype h5py reports: integers and IEEE floats
|
||||||
|
of every width in either byte order, `bool`, enums (with
|
||||||
|
`dtype.metadata['enum']`), complex, `S<n>` fixed strings, `object` for
|
||||||
|
variable-length strings (`bytes` values) and sequences (array values),
|
||||||
|
`V<n>` opaque, array types, and compounds as structured dtypes.
|
||||||
|
Other types raise `TypeError`.
|
||||||
|
- Keys are h5py's: integers, slices with a positive step, `...`, one
|
||||||
|
increasing list of integers, compound field names. Each maps onto a
|
||||||
|
hyperslab selection. `None`, negative steps and boolean masks are refused
|
||||||
|
with h5py's errors.
|
||||||
|
- The bytes the library reads become the numpy array's buffer without a
|
||||||
|
copy, and the read runs with the GIL released, so threads read in
|
||||||
|
parallel.
|
||||||
|
- Attributes return what h5py returns; `clawhdf5.Empty` stands for a null
|
||||||
|
dataspace (h5py's `Empty`).
|
||||||
|
|
||||||
|
## Writing
|
||||||
|
|
||||||
|
`clawhdf5.File(path, "w")` with `create_dataset(name, data=array,
|
||||||
|
chunks=..., compression="gzip")`, `create_group` and `attrs[...] = ...`
|
||||||
|
writes `float64`, `float32`, `int64`, `int32` and `uint8` arrays; the file is
|
||||||
|
written on `close()`.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install pytest h5py
|
||||||
|
pytest crates/clawhdf5-py/tests
|
||||||
|
```
|
||||||
|
|
||||||
|
`tests/test_read_vs_h5py.py` compares every read with h5py on a file h5py
|
||||||
|
writes. `scripts/ci-test.sh` builds the wheel and runs these in CI.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
MIT
|
MIT
|
||||||
|
|||||||
+9
-6
@@ -395,19 +395,22 @@ clawhdf5 --path agent.h5 snapshot backup_2026-03-19.h5
|
|||||||
Read HDF5 files from Python without libhdf5:
|
Read HDF5 files from Python without libhdf5:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install clawhdf5 # coming soon — build from source for now
|
# Not on PyPI yet: build from source into a virtualenv
|
||||||
cd crates/clawhdf5-py && maturin develop
|
pip install maturin numpy
|
||||||
|
cd crates/clawhdf5-py && maturin develop --release
|
||||||
```
|
```
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import clawhdf5
|
import clawhdf5
|
||||||
|
|
||||||
# Read
|
# Read (h5py-style)
|
||||||
f = clawhdf5.open("data.h5")
|
with clawhdf5.File("data.h5", "r") as f:
|
||||||
temps = f.read_f64("temperatures")
|
temps = f["temperatures"][:]
|
||||||
print(temps) # [22.5, 23.1, 21.8]
|
print(temps) # [22.5 23.1 21.8]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
See `crates/clawhdf5-py/README.md` for the supported types and indexing.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Common Patterns
|
## Common Patterns
|
||||||
|
|||||||
Reference in New Issue
Block a user