osobhandClaude Opus 5.5
910d81904c
The Python bindings could not open a remote file: they parsed through File::as_bytes() in eight places (path lookups, object headers, dataspaces, attributes, group listings, the global heap of variable-length data), which a storage-backed file does not have. - Every object of a File now shares one handle (src/handle.rs) that runs all file access, metadata included, with the GIL released and parses through File::storage() and the clawhdf5_format *_in functions. Local files take the same path (their storage is the mmap). - clawhdf5.File(url) opens any scheme://... through clawhdf5_remote::storage_for_url (read-only; another mode is a ValueError). File.open_url(url, **options) takes the cache and HTTP options (block_size, cache_size, headers, retries, timeout, allow_full_download, max_full_download, require_validator, max_redirects, max_parallel); File.remote_stats gives the block cache's counters. - Default build: plain HTTP only, no C. https (rustls/ring) and s3/gcs/azure (aws-lc-rs) are opt-in features of clawhdf5-py, and ci-test.sh's no-C check now covers the crate. - A failed storage read (network error, file changed on the server) is an OSError, never KeyError/ValueError and never data; `key in group` raises it instead of answering False. Tests: the read-vs-h5py suite runs locally and over HTTP (1 MiB and 1 KiB blocks) against a range-capable http.server in the test process (conftest.RangeServer); test_remote.py covers request counts, cache hits, a server without Range support, a changed file, a server that hangs up, 16 threads, and a spinning thread that keeps running while a read waits on 0.2 s requests. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
39 lines
1.2 KiB
TOML
39 lines
1.2 KiB
TOML
[package]
|
|
name = "clawhdf5-py"
|
|
version = "2.7.0"
|
|
edition = "2024"
|
|
rust-version.workspace = true
|
|
description = "Python bindings for clawhdf5 — a pure-Rust HDF5 library"
|
|
license = "MIT"
|
|
repository = "https://git.redclaw.dev/quantumclaw/clawhdf5"
|
|
readme = "README.md"
|
|
keywords = ["hdf5", "python", "bindings", "science"]
|
|
categories = ["api-bindings", "science"]
|
|
|
|
[lib]
|
|
name = "clawhdf5"
|
|
crate-type = ["cdylib", "rlib"]
|
|
|
|
[dependencies]
|
|
clawhdf5_rs = { path = "../clawhdf5", version = "2.7.0", package = "clawhdf5" }
|
|
clawhdf5-format = { path = "../clawhdf5-format", version = "2.7.0" }
|
|
# Remote files (`clawhdf5.File(url)`): plain HTTP by default, which builds
|
|
# no C. HTTPS and the object stores are opt-in features below.
|
|
clawhdf5-remote = { path = "../clawhdf5-remote", version = "2.7.0" }
|
|
pyo3 = "0.29"
|
|
numpy = "0.29"
|
|
|
|
[features]
|
|
extension-module = ["pyo3/extension-module"]
|
|
# https:// URLs (rustls with ring, which compiles C and assembly).
|
|
https = ["clawhdf5-remote/https"]
|
|
# s3://, gs://, az:// URLs (object_store; its cloud clients build aws-lc-rs, C).
|
|
s3 = ["clawhdf5-remote/s3"]
|
|
gcs = ["clawhdf5-remote/gcs"]
|
|
azure = ["clawhdf5-remote/azure"]
|
|
|
|
[package.metadata.docs.rs]
|
|
features = []
|
|
no-default-features = true
|
|
targets = ["x86_64-unknown-linux-gnu"]
|