cached() mapped an error of its open-time prefetch (a network error, a
changed file) to Error::Hdf5(Format(Storage)), misclassifying it for
callers that match on the variant. It is now Error::Remote
(RemoteError::Backend with the backend's message). open_object and the
s3/gs/az URLs fetch the first block of an ObjectStoreStorage directly, so
their errors keep their kind (FileChanged, ObjectStore).
Test: cached() over a backend whose reads fail gives Error::Remote(Backend)
(Error::Hdf5 before).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
It refused whenever Handle::try_current() was Ok, which is also the case
inside spawn_blocking threads — so the workaround its own error message
recommended failed the same way, and the backend could only be used from
a bare std::thread in a tokio application.
Reads are now spawned on the storage's own runtime and the caller waits on
a channel: the future never runs on the caller's thread, so neither a
spawn_blocking thread nor a current-thread runtime can deadlock or panic
(a read inside a runtime blocks that thread, like any blocking call; the
docs still recommend spawn_blocking there).
Tests: a read in spawn_blocking of a multi-thread runtime and a read inside
a current-thread runtime's task give File::open's values (both errors
before).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Every RemoteError message and HttpStorage's Debug output held the URL as
given, with any user:password@ and the query string — for a presigned
S3/GCS/Azure URL, its signature or token. An application logging the
error leaked the credential.
- New clawhdf5_remote::redact_url: no userinfo, no fragment, query values
replaced by REDACTED (plain key names kept).
- HttpStorage formats every message with the redacted URL, and scrubs the
URL's secret parts from errors of the HTTP client (whose texts can echo
the URI); Debug shows the redacted URL. storage_for_url's and the object
store URL errors are redacted too. HttpStorage::url() still returns the
URL as given, documented as not for logging.
- h5rs prints FILE arguments that are URLs redacted: in errors and in
dump/stat/check/diff output.
- The test server can force a status and send a wrong Content-Range.
Tests: 404, 403 (at open and on a read), wrong Content-Range (at open and
on a read), no range support, encoded body, ETag change, timeout,
connection closed and bad scheme errors, Display and Debug, contain none
of the secrets; h5rs likewise for every subcommand.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
ObjectStoreStorage (feature `object-store`, pure Rust) reads one object
of any object_store store by ranged get_opts, pinned at open by a head
request: If-Match with its ETag (and the ETag and size of every response
compared), else its version or modification time. A change is
RemoteError::FileChanged. object_store is async and Storage is not, so
the storage owns a small multi-threaded tokio runtime (two workers) and
blocks the calling thread on it; the ranges of one read_ranges call are
fetched concurrently (up to 8). From inside another tokio runtime it
refuses with RemoteError::Usage instead of blocking a worker, and it
shuts its runtime down in the background on drop so dropping it in async
code does not panic.
open_object(store, path, options) opens a file through a block cache
(first block prefetched); open_url accepts s3://, gs:// and az:// with
the `s3`, `gcs` and `azure` features, configured from the environment by
object_store's from_env builders. Those pull object_store's cloud clients
and aws-lc-rs (C), so they are opt-in; without them the URL is a clean
UnsupportedScheme error naming the feature.
Tests against object_store's in-memory and local-file stores (no cloud):
every fixture's transcript equals File::open's, a multi-block object is
fetched in coalesced block runs, an object replaced while open is an
error, and a missing object or a read from inside a runtime is a clean
error. ci-test.sh lints all backends, runs these tests (with s3 for its
URL parsing test) and checks object-store for C in the no-C step.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
