clawhdf5-remote: object stores through object_store (S3, GCS, Azure)

ObjectStoreStorage (feature `object-store`, pure Rust) reads one object
of any object_store store by ranged get_opts, pinned at open by a head
request: If-Match with its ETag (and the ETag and size of every response
compared), else its version or modification time. A change is
RemoteError::FileChanged. object_store is async and Storage is not, so
the storage owns a small multi-threaded tokio runtime (two workers) and
blocks the calling thread on it; the ranges of one read_ranges call are
fetched concurrently (up to 8). From inside another tokio runtime it
refuses with RemoteError::Usage instead of blocking a worker, and it
shuts its runtime down in the background on drop so dropping it in async
code does not panic.

open_object(store, path, options) opens a file through a block cache
(first block prefetched); open_url accepts s3://, gs:// and az:// with
the `s3`, `gcs` and `azure` features, configured from the environment by
object_store's from_env builders. Those pull object_store's cloud clients
and aws-lc-rs (C), so they are opt-in; without them the URL is a clean
UnsupportedScheme error naming the feature.

Tests against object_store's in-memory and local-file stores (no cloud):
every fixture's transcript equals File::open's, a multi-block object is
fetched in coalesced block runs, an object replaced while open is an
error, and a missing object or a read from inside a runtime is a clean
error. ci-test.sh lints all backends, runs these tests (with s3 for its
URL parsing test) and checks object-store for C in the no-C step.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

scripts/ci-test.sh

@@ -109,11 +109,12 @@ run_step "cargo clippy (fast-deflate / zlib-ng)" cargo clippy \
     --features clawhdf5-format/fast-deflate,clawhdf5-filters/fast-deflate \
     -- -D warnings
 
-# clawhdf5-remote's HTTPS backend (rustls).
-run_step "cargo clippy (remote, https)" cargo clippy \
+# clawhdf5-remote's optional backends: object_store (in-memory and local
+# stores in the tests), HTTPS through rustls, and the cloud stores.
+run_step "cargo clippy (remote, all backends)" cargo clippy \
     -p clawhdf5-remote \
     --all-targets \
-    --features https \
+    --features object-store,https,s3,gcs,azure \
     -- -D warnings
 
 # The README promises that the core crates build no C by default. Hold it to
@@ -121,15 +122,16 @@ run_step "cargo clippy (remote, https)" cargo clippy \
 # default dependency tree of any of them. clawhdf5-migrate (bundled SQLite),
 # clawhdf5-napi (Node) and clawhdf5-gpu (graphics drivers) are exempt.
 # js-sys (clawhdf5-wasm's bindings to JavaScript) builds no C.
-# clawhdf5-remote is checked by default (plain HTTP); its https feature
-# (ring) builds C and is opt-in.
+# clawhdf5-remote is checked by default (plain HTTP) and with its
+# object-store feature; its https (ring) and s3/gcs/azure (aws-lc-rs)
+# features build C and are opt-in.
 no_c_in_default_build() {
     local entry crate features found=0
     for entry in clawhdf5-format clawhdf5-io clawhdf5-filters clawhdf5 \
         clawhdf5-agent clawhdf5-ann clawhdf5-accel clawhdf5-netcdf4 clawhdf5-cli \
         clawhdf5-tools \
         clawhdf5-wasm \
-        clawhdf5-remote; do
+        clawhdf5-remote clawhdf5-remote:object-store; do
         crate=${entry%%:*}
         features=()
         [ "$entry" != "$crate" ] && features=(--features "${entry#*:}")
@@ -204,6 +206,10 @@ run_step "cargo test (facade parallel)" cargo test \
     -p clawhdf5 \
     --features parallel
 
+run_step "cargo test (remote, object_store backend, s3 URLs)" cargo test \
+    -p clawhdf5-remote \
+    --features object-store,s3
+
 run_step "cargo test (ann parallel)" cargo test \
     -p clawhdf5-ann \
     --features parallel