clawhdf5-remote: ObjectStoreStorage works from any thread

It refused whenever Handle::try_current() was Ok, which is also the case
inside spawn_blocking threads — so the workaround its own error message
recommended failed the same way, and the backend could only be used from
a bare std::thread in a tokio application.

Reads are now spawned on the storage's own runtime and the caller waits on
a channel: the future never runs on the caller's thread, so neither a
spawn_blocking thread nor a current-thread runtime can deadlock or panic
(a read inside a runtime blocks that thread, like any blocking call; the
docs still recommend spawn_blocking there).

Tests: a read in spawn_blocking of a multi-thread runtime and a read inside
a current-thread runtime's task give File::open's values (both errors
before).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 18:35:36 -05:00
co-authored by Claude Opus 5.5
parent 30a1ed6b9c
commit c5b2afbc35
5 changed files with 75 additions and 29 deletions
+5 -5
View File
@@ -62,11 +62,11 @@ checks both).
## Object stores
`object_store` is async; `Storage` is synchronous (parsing is CPU work).
`ObjectStoreStorage` owns a small tokio runtime (two worker threads) and
blocks the calling thread on it for each read, so it is read from ordinary
threads, several at once. From inside an async runtime it refuses
(`RemoteError::Usage`) rather than block a worker: read in
`tokio::task::spawn_blocking`. The object is pinned by its ETag
`ObjectStoreStorage` owns a small tokio runtime (two worker threads): each
read runs there while the calling thread waits, so it works from any
thread, several at once — including `tokio::task::spawn_blocking` and code
inside another runtime (where `spawn_blocking` is still the better place,
since a read blocks the thread it is called on). The object is pinned by its ETag
(`If-Match`, and compared on every response), else its version or
modification time, and its size. The ranges of one read are fetched
concurrently (up to 8).