wasm: openUrl reads remote files by HTTP range requests (range-read M4)

openUrl(url, opts) returns a RemoteFile with the methods of H5File
(kind, list, info, attrs, attrErrors, read, readHyperslab), each a
promise, and stats(). It runs every call through the restartable
LazyStorage: a pass that misses reports the byte ranges, js/remote.js
fetches them with fetch() and Range headers (six at a time), and the
pass is re-run. This keeps the main thread free without a Worker or
synchronous XHR (h5wasm's lazy files need both), as the design doc
recommends; the cost is re-running a pass per wave of misses.

Every answer is checked: a 206 with exactly the bytes asked for, and
the same ETag/Last-Modified and length as at open, else an error (never
data). A server that ignores Range (200) is downloaded whole, up to
maxDownload (512 MiB), unless fallback: "error". Options: blockSize,
cacheSize, headers, credentials, parallel, fetch.

test/serve.py is a range-capable static server with request counting
(and /norange/ for a server without range support). test.mjs repeats
every fixture check on files opened by URL (1 MiB and 512 B blocks),
checks the request budget on a 200 MB h5py file (list, three small
reads and a window of the big dataset: 5 requests, 6 MiB), the
download fallback, and HTTP errors, changed files and wrong answers;
with CLAWHDF5_WASM_CORPUS every corpus file is compared with open(bytes).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

This commit is contained in:

osobh

2026-09-27 06:44:45 -05:00

co-authored by

Claude Opus 5.5

parent 076feb089a

commit 5107583b97

8 changed files with 1209 additions and 101 deletions

									
										crates/clawhdf5-wasm/Cargo.toml
									
		+2
		
												View File
												
				@@ -24,6 +24,8 @@ clawhdf5-format = { path = "../clawhdf5-format", version = "2.7.0" }

				# Must match the wasm-bindgen CLI exactly; build.sh checks.

				wasm-bindgen = "0.2.129"

				js-sys = "0.3.106"

				# Promises for openUrl and RemoteFile (pure Rust over js-sys).

				wasm-bindgen-futures = "0.4.79"

				[dev-dependencies]

				serde_json = "1"