osobhandClaude Opus 5.5
5107583b97
openUrl(url, opts) returns a RemoteFile with the methods of H5File (kind, list, info, attrs, attrErrors, read, readHyperslab), each a promise, and stats(). It runs every call through the restartable LazyStorage: a pass that misses reports the byte ranges, js/remote.js fetches them with fetch() and Range headers (six at a time), and the pass is re-run. This keeps the main thread free without a Worker or synchronous XHR (h5wasm's lazy files need both), as the design doc recommends; the cost is re-running a pass per wave of misses. Every answer is checked: a 206 with exactly the bytes asked for, and the same ETag/Last-Modified and length as at open, else an error (never data). A server that ignores Range (200) is downloaded whole, up to maxDownload (512 MiB), unless fallback: "error". Options: blockSize, cacheSize, headers, credentials, parallel, fetch. test/serve.py is a range-capable static server with request counting (and /norange/ for a server without range support). test.mjs repeats every fixture check on files opened by URL (1 MiB and 512 B blocks), checks the request budget on a 200 MB h5py file (list, three small reads and a window of the big dataset: 5 requests, 6 MiB), the download fallback, and HTTP errors, changed files and wrong answers; with CLAWHDF5_WASM_CORPUS every corpus file is compared with open(bytes). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
59 lines
2.1 KiB
Bash
Executable File
59 lines
2.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Build the wasm package (../build.sh), test it under Node against files
|
|
# written by h5py and netCDF4 (make_fixture.py) — opened from bytes, and
|
|
# opened by URL from a local range-capable HTTP server (serve.py) — then
|
|
# load the viewer page in headless Chromium if one is found (browser.sh).
|
|
#
|
|
# Needs node, the wasm-bindgen CLI (see ../build.sh) and a Python with h5py,
|
|
# netCDF4 and numpy: CLAWHDF5_PYTHON names it (default python3). Without that
|
|
# Python the test is skipped, unless CLAWHDF5_REQUIRE_INTEROP=1.
|
|
#
|
|
# WASM_BIG_MB (default 200) sizes the large file of the range-request
|
|
# budget test (0 leaves it out); it is written under TMPDIR.
|
|
# CLAWHDF5_WASM_CORPUS=DIR also compares every HDF5 file under DIR (up to
|
|
# 16 MiB) read over HTTP with the same file read from bytes.
|
|
set -euo pipefail
|
|
|
|
HERE="$(cd "$(dirname "$0")" && pwd)"
|
|
PY="${CLAWHDF5_PYTHON:-python3}"
|
|
|
|
command -v node >/dev/null || { echo "node not found" >&2; exit 1; }
|
|
if ! "$PY" -c "import h5py, netCDF4, numpy" >/dev/null 2>&1; then
|
|
if [ "${CLAWHDF5_REQUIRE_INTEROP:-0}" = "1" ]; then
|
|
echo "CLAWHDF5_REQUIRE_INTEROP=1 but $PY lacks h5py/netCDF4/numpy" >&2
|
|
exit 1
|
|
fi
|
|
echo "SKIP: $PY lacks h5py/netCDF4/numpy"
|
|
exit 0
|
|
fi
|
|
|
|
bash "$HERE/../build.sh"
|
|
fix="$(mktemp -d)"
|
|
server=""
|
|
cleanup() {
|
|
[ -n "$server" ] && kill "$server" 2>/dev/null || true
|
|
rm -rf "$fix"
|
|
}
|
|
trap cleanup EXIT
|
|
WASM_BIG_MB="${WASM_BIG_MB:-200}" "$PY" "$HERE/make_fixture.py" "$fix"
|
|
|
|
# The fixtures (and the corpus) over HTTP with range support.
|
|
roots=(--root "fix=$fix")
|
|
[ -n "${CLAWHDF5_WASM_CORPUS:-}" ] && roots+=(--root "corpus=$CLAWHDF5_WASM_CORPUS")
|
|
"$PY" "$HERE/serve.py" "${roots[@]}" > "$fix/port" &
|
|
server=$!
|
|
for _ in $(seq 50); do
|
|
[ -s "$fix/port" ] && break
|
|
sleep 0.1
|
|
done
|
|
node "$HERE/test.mjs" "$HERE/../pkg" "$fix" "http://127.0.0.1:$(head -1 "$fix/port")"
|
|
|
|
# The page itself, in headless Chromium when one is available.
|
|
status=0
|
|
bash "$HERE/browser.sh" "$fix" || status=$?
|
|
if [ "$status" = 3 ]; then
|
|
echo "SKIP: viewer page in a browser (no Chromium; set CHROME)"
|
|
elif [ "$status" != 0 ]; then
|
|
exit "$status"
|
|
fi
|