osobhandClaude Opus 5.5
5107583b97
openUrl(url, opts) returns a RemoteFile with the methods of H5File (kind, list, info, attrs, attrErrors, read, readHyperslab), each a promise, and stats(). It runs every call through the restartable LazyStorage: a pass that misses reports the byte ranges, js/remote.js fetches them with fetch() and Range headers (six at a time), and the pass is re-run. This keeps the main thread free without a Worker or synchronous XHR (h5wasm's lazy files need both), as the design doc recommends; the cost is re-running a pass per wave of misses. Every answer is checked: a 206 with exactly the bytes asked for, and the same ETag/Last-Modified and length as at open, else an error (never data). A server that ignores Range (200) is downloaded whole, up to maxDownload (512 MiB), unless fallback: "error". Options: blockSize, cacheSize, headers, credentials, parallel, fetch. test/serve.py is a range-capable static server with request counting (and /norange/ for a server without range support). test.mjs repeats every fixture check on files opened by URL (1 MiB and 512 B blocks), checks the request budget on a 200 MB h5py file (list, three small reads and a window of the big dataset: 5 requests, 6 MiB), the download fallback, and HTTP errors, changed files and wrong answers; with CLAWHDF5_WASM_CORPUS every corpus file is compared with open(bytes). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
HDF5 viewer in the browser
A single page that opens an HDF5 or NetCDF-4 file entirely in the browser
with clawhdf5-wasm (clawhdf5's reader compiled to WebAssembly): drop a
file, browse its groups, and look at a dataset's type, shape, attributes
and values (a 50 x 12 window at a time, read as a hyperslab, with the
leading dimensions of a 3-D+ dataset held at chosen indices). The file never
leaves the page.
Build and open
rustup target add wasm32-unknown-unknown
cargo install wasm-bindgen-cli --version 0.2.129 # must equal the crate version; build.sh checks
bash examples/wasm-viewer/build.sh # writes examples/wasm-viewer/pkg/ (not committed)
python3 -m http.server -d examples/wasm-viewer 8000 # wasm cannot load from file://
Then open http://localhost:8000/. ?file=<url>&path=<object> opens a
file from a URL (same origin, or one serving CORS headers) and selects an
object in it, e.g. ?file=data/run1.h5&path=/results/energy.
JavaScript API
import init, { open } from "./pkg/clawhdf5_wasm.js";
await init();
const f = open(new Uint8Array(await blob.arrayBuffer()));
f.list("/"); // [{ name, kind: "group" | "dataset" }], groups first
f.info("/grid"); // { shape, maxshape, dtype, elementShape }
f.attrs("/grid"); // [{ name, value, dtype }]
f.read("/grid"); // { shape, dtype, data }
f.readHyperslab("/grid", [0, 0], [10, 5], [2, 1]); // start, count, stride?, block?
f.free();
data is the typed array of the stored width (Float64Array,
Float32Array also for f16, Int8Array ... BigInt64Array,
BigUint64Array), or an array of strings for fixed- and variable-length
strings and enumerations (h5py booleans read as "TRUE"/"FALSE"). Array
datatypes are flattened, their dimensions appended to shape. Anything
else throws an Error naming the type.
Limits
- Read-only, and the whole file is held in memory (no range requests).
- Compound, reference, opaque and variable-length-sequence datasets are
refused with an error. Attributes of those types are listed with
value: nulland theirdtype. - No Zstd or SZIP filters (they link C): such a dataset fails with
unsupported filter. Deflate, shuffle, Fletcher-32, LZ4, N-Bit and scale-offset are read (within the limits indocs/known-issues.md). - Virtual datasets whose sources are in other files, and external links, cannot be followed: there is no file system.
Tests
test/run.sh builds the package, writes fixture.h5 (h5py) and
fixture.nc (netCDF4) with test/make_fixture.py, then:
- runs
test/test.mjsunder Node: every dataset (whole and a strided hyperslab), listing and attribute is compared with what libhdf5 reads back, error paths are checked, and so are the page's DOM-free helpers (viewer-lib.js); - runs
test/browser.sh: loads the page in headless Chromium with?file=fixture.h5&path=...for eight objects and checks the rendered tree, types, shapes, attribute and value cells, and the error shown for an unsupported type. Skipped when no Chromium is found (CHROMEnames one; a Playwright download under~/.cache/ms-playwrightis picked up). Drag-and-drop and the file picker are not driven by it; they shareload()with the?file=path.
The same expectations are checked natively, without Node, by
crates/clawhdf5-wasm/tests/h5py_interop.rs, which is what CI runs (the CI
container has no Node or browser).
Size
Measured 2026-09-26 on tank (rustc 1.98.1, wasm-bindgen 0.2.129, gzip 1.14,
gzip -9 -n), after bash examples/wasm-viewer/build.sh:
| raw | gzip -9 | |
|---|---|---|
pkg/clawhdf5_wasm_bg.wasm (profile wasm-release, opt-level s) |
627,501 B | 191,639 B |
pkg/clawhdf5_wasm.js (wasm-bindgen glue) |
21,826 B | 4,487 B |
same wasm at opt-level z |
693,068 B | 192,550 B |
same wasm at opt-level 3 |
544,035 B | 198,803 B |
h5wasm 0.10.3: wasm embedded in dist/esm/hdf5_util.js |
3,544,184 B | 907,096 B |
h5wasm 0.10.3: dist/esm/hdf5_util.js as shipped |
4,150,134 B | 986,699 B |
h5wasm figures: npm pack [email protected] (npm reports
dist.unpackedSize 14,731,385 B for the whole package), wasm extracted from
the binaryDecode literal in hdf5_util.js. h5wasm is the whole of libhdf5
(writing, every datatype, plugins), so this compares download size, not
equal functionality. No wasm-opt pass was applied (binaryen is not
installed on tank). opt-level s is used because it is the smallest
compressed.