diff --git a/CHANGELOG.md b/CHANGELOG.md index 1fc429d..7c44dbf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -280,6 +280,32 @@ infinity; batches are all or nothing. CLI: `create --float16`. See `BENCHMARKS.md`, "float16 embedding storage". +### Browser (WebAssembly) +- **New crate `clawhdf5-wasm`:** the reader compiled to + `wasm32-unknown-unknown` with a wasm-bindgen JavaScript API — + `open(bytes)`, `list`, `info`, `attrs`, `read`, `readHyperslab` — returning + typed arrays of the stored width (`BigInt64Array` for 64-bit integers), + string arrays for strings and enums, and a thrown `Error` for types with no + typed-array form (compound, reference, opaque, VL sequences) or filters the + build lacks (Zstd, SZIP). Read-only; the file is held in memory. +- **`examples/wasm-viewer/`:** a drop-a-file HDF5/NetCDF-4 viewer page (tree, + type/shape/attributes, values paged as hyperslabs; `?file=&path=` opens a + URL). `build.sh` produces the package; `test/run.sh` checks it under Node + (251 checks against values h5py/libhdf5 read back from an h5py- and a + netCDF4-written file) and renders the page in headless Chromium. Size, + measured 2026-09-26 on tank (`gzip -9 -n`): 627,501 B of wasm, 191,639 B + gzipped, plus 21,826 B (4,487 B) of JS glue; h5wasm 0.10.3's embedded wasm + is 3,544,184 B (907,096 B) — full libhdf5, so not equal functionality. See + `examples/wasm-viewer/README.md`. +- The facade's read path already built for `wasm32-unknown-unknown` (nothing + needed gating); `ci-test.sh` now builds it (`--no-default-features`) and + lints `clawhdf5-wasm` for that target, and CI installs the target. The Node + and browser tests run in `ci-test.sh` only where `node` and `wasm-bindgen` + exist (not the CI container); CI checks the same expectations natively + (`clawhdf5-wasm`'s `h5py_interop` test). +- `Dataset::raw_datatype()` (facade) returns the full stored datatype, for + decoding `read_selection` bytes with `clawhdf5_format::data_read`. + ### Build - **Pure-Rust default.** `clawhdf5-format`, `clawhdf5-filters` and the `clawhdf5` facade default to the `zlib-rs` deflate backend; `fast-deflate` diff --git a/CLAUDE.md b/CLAUDE.md index 4969c6a..c21af88 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -150,6 +150,14 @@ Cargo workspace with 17 crates under `crates/` (plus `libaec-sys`, an internal F `MemorySource` for this bookkeeping is inferred from the caller-supplied `source_channel` string (a heuristic, not an authenticated trust boundary). - GPU-accelerated vector distance computation (`clawhdf5-gpu`, wgpu); HDF5 I/O itself is CPU-only +- Browser: `clawhdf5-wasm` (wasm-bindgen, read-only, file held in memory; + no Zstd/SZIP since they link C) and the `examples/wasm-viewer/` page. + `examples/wasm-viewer/test/run.sh` builds the package (needs the + `wasm-bindgen` CLI at the crate's exact version) and tests it under Node + and headless Chromium (a Playwright download in `~/.cache/ms-playwright` + on tank); the CI container has neither, so CI runs the native + `clawhdf5-wasm` `h5py_interop` test on the same fixture. Size numbers are + in the example's README. - Python and Node.js bindings for cross-language use - NetCDF-4 compatibility for scientific data interop diff --git a/docs/known-issues.md b/docs/known-issues.md index 4ea7518..ca4282b 100644 --- a/docs/known-issues.md +++ b/docs/known-issues.md @@ -422,6 +422,23 @@ the same agent-store interop test. **Fix:** an empty contiguous dataset gets the undefined address (all `0xff`), which is what libhdf5 itself writes. +## `clawhdf5-wasm` (browser) limits + +**Status:** open (by design for now; added 2026-09-26). + +- The whole file is held in memory: `open()` takes its bytes. There are no + HTTP range reads, so a multi-GB file does not fit a browser tab. +- Compound, reference, opaque, bitfield, time and VL-sequence datasets are + refused with an error naming the type; attributes of those types come back + as `value: null` with their `dtype`. +- No Zstd or SZIP (both link C): such datasets fail with + `unsupported filter: 32015` / `: 4`. pcodec is not enabled either. +- External links and virtual-dataset sources in other files cannot be + followed (no file system). +- Variable-length string datasets are read by decoding `read_selection`'s + bytes with `clawhdf5_format::vl_data` in the wasm crate; `File` itself still + cannot (see the audit gaps above). + ## The Node.js package (`packages/clawhdf5-node`) does not work **Status:** open (found 2026-09-25). Unpublished; not built or tested in CI.