docs: clawhdf5-wasm in the changelog, known issues and CLAUDE.md

Changelog entry with the sizes measured on tank on 2026-09-26 (and the
h5wasm 0.10.3 comparison), the browser build's limits as a known-issues
entry, and where its tests run.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-26 00:06:58 -05:00
co-authored by Claude Opus 5.5
parent b58d61cfb7
commit 34987ec194
3 changed files with 51 additions and 0 deletions
+26
View File
@@ -280,6 +280,32 @@
infinity; batches are all or nothing. CLI: `create --float16`. See infinity; batches are all or nothing. CLI: `create --float16`. See
`BENCHMARKS.md`, "float16 embedding storage". `BENCHMARKS.md`, "float16 embedding storage".
### Browser (WebAssembly)
- **New crate `clawhdf5-wasm`:** the reader compiled to
`wasm32-unknown-unknown` with a wasm-bindgen JavaScript API —
`open(bytes)`, `list`, `info`, `attrs`, `read`, `readHyperslab` — returning
typed arrays of the stored width (`BigInt64Array` for 64-bit integers),
string arrays for strings and enums, and a thrown `Error` for types with no
typed-array form (compound, reference, opaque, VL sequences) or filters the
build lacks (Zstd, SZIP). Read-only; the file is held in memory.
- **`examples/wasm-viewer/`:** a drop-a-file HDF5/NetCDF-4 viewer page (tree,
type/shape/attributes, values paged as hyperslabs; `?file=&path=` opens a
URL). `build.sh` produces the package; `test/run.sh` checks it under Node
(251 checks against values h5py/libhdf5 read back from an h5py- and a
netCDF4-written file) and renders the page in headless Chromium. Size,
measured 2026-09-26 on tank (`gzip -9 -n`): 627,501 B of wasm, 191,639 B
gzipped, plus 21,826 B (4,487 B) of JS glue; h5wasm 0.10.3's embedded wasm
is 3,544,184 B (907,096 B) — full libhdf5, so not equal functionality. See
`examples/wasm-viewer/README.md`.
- The facade's read path already built for `wasm32-unknown-unknown` (nothing
needed gating); `ci-test.sh` now builds it (`--no-default-features`) and
lints `clawhdf5-wasm` for that target, and CI installs the target. The Node
and browser tests run in `ci-test.sh` only where `node` and `wasm-bindgen`
exist (not the CI container); CI checks the same expectations natively
(`clawhdf5-wasm`'s `h5py_interop` test).
- `Dataset::raw_datatype()` (facade) returns the full stored datatype, for
decoding `read_selection` bytes with `clawhdf5_format::data_read`.
### Build ### Build
- **Pure-Rust default.** `clawhdf5-format`, `clawhdf5-filters` and the - **Pure-Rust default.** `clawhdf5-format`, `clawhdf5-filters` and the
`clawhdf5` facade default to the `zlib-rs` deflate backend; `fast-deflate` `clawhdf5` facade default to the `zlib-rs` deflate backend; `fast-deflate`
+8
View File
@@ -150,6 +150,14 @@ Cargo workspace with 17 crates under `crates/` (plus `libaec-sys`, an internal F
`MemorySource` for this bookkeeping is inferred from the caller-supplied `MemorySource` for this bookkeeping is inferred from the caller-supplied
`source_channel` string (a heuristic, not an authenticated trust boundary). `source_channel` string (a heuristic, not an authenticated trust boundary).
- GPU-accelerated vector distance computation (`clawhdf5-gpu`, wgpu); HDF5 I/O itself is CPU-only - GPU-accelerated vector distance computation (`clawhdf5-gpu`, wgpu); HDF5 I/O itself is CPU-only
- Browser: `clawhdf5-wasm` (wasm-bindgen, read-only, file held in memory;
no Zstd/SZIP since they link C) and the `examples/wasm-viewer/` page.
`examples/wasm-viewer/test/run.sh` builds the package (needs the
`wasm-bindgen` CLI at the crate's exact version) and tests it under Node
and headless Chromium (a Playwright download in `~/.cache/ms-playwright`
on tank); the CI container has neither, so CI runs the native
`clawhdf5-wasm` `h5py_interop` test on the same fixture. Size numbers are
in the example's README.
- Python and Node.js bindings for cross-language use - Python and Node.js bindings for cross-language use
- NetCDF-4 compatibility for scientific data interop - NetCDF-4 compatibility for scientific data interop
+17
View File
@@ -422,6 +422,23 @@ the same agent-store interop test.
**Fix:** an empty contiguous dataset gets the undefined address (all `0xff`), **Fix:** an empty contiguous dataset gets the undefined address (all `0xff`),
which is what libhdf5 itself writes. which is what libhdf5 itself writes.
## `clawhdf5-wasm` (browser) limits
**Status:** open (by design for now; added 2026-09-26).
- The whole file is held in memory: `open()` takes its bytes. There are no
HTTP range reads, so a multi-GB file does not fit a browser tab.
- Compound, reference, opaque, bitfield, time and VL-sequence datasets are
refused with an error naming the type; attributes of those types come back
as `value: null` with their `dtype`.
- No Zstd or SZIP (both link C): such datasets fail with
`unsupported filter: 32015` / `: 4`. pcodec is not enabled either.
- External links and virtual-dataset sources in other files cannot be
followed (no file system).
- Variable-length string datasets are read by decoding `read_selection`'s
bytes with `clawhdf5_format::vl_data` in the wasm crate; `File` itself still
cannot (see the audit gaps above).
## The Node.js package (`packages/clawhdf5-node`) does not work ## The Node.js package (`packages/clawhdf5-node`) does not work
**Status:** open (found 2026-09-25). Unpublished; not built or tested in CI. **Status:** open (found 2026-09-25). Unpublished; not built or tested in CI.