quantumclaw/clawhdf5

Fork 0

Files

T

History

osobhandClaude Opus 5.5 e4ba09946f

CI / test-arm64 (pull_request) Successful in 1m38s

Details

CI / test (pull_request) Successful in 21m11s

Details

HDF5 2.0 native complex as a first-class type on read; Python libver=

- Datatype::parse returns Datatype::Complex for class 11 (also inside
  compounds, arrays and VL types) instead of the {r, i} compound view.
- Facade: DType::Complex(Box<DType>); read_complex_f32/f64 accept it.
- h5rs dump/ls/diff print native complex as h5dump/h5ls/h5diff 2.2.0 do
  (checked against a fixture written by h5py 3.16 / libhdf5 2.0.0);
  dump --json keeps the {r, i} compound (hdf5-json has no complex class).
- clawhdf5-wasm reads native complex datasets as [re, im] pairs.
- Python: clawhdf5.File(path, 'w', libver=...) with h5py's values,
  mapped to FileBuilder::libver_bounds; 'v108' output opens in HDF5 1.8.23.
- Docs: known-issues entry moved to Fixed (history), CHANGELOG, READMEs.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

2026-09-29 20:47:33 -05:00

wasm: openUrl takes Headers, aborts sibling requests on failure, checks parallel

2026-09-27 07:53:22 -05:00

src

HDF5 2.0 native complex as a first-class type on read; Python libver=

2026-09-29 20:47:33 -05:00

tests

format: look a name up in a v1 group down its B-tree, as libhdf5 does

2026-09-27 22:47:47 -05:00

Cargo.toml

wasm: openUrl reads remote files by HTTP range requests (range-read M4)

2026-09-27 06:44:45 -05:00

README.md

docs: crate READMEs describe each crate as it is today

2026-09-28 11:13:30 -05:00

README.md

clawhdf5-wasm

clawhdf5's HDF5 and NetCDF-4 reader compiled to WebAssembly with wasm-bindgen, for the browser (and Node). Read-only. Two ways in:

open(bytes) — a file already in memory (a dropped file, a fetched blob);
openUrl(url, opts) — a file on a web server, read by HTTP range requests as each call needs its bytes, without downloading it (range-read milestone M4, docs/design/range-reads.md).

Both give list, info, attrs, read and readHyperslab; the remote file's methods return promises, and stats() counts requests and bytes. The JavaScript API, options, limits, package size and tests are documented with the demo page, examples/wasm-viewer/README.md.

Layout

src/core.rs — the reader over any clawhdf5_format::storage::Storage (Reader::open_storage), plain Rust and tested natively.
src/lazy.rs — the restartable "NeedBytes" cache behind openUrl: a call runs as a pass over the blocks fetched so far; a pass that misses is abandoned, the missing (and hinted) blocks are fetched, and the pass is run again. No block is evicted while a call runs.
js/remote.js — the HTTP side: fetch with Range, checking every answer (a 206 with exactly the bytes asked for, same ETag/Last-Modified and length) so a call fails rather than return another file's bytes.
src/lib.rs — the wasm-bindgen exports.

Build and test

rustup target add wasm32-unknown-unknown
cargo install wasm-bindgen-cli --version 0.2.129   # must equal the crate's wasm-bindgen
bash examples/wasm-viewer/build.sh                  # -> examples/wasm-viewer/pkg/
cargo test -p clawhdf5-wasm                         # native: h5py_interop, lazy, vl_strings
bash examples/wasm-viewer/test/run.sh               # Node + headless Chromium (not in CI)

CLAWHDF5_WASM_CORPUS=conformance/.cache/corpus cargo test -p clawhdf5-wasm --test lazy compares every corpus file read lazily with the same file read from bytes.

Built without mmap and parallel and without the Zstd and SZIP filters (they link C): such datasets fail with unsupported filter. No C is compiled; publish = false (it is distributed as the package build.sh makes).

License

MIT