osobh
b2dce41532
CI / test (push) Failing after 3s
than h5py (5e) stable-worldmodel (arXiv 2605.21800, LeCun/Balestriero) supports HDF5 as one of three native formats and measures generic HDF5 at 1,416-1,474 samples/s for per-frame sample loading. This measures clawhdf5 against that shape, hardware-controlled: clawhdf5 and h5py reading the SAME file on the SAME machine. worldmodel_sampling example: mmap an (N,H,W,C) uint8 observation dataset, read each frame once per pass in shuffled (dataloader) order. The file is written by h5py (benchmarks/gen_worldmodel_frames.py) — clawhdf5 parsing an externally-produced HDF5 file is itself the interop result — and read by both clawhdf5 and the h5py counterpart (benchmarks/bench_worldmodel_h5py.py, opening exactly stable-worldmodel's HDF5Dataset: swmr + 256 MB cache). Results (tank, Ryzen 7 7800X3D, 20000x64x64x3 = 246 MB, in page cache, median of 3): clawhdf5 zero-copy view 593k samples/sec 8.1x clawhdf5 materialised copy 518k samples/sec 7.1x h5py (swmr, 256 MB cache) 73k samples/sec 1.0x The materialised-copy row is the fair equal-work comparison (to_vec per frame, matching h5py's numpy materialisation) and is still 7.1x faster; that the copy costs almost nothing shows the gap is h5py's per-frame call overhead, not data movement. Honest caveats in BENCHMARKS.md: absolute numbers are NOT comparable to the paper's (different hardware, smaller frames, no torch/transform), only the same-machine ratio is; this is an in-page-cache measurement isolating read-path overhead, not disk bandwidth. Adds only an example, two benchmark scripts, and a BENCHMARKS.md section — no library code. (Workspace clippy has pre-existing toolchain drift unrelated to this change; tracked separately.)
28 lines
1.1 KiB
Python
28 lines
1.1 KiB
Python
#!/usr/bin/env python3
|
|
"""Generate a world-model-shaped dataset: N frames of HxWxC uint8 observations,
|
|
contiguous (N,H,W,C), matching stable-worldmodel's per-frame sample-loading
|
|
access pattern. Also emits ep_len/ep_offset like their format."""
|
|
import sys, time, numpy as np, h5py
|
|
|
|
path = sys.argv[1]
|
|
N = int(sys.argv[2]) if len(sys.argv) > 2 else 20000
|
|
H = W = 64
|
|
C = 3
|
|
rng = np.random.default_rng(0)
|
|
t0 = time.perf_counter()
|
|
with h5py.File(path, "w", libver="latest") as f:
|
|
# Contiguous (N,H,W,C) uint8 — the fair, both-APIs-support-it layout.
|
|
obs = f.create_dataset("observation", shape=(N, H, W, C), dtype=np.uint8)
|
|
# Write in blocks to bound memory.
|
|
B = 2000
|
|
for i in range(0, N, B):
|
|
n = min(B, N - i)
|
|
obs[i:i+n] = rng.integers(0, 256, size=(n, H, W, C), dtype=np.uint8)
|
|
# Episode metadata like their format: 100-step episodes.
|
|
ep = 100
|
|
n_ep = N // ep
|
|
f.create_dataset("ep_len", data=np.full(n_ep, ep, dtype=np.int32))
|
|
f.create_dataset("ep_offset", data=(np.arange(n_ep) * ep).astype(np.int64))
|
|
print(f"wrote {N} frames {H}x{W}x{C} to {path} in {time.perf_counter()-t0:.1f}s "
|
|
f"({N*H*W*C/1e6:.0f} MB)")
|