# clawhdf5-bench

The measurement harnesses behind [`BENCHMARKS.md`](../../BENCHMARKS.md):
HDF5 read and write speed (against libhdf5 and h5py where noted) and the
agent store's search, footprint and retrieval quality. Not meant for
publishing; nothing else in the workspace depends on it. Run everything with
`--release`, and quote numbers with the machine, date and command, as
`BENCHMARKS.md` does.

## Binaries

| Binary | Measures |
|---|---|
| `read_harness` | full reads vs hyperslab selections of a chunked 2-D dataset (compressed and not) and a contiguous one: does a selection cost scale with the selection or the dataset? (`-- --large` for 512 MB) |
| `concurrent_read` | decoded read throughput vs threads on one open `File`; `scripts/concurrent_read_h5py.py` runs the same workload with h5py (threads and processes) and `scripts/compare_concurrent_read.py` tabulates both |
| `search_harness` | HNSW recall@10 vs exact search, QPS and latency per `ef`, and end-to-end `HDF5Memory` ingest/checkpoint/open/search at 1K–100K (`--full`); studies: `--float16-study`, `--options-study`, `--signing-study`, `--ann-only --uniform` |
| `longmemeval_bench` | LongMemEval retrieval recall (turn and session Hit@k, MRR) — **retrieval, not QA accuracy**. Oracle or full `longmemeval_s` haystack; `--features embeddings` (or `embeddings-cuda`) embeds with MiniLM, otherwise the vector stage is inert and the run is BM25-only |
| `memory_arena` | a deterministic multi-session retrieval benchmark (BM25-only) |
| `footprint_bench` | file size and bytes per record at 100–100K records, float16 or `--f32`, WAL on/off, compressed or not |
| `consolidation_efficiency` | retrieval before and after consolidation on signal + noise records |
| `ephemeral_perf` | the in-memory ephemeral tier's set/get latency |
| `mpi_io_bench` | `clawhdf5-io`'s `MpiVol` (root-read + broadcast, not collective I/O); needs `--features mpi-io` and `mpirun` |

```bash
cargo run --release -p clawhdf5-bench --bin search_harness -- --full
cargo run --release -p clawhdf5-bench --bin read_harness
```

## Criterion benches and example

- `cargo bench -p clawhdf5-bench` runs `h5bench_write`, `h5bench_read` and
  `h5bench_meta` (h5bench-style sequential, chunked, strided and metadata
  workloads). `--features libhdf5-compare` adds the same workloads through
  libhdf5 (the `hdf5-metno` crate; needs a system libhdf5 1.14).
- `examples/worldmodel_sampling.rs`: shuffled per-frame reads of a
  `(N, H, W, C)` `uint8` dataset, clawhdf5 against h5py on the same file.

## Features

| Feature | What | Builds C |
|---|---|---|
| `libhdf5-compare` | libhdf5 variants of the Criterion benches | links the system libhdf5 |
| `mpi-io` | `mpi_io_bench` | yes (`mpi-sys`; needs an MPI installation) |
| `embeddings` | MiniLM embeddings for `longmemeval_bench` (candle) | yes (a `cc` build dependency in the candle/tokenizers tree) |
| `embeddings-cuda` | the same on a CUDA GPU (minutes instead of hours on the full haystack) | yes (CUDA) |

## License

MIT