Files
clawhdf5/BENCHMARKS.md
T
Omar SobhandClaude Sonnet 5 88195d1c33 docs: fix untraceable benchmark claims, add dual-audience framing, validate on second machine
- README's "HDF5 Core I/O" table claimed 19ns/2,080µs labeled 308× (real ratio
  ~109,000×) and a 313ns zero-copy mmap figure — neither traced to any dated
  benchmark in BENCHMARKS.md. Replaced the table wholesale with the existing
  "vs libhdf5 Summary" figures, relabeled from "h5py/C HDF5" to "libhdf5"
  (BENCHMARKS.md never benchmarks against h5py, only libhdf5 directly).
- Added two new Criterion benchmarks to close the coverage gaps that produced
  the untraceable numbers: metadata_open_from_disk (I/O-inclusive, fair
  clawhdf5-vs-libhdf5 file-open comparison) and metadata_parse_in_memory
  (clawhdf5-only, explicitly labeled as excluding I/O) in h5bench_meta.rs;
  read_zerocopy_mmap in h5bench_read.rs (forces real page-ins by summing
  elements rather than just returning a slice length — the mmap path turns
  out to be slower than a plain copy at these sizes, an honest, unflattering
  but real result now documented instead of a fabricated 313ns).
- Re-ran the full existing benchmark suite plus the two new ones on a second,
  independently administered machine (tank: Ryzen 7 7800X3D) to validate the
  numbers before publishing them. 5 of 6 rows landed within ~15% of the
  original i7-12650H figures; recorded both in BENCHMARKS.md's new
  "Independent Validation" section. README now cites the tank numbers.
- Added a short top-of-file README callout naming both halves of the project
  (general-purpose HDF5 library vs. agent memory layer) with links to
  BENCHMARKS.md and the Crate Map, so a data-infra reader isn't 60% through
  a memory-store pitch before finding the part relevant to them.
- Added one factual, no-names line noting benchmark numbers are being
  validated in collaboration with HDF5 Group engineers.
- Fixed the same untraceable "2-300x faster than h5py/C HDF5" / "313 ns"
  claims in docs/QUICKSTART.md, one click from the README's own "New here?"
  link.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-03 17:46:55 -07:00

27 KiB
Raw Blame History

ClawhDF5 Benchmark Results

Pure Rust. Zero C dependencies. Single file. Fast enough to forget it's there.

System: Intel i7-12650H (10C/16T, 4.7 GHz boost) · 32 GB DDR5 · Linux 6.8.0
Rust: 1.96.0-nightly (2026-03-14) · --release profile
Date: 2026-07-01


Vector Search Latency

Brute-force cosine similarity over 384-dimensional embeddings (OpenAI text-embedding-3-small size).

Scale Flat Search Pre-norm IVF (nprobe=10) IVF-PQ RAIRS
1K 54 µs 62 µs
10K 753 µs 706 µs 27 µs 159 µs
100K 11.4 ms 1.32 ms 1.19 ms

Key insight: At 10K records (typical agent memory), IVF search delivers 27 µs — that's 26x faster than flat search. Even at 100K records, IVF-PQ keeps search under 1.2 ms.

Comparison to MemX (arxiv:2603.16171)

MemX claims end-to-end search under 90ms at 100K records (Rust + libSQL + FTS5).

Metric MemX (claimed) ClawhDF5 Speedup
100K flat search <90 ms 11.4 ms ~8x
100K IVF-PQ search 1.19 ms ~76x
Keyword search 10K 1,100x improvement over unindexed 583 µs (BM25) Comparable

SIMD & Parallelism

384-dimensional cosine similarity at 10K scale.

Strategy Latency vs Sequential
Sequential (scalar) 1.07 ms 1.0x
SIMD (auto-vectorized) 545 µs 2.0x
Rayon (parallel) 553 µs 1.9x
Adaptive (auto-select) 564 µs 1.9x

At 100K:

Strategy Latency
SIMD 13.7 ms
Rayon parallel 8.3 ms

Hybrid Search (Vector + BM25)

1K records, 384-dimensional embeddings with BM25 keyword index.

Method Latency Notes
Weighted fusion 198 µs Original min-max normalization
RRF (k=60) 222 µs Reciprocal Rank Fusion — better quality, ~12% overhead
BM25-only 1K 67 µs Keyword search alone
Hybrid 10K 2.04 ms Full hybrid at 10K scale

Knowledge Graph

Graph traversal and entity operations.

Operation Scale Latency
BFS traversal 100 entities 5.4 µs
BFS traversal 1,000 entities 24 µs
Spreading activation 100 entities 16.9 µs
Entity resolution (Levenshtein) 100 entities 64 µs
Alias resolution (short query) 100 aliases 10.4 µs
Alias resolution (long query) 100 aliases 11.6 µs

All graph operations complete in microseconds. Spreading activation across 100 entities with 5 propagation steps finishes in 17 µs.


Memory Consolidation

Hippocampal-inspired tiered memory management.

Operation Scale Latency
Consolidation cycle 100 records 15 µs
Consolidation cycle 1,000 records 164 µs
Importance scoring 100 records 25 µs

A full consolidation pass over 1,000 memories (eviction + promotion across Working → Episodic → Semantic) completes in 164 µs. This can run on every memory write without perceptible latency.


Temporal Index

Sorted timestamp index with binary search.

Operation Scale Latency
Range query 10K timestamps 716 ns
Batch insert 10K timestamps 4.69 ms

Sub-microsecond temporal queries. "What happened between 3pm and 5pm?" over 10K records: 716 nanoseconds.


Write Path

HDF5 persistence with optional Write-Ahead Log.

Operation Latency Notes
Single save (no WAL) 61 µs Direct HDF5 write (owned-Vec IO path)
Single save (with WAL) 18 µs WAL group-commit append; HDF5 write batched at flush
Batch 100 723 µs 7.2 µs per record
Batch 1,000 6.17 ms 6.2 µs per record
WAL save (1K existing) 539 µs Incremental append
WAL flush 100 entries 787 µs Merge WAL → HDF5
Session tick 1K 5.76 ms Full session maintenance
Session tick 10K 89.8 ms Background operation

Decision Gate

Trivial/non-trivial classification for memory write filtering.

Check Latency
Trivial skip ("ok", "yes") 61 ns
Short phrase skip 86 ns
Non-trivial pass 705 ns
Ratio check 488 ns

Sub-microsecond filtering. The gate decides whether to save a memory in under 1 µs.


Memory Strategy

End-to-end strategy evaluation including embedding operations.

Strategy Condition Latency
SaveEveryExchange (substantive) Saves 923 ns
SaveEveryExchange (trivial) Skips 67 ns
SaveOnSemanticShift (empty store) Saves 941 ns

Summary

Capability Typical Latency Scale
Full memory search <1 ms 10K records
Hybrid vector+keyword <200 µs 1K records
Knowledge graph query <25 µs 1K entities
Temporal range query <1 µs 10K timestamps
Memory write <20 µs Per record (WAL group-commit append)
Consolidation cycle <165 µs 1K records
Importance gate <1 µs Per record

The entire memory pipeline — search, retrieve, re-rank, filter — runs in single-digit milliseconds at agent-typical scales. Fast enough that memory becomes invisible infrastructure.


Latency benchmarks generated with Criterion.rs (50-100 samples per benchmark). Results may vary by hardware.


LongMemEval Results

Dataset: LongMemEval oracle (500 questions, 6 question types, variable-length chat histories) Mode: BM25-only retrieval — zero embeddings, vector_weight=0.0, keyword_weight=1.0 Reference: MemX (arxiv:2603.16171) with full embedding system: Hit@5=51.6%, MRR=0.380

Run: cargo run --release --bin longmemeval_bench

Session-Level Recall (n=500)

Metric ClawhDF5 (BM25-only)
Hit@1 100.0%
Hit@5 100.0%
Hit@10 100.0%
MRR 1.0000

Perfect session-level recall across all 500 questions and all 6 question types.

Turn-Level Recall (n=500)

Metric ClawhDF5 (BM25-only) MemX (full system)¹
Hit@1 52.6%
Hit@5 84.4% 51.6%
Hit@10 90.4%
MRR 0.6597 0.380

clawhdf5 outperforms MemX at turn-level retrieval — Hit@5 84.4% vs 51.6%, MRR 0.66 vs 0.38 — with BM25 alone, no embeddings needed.

¹ MemX uses dense embeddings + FTS5 + four-factor re-ranking. Our BM25-only result exceeds their full pipeline.

Per-Type Breakdown (session-level)

Question Type N Hit@1 Hit@5 Hit@10 MRR
single-session-user 70 100.0% 100.0% 100.0% 1.0000
single-session-assistant 56 100.0% 100.0% 100.0% 1.0000
single-session-preference 30 100.0% 100.0% 100.0% 1.0000
temporal-reasoning 133 100.0% 100.0% 100.0% 1.0000
multi-session 133 100.0% 100.0% 100.0% 1.0000
knowledge-update 78 100.0% 100.0% 100.0% 1.0000

Search Latency (LongMemEval, n=500 queries)

Metric Latency
avg 1,004 µs
p50 1,017 µs
p95 2,031 µs
p99 2,912 µs

Sub-millisecond median search across variable-length chat histories.


Multi-Session Benchmark (MemoryArena)

Dataset: Deterministic synthetic conversations — 50 sessions × ~20 turns = 999 turns Topics: Personal info, food preferences, music, travel, work/schedule, hobbies Queries: 35 questions across 4 types

Run: cargo run --release --bin memory_arena

Results by Query Type

Query Type N Hit@1 Hit@5 Hit@10 MRR Avg Latency
single-session 25 40.0% 92.0% 100.0% 0.5788 7,853 µs
multi-session 5 40.0% 60.0% 80.0% 0.5333 7,887 µs
temporal 3 33.3% 66.7% 66.7% 0.5000 7,899 µs
knowledge-update 2 0.0% 50.0% 50.0% 0.2500 7,899 µs
OVERALL 35 37.1% 82.9% 91.4% 0.5468 7,870 µs

Key findings:

  • Hit@10 of 91.4% across all query types with BM25-only (no embeddings)
  • Single-session recall strongest at 100% Hit@10
  • Knowledge-update hardest (requires temporal disambiguation) — would improve significantly with vector similarity
  • Latency dominated by BM25 index build over 999 turns (~7.9 ms)

Memory Footprint

HDF5 file size at various record counts — 384-dimensional embeddings, 200-char text.

Run: cargo run --release --bin footprint_bench

Uncompressed (no WAL)

Records File Size Raw Data Bytes/Record Throughput
100 176.4 KB 169.5 KB 1.8 KB 100,000 rec/s
1K 1.7 MB 1.7 MB 1.8 KB 109,643 rec/s
10K 17.0 MB 16.6 MB 1.7 KB 118,100 rec/s
50K 85.0 MB 82.8 MB 1.7 KB 110,723 rec/s
100K 169.8 MB 165.6 MB 1.7 KB 111,422 rec/s

1.7 KB per record — HDF5 overhead is near-zero. Ingestion throughput exceeds 100K records/sec.

With Gzip Compression (level 6)

Records Compressed Ratio Bytes/Record
100 31.5 KB 5.37x 323 B
1K 277.1 KB 6.12x 283 B
10K 2.7 MB 6.17x 281 B
50K 13.4 MB 6.17x 281 B
100K 26.9 MB 6.15x 282 B

6.2x compression ratio — 100K agent memories in 27 MB compressed.

Text Length Comparison (10K records, no compression)

Text Length File Size Bytes/Record Throughput
short (50 chars) 15.6 MB 1.6 KB 177,925 rec/s
medium (200 chars) 17.0 MB 1.7 KB 172,152 rec/s
long (1000 chars) 24.6 MB 2.5 KB 157,807 rec/s

WAL Overhead (1K records)

Mode File Size Ingest Time Overhead
No WAL 1.7 MB 5.7 ms
With WAL 1.7 MB + 9 B WAL 5.3 ms ±8% (negligible)

Consolidation Efficiency

Hippocampal-inspired memory consolidation improves both retrieval quality and search speed.

Run: cargo run --release --bin consolidation_efficiency

Retrieval Quality Before vs. After Consolidation

Setup: 1,000 records (10 signal + 990 noise), working_capacity=100

Metric Before After Delta
Records in store 1,000 100 90%
Hit@1 100.0% 100.0%
Hit@5 100.0% 100.0%
Hit@10 100.0% 100.0%
MRR 1.0000 1.0000
Search latency 2,752 µs 312 µs 8.8x faster

Signal records survive consolidation because they are accessed 15+ times, giving them high decay scores. 900 noise records evicted, search speeds up 8.8x, and zero quality loss — perfect recall maintained.

Consolidation Cycle Time

Records Cycle Time Evictions Promotions
100 21 µs 100 0
1K 345 µs 1,000 0
10K 17.3 ms 10,000 0

Ephemeral Tier (Redis Comparison)

In-memory key-value store with TTL, capacity eviction, and embedding search. No network hop, no serialization — direct HashMap operations.

Run: cargo run --release --bin ephemeral_perf

Latency Comparison

Operation clawhdf5 Ephemeral Redis (single-node)¹ Speedup
SET 356 ns/op ~25,000 ns/op 70x
GET (hit) 179 ns/op ~25,000 ns/op 140x
GET (miss) 62 ns/op ~25,000 ns/op 403x
DELETE 124 ns/op ~25,000 ns/op 202x
SET+embedding 268 ns/op N/A

¹ Redis latency includes network round-trip (loopback). clawhdf5 ephemeral is in-process — no network.

Throughput

Operation ops/sec
SET 2,810,649
GET 5,584,684
DELETE 8,093,731
SET+EMB (384d) 3,725,877

Embedding Search (ephemeral tier)

Scale Latency
10K entries @ 384d 2.9 ms/query

Cross-Platform Notes

Run: ./benchmarks/cross_platform.sh [--full] [--output results.json]

Measured Platforms

Platform CPU 10K IVF Search Notes
Linux x86_64 Intel i7-12650H (10C, 4.7 GHz) 27 µs Primary CI target
macOS aarch64 Apple M3 Max (14C) ~18 µs ~33% faster via NEON SIMD

Reproducibility

rustup override set nightly

# Latency benchmarks (Criterion)
cargo bench -p clawhdf5-agent

# Full benchmark suite
cargo run --release --bin longmemeval_bench
cargo run --release --bin memory_arena
cargo run --release --bin footprint_bench
cargo run --release --bin consolidation_efficiency
cargo run --release --bin ephemeral_perf

h5bench-Equivalent I/O Benchmarks

Criterion harness mirroring h5bench serial workloads. clawhdf5 benchmarks dated 2026-07-01; libhdf5 1.14.6 head-to-head comparison dated 2026-06-30 (same hardware, same Criterion harness).

cargo bench -p clawhdf5-bench                         # clawhdf5-only
cargo bench -p clawhdf5-bench --features libhdf5-compare  # head-to-head

Sequential Read Throughput

Both read a 1-D contiguous f32 dataset. clawhdf5 parses from Vec<u8> (zero-copy); libhdf5 reads from a temp file including open + read + close overhead.

Workload n=1K n=10K n=100K
clawhdf5 f32 634 ns / 5.9 GiB/s 2.44 µs / 15.3 GiB/s 24.5 µs / 15.2 GiB/s
libhdf5 f32 45.2 µs / 85 MiB/s 47.8 µs / 799 MiB/s 73.9 µs / 5.0 GiB/s
Speedup 71× 20× 3.0×
clawhdf5 f64 743 ns / 10.0 GiB/s 4.17 µs / 17.8 GiB/s 43.3 µs / 17.2 GiB/s
clawhdf5 from_disk (f64, OS I/O) 10.1 µs / 7.4 GiB/s 77.6 µs / 9.6 GiB/s
clawhdf5 hyperslab (f64, 10% slice) 4.09 µs / 1.8 GiB/s 50.1 µs / 1.5 GiB/s

libhdf5 f64 comparison excluded — clawhdf5's datatype encoding differs from libhdf5's (known gap), making cross-format reads unreliable for comparison.

Chunked Read Throughput

Matrix size Latency Throughput
64×64 f32 6.39 µs 2.4 GiB/s
256×256 f32 41.7 µs 5.9 GiB/s
512×512 f32 176 µs 5.5 GiB/s

Sequential Write Throughput

Both write to disk. At 100K elements both converge on the OS write() syscall ceiling.

Workload n=1K n=10K n=100K
clawhdf5 f32 9.44 µs / 404 MiB/s 25 µs / 1.49 GiB/s 228 µs / 1.63 GiB/s
libhdf5 f32 77.9 µs / 49 MiB/s 87.8 µs / 435 MiB/s 214 µs / 1.74 GiB/s
Speedup 8.2× 3.5× ≈ tie
clawhdf5 f64 embeddings 6.50 µs (n=128) 8.67 µs (n=512) / 450 MiB/s 10.27 µs (n=1K) / 761 MiB/s

Chunked Write: Codec Comparison (with auto-shuffle)

Auto-shuffle is applied before all compression codecs by default — AoS→SoA byte transpose, implements byte-grouping pre-filter per arXiv:2506.18062. Shuffle dramatically improves throughput for float/int data by creating long runs of similar bytes.

Matrix size Zstd-3 + shuffle Deflate-6 + shuffle Speedup
32×32 f32 48 µs / 81 MiB/s 39 µs / 100 MiB/s Deflate 1.23× faster (small chunk)
128×128 f32 148 µs / 422 MiB/s 153 µs / 407 MiB/s Parity
512×512 f32 1.34 ms / 748 MiB/s 1.39 ms / 719 MiB/s Zstd 1.04× faster

Impact of auto-shuffle vs no-shuffle baseline:

Matrix size Zstd-3 speedup Deflate-6 speedup
32×32 +19% +38%
128×128 +25% +204%
512×512 +25% +157%

Both codecs perform at parity at large sizes (~720750 MiB/s). Use .with_zstd(3) or .with_deflate(6) for write-heavy workloads. Use .without_shuffle() only for byte arrays or data that doesn't benefit from AoS→SoA transposition.

Chunked Write vs libhdf5 (deflate-6)

clawhdf5 compresses all chunks in memory and issues a single write(). libhdf5 flushes each chunk individually via its Virtual File Layer (one pwrite() per chunk).

Matrix clawhdf5 deflate-6 + shuffle libhdf5 deflate-6 Speedup
32×32 f32 39 µs / 100 MiB/s 172 µs / 23 MiB/s 4.4×
128×128 f32 153 µs / 407 MiB/s 3,150 µs / 20 MiB/s 20.6×
512×512 f32 1,390 µs / 719 MiB/s 53,300 µs / 19 MiB/s 38.4×

The 32×32 speedup (4.4×) is lower than the 512×512 speedup (38.4×) because shuffle adds overhead that dominates at 4 KB chunks. libhdf5 was benchmarked without shuffle. The speedup compounds with matrix size because libhdf5's per-chunk VFL overhead is proportional to chunk count while clawhdf5's single-pass cost is constant.

Codec Comparison: Pcodec vs Zstd-3

Pcodec (arXiv:2502.06112) is a pure-Rust lossless numerical codec with 3094% better compression ratio than Zstd for f32/f64 columns. Both sides benchmarked without auto-shuffle here (shuffle degrades Pcodec which handles byte organization internally; Zstd-3 without shuffle numbers shown for an apples-to-apples comparison).

Matrix size Pcodec Zstd-3 (no shuffle) Winner
32×32 f32 95 µs / 41 MiB/s 57 µs / 68 MiB/s Zstd-3 (1.66×)
128×128 f32 528 µs / 118 MiB/s 179 µs / 349 MiB/s Zstd-3 (2.95×)
512×512 f32 1.69 ms / 591 MiB/s 1.64 ms / 610 MiB/s Parity (3% diff)

Pcodec's fixed per-chunk distributional analysis overhead (~400 µs) dominates at 32×32 (4 KB). At 512×512 (1 MB) the speeds converge. Pcodec's advantage is compression ratio, not encode speed — less data on disk means faster reads and lower storage cost. Enable with .with_pcodec() for write-once/read-many workloads (embedding archives, scientific datasets).

Metadata Throughput

clawhdf5 accumulates all metadata in memory and serializes in one pass. libhdf5 acquires a global file mutex and flushes to disk on every attribute write or group creation.

Attributes and datasets (k = attribute or dataset count):

Workload k=4 k=16 k=64 k=128
clawhdf5 attrs_write (i64) 8.05 µs / 494 Kop/s 17.2 µs / 932 Kop/s 49.2 µs / 1.30 Mop/s 87.3 µs / 1.47 Mop/s
libhdf5 attrs_write 100 µs / 40 Kop/s 170 µs / 94 Kop/s 472 µs / 136 Kop/s 929 µs / 138 Kop/s
Speedup 12.4× 9.9× 9.6× 10.6×
clawhdf5 attrs_read 1.06 µs / 3.78 Mop/s 3.64 µs / 4.39 Mop/s 15.7 µs / 4.08 Mop/s 31.3 µs / 4.09 Mop/s
clawhdf5 string_attrs (write+read) 5.17 µs / 774 Kop/s 16.5 µs / 967 Kop/s 33.6 µs / 951 Kop/s
clawhdf5 multi_dataset_write 10.1 µs / 397 Kop/s 31.5 µs / 508 Kop/s 104 µs / 614 Kop/s

Groups (k = group count):

Workload k=4 k=16 k=32 k=64
clawhdf5 groups_create 12.1 µs / 330 Kop/s 33.7 µs / 475 Kop/s 66.7 µs / 480 Kop/s 121 µs / 529 Kop/s
libhdf5 groups_create 140 µs / 28 Kop/s 433 µs / 37 Kop/s 690 µs / 46 Kop/s 1,340 µs / 48 Kop/s
Speedup 11.6× 12.8× 9.5× 11.1×
clawhdf5 groups_traverse 664 ns / 6.0 Mop/s 3.55 µs / 4.5 Mop/s 4.87 µs / 6.6 Mop/s 10.6 µs / 6.0 Mop/s

vs libhdf5 Summary

Workload clawhdf5 libhdf5 Speedup
Sequential read, 1K f32 634 ns 45.2 µs 71×
Sequential read, 100K f32 24.5 µs · 15.2 GiB/s 73.9 µs · 5.0 GiB/s 3.0×
Sequential write, 100K f32 228 µs · 1.63 GiB/s 214 µs · 1.74 GiB/s ≈ tie
Chunked write deflate-6, 512×512 1,390 µs · 719 MiB/s 53,300 µs · 19 MiB/s 38.4×
Attribute write, 128 attrs 87.3 µs · 1.47 Mop/s 929 µs · 138 Kop/s 10.6×
Group create, 64 groups 121 µs · 529 Kop/s 1,340 µs · 48 Kop/s 11.1×

Why the Gaps

Metadata (1013×): libhdf5 was designed for MPI parallel filesystems where every metadata write must be immediately visible to other processes. It acquires a global file mutex and flushes to disk per operation. clawhdf5 builds the entire file in memory and writes it in one shot — no locking, no flushing, no C heap allocation per message.

Chunked compressed write (438×): libhdf5 writes each chunk individually through its VFL (Virtual File Layer), one pwrite() per chunk. clawhdf5 compresses all chunks in memory (Rayon parallel when > 2 chunks), lays them out contiguously, and issues a single write(). The speedup compounds with matrix size: libhdf5's per-chunk overhead is proportional to chunk count while clawhdf5's architectural cost is constant.

Small reads (2071×): libhdf5's per-open overhead (chunk cache init, SWMR lock, metadata read) dominates at sub-millisecond payloads. clawhdf5 has no global state — File::from_bytes() starts parsing immediately.

Large contiguous writes (≈ tie at 100K): Both are bottlenecked by the OS write() syscall to the page cache. There is no algorithmic headroom above ~1.7 GiB/s on this hardware.

Caveats

  • libhdf5 f64 read comparison excluded — clawhdf5's f32 datatype encoding differs from libhdf5's (known compatibility gap). f64 results are clawhdf5-only.
  • Serial benchmarks. clawhdf5 uses Rayon for chunk compression when > 2 chunks; that parallelism is already reflected in the chunked write numbers.
  • clawhdf5 reads from Vec<u8> (zero-copy from mmap in production); libhdf5 reads from a temp file. This gives clawhdf5 a structural read advantage that reflects realistic API usage.

Independent Validation: tank (Ryzen 7 7800X3D), 2026-08-03

The vs libhdf5 Summary numbers above were re-run on a second, independently administered machine (tank: AMD Ryzen 7 7800X3D, 8C/16T, Ubuntu 26.04, libhdf5 1.14.6 via apt) to confirm they reproduce off the original i7-12650H box, and to add benchmark coverage for two claims that a documentation review found were not traceable to any dated benchmark run (see git history around 2026-08-03 for context). This section documents both.

Reproduction of the vs-libhdf5 Summary table

Workload clawhdf5 (tank) libhdf5 (tank) Speedup (tank) Speedup (i7-12650H, above)
Sequential read, 1K f32 553 ns 44.2 µs 79.9× 71×
Sequential read, 100K f32 23.3 µs 63.6 µs 2.7× 3.0×
Sequential write, 100K f32 210 µs 189 µs ≈ tie (clawhdf5 ~11% behind) ≈ tie (clawhdf5 ~7% behind)
Chunked write deflate-6, 512×512 1.44 ms 65.0 ms 45.3× 38.4×
Attribute write, 128 attrs 85.2 µs 877 µs 10.3× 10.6×
Group create, 64 groups 130 µs 1.37 ms 10.6× 11.1×

Five of six rows land within ~15% of the original i7-12650H figures — consistent with normal cross-machine variance, not a methodology artifact. The chunked-write row moved further (38.4× → 45.3×, +18%): tank's libhdf5 per-chunk write cost scales worse relative to its own sequential-write throughput than on the i7, likely IPC/ memory-subsystem dependent. Both figures are real and dated; we report both rather than picking one.

New coverage: replacing the retracted "metadata parse / 308×" and "zero-copy mmap / 313 ns" claims

An earlier README revision cited 19 ns vs 2,080 µs (labeled, incorrectly, 308×) for "metadata parse," and 313 ns for "zero-copy mmap" — neither figure traced to any benchmark in this file. Both have been retracted from the README. In their place, two new Criterion benchmarks were added (crates/clawhdf5-bench/benches/h5bench_meta.rs, crates/clawhdf5-bench/benches/h5bench_read.rs) and run on tank:

metadata_open_from_disk — opens a small file from disk (std::fs::read / hdf5::File::open) and resolves one attribute. Both sides pay real OS I/O, unlike the retracted claim.

Operation clawhdf5 libhdf5 Speedup
Open file + read 1 attribute 4.01 µs 39.3 µs 9.8×

metadata_parse_in_memory (clawhdf5-only) — times File::from_bytes() alone, given bytes already resident in memory, i.e. header-parse cost with disk I/O excluded. There is no fair libhdf5-side equivalent (its API has no "parse from an in-memory buffer, skip the OS open" path), so this is reported standalone rather than as a speedup multiple — this is the honest version of what the old 19 ns number was trying to claim.

Operation clawhdf5 (in-memory, no I/O)
Parse superblock + resolve 1 attribute 549 ns

read_zerocopy_mmap — opens via MmapFile and reads an f64 dataset through read_f64_zerocopy(), summing every element to force the mapped pages to actually fault in (returning only a slice length, as an earlier draft of this benchmark did, would repeat the exact "measures nothing" mistake being fixed here).

n (f64 elements) clawhdf5 mmap (zerocopy, page-fault-forced) clawhdf5 (Vec<u8> copy) libhdf5 (disk open + copy)
1,000 7.86 µs 4.50 µs 44.2 µs
10,000 19.0 µs 9.53 µs 47.1 µs
100,000 112 µs 72.0 µs 81.2 µs

Honest result: at these sizes, forcing full materialization through the mmap path is not faster than the plain Vec<u8> copy path — mmap()/page-fault overhead per call outweighs the copy it avoids. This contradicts the retracted 313 ns claim outright and is a genuinely useful finding: MmapFile's real advantage is avoiding the allocation/copy for large files or sparse access patterns (lower peak RSS, share pages across processes), not raw single-shot read latency at these sizes. No README claim is made from this row; it's recorded here for the record and to keep future readers from reintroducing the old number.

Reproduce:

cargo bench -p clawhdf5-bench --features libhdf5-compare --bench h5bench_meta -- metadata_open_from_disk
cargo bench -p clawhdf5-bench --features libhdf5-compare --bench h5bench_meta -- metadata_parse_in_memory
cargo bench -p clawhdf5-bench --features libhdf5-compare --bench h5bench_read -- read_zerocopy_mmap