Numbers, API names, feature defaults and PR references checked against
CONFORMANCE.md, BENCHMARKS.md, CHANGELOG.md, the code and git history.
- int8 index figures (1.74x memory, 1.63x QPS) carry the dates git gives
them (2026-09-19/20, machine not recorded, not re-run) instead of none;
the Pi 5 1.18x carries 2026-09-21.
- BENCHMARKS headline: the libhdf5 chunked-write figure is the newest
measurement (35x, 2026-09-23), not 45.3x (2026-08-03).
- Conformance counts follow the 2026-09-28 run (1 our-error, 2 ref-bug)
in conformance/README.md, ROADMAP.md and CLAUDE.md, with a pointer to
the bad_nbit_parms_walk.h5 flip.
- README: LZ4 is opt-in; the browser refuses reference/opaque/bitfield/
time datasets too; zlib-rs byte-identity scoped to what was measured;
macOS default links the system libz for inflate.
- Crate READMEs: system-zlib-decompress does something (macOS), SweepDetector
lives in prefetch, checkpoint after more than 500 WAL entries, NetCDF-4
unlimited-dimension size warning.
- agent-memory.md: string-dataset compression threshold, agents-md prints
Markdown, float16 file sizes linked to their study.
- known-issues.md: contiguous selection reads, 1.21x vs h5py threads.
- docs/README.md, USE_CASES.md, ROADMAP.md, CLAUDE.md: range-read
milestones M0-M5 and PRs #17-#19, missing README rows, CLI keygen/verify,
dated figures, fast-math is not BLAS.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Found while checking the README refresh: clawhdf5-netcdf4 reads an
unlimited dimension's size from its dimension-scale dataset, which
netCDF-C never extends, so a dimension with 2 records reports 0.
Variable shapes and values are right. To be fixed separately.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- docs/README.md links the improvement logs and June plans where the
refresh archived them (docs/archive/).
- clawhdf5-py README: 'r+' creates and replaces attributes (compact or
dense); only deleting them is unsupported.
- scripts/run-benchmarks.sh benchmarked the pre-rename rustyhdf5-format
and overwrote BENCHMARKS.md; nothing referenced it. Removed.
- Cargo.toml descriptions no longer name rustyhdf5/edgehdf5; clawhdf5-gpu
says it is not HDF5 I/O.
- benchmarks/cross_platform.sh pointed at a ROADMAP section that no longer
exists.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Moved to docs/archive/ (kept for their history, not deleted), each with a
one-line header saying it is historical and what supersedes it:
- IMPROVEMENT_LOG.md: three automated-loop PRs from April-May 2026, on
the earlier quantumclaw PR numbering, which now collides with this
repo's #12-#15. Superseded by CHANGELOG.md and git log.
- IMPROVEMENT_SCAN.md: one scan's notes (2026-05-04) of changes merged
long ago. Superseded by CHANGELOG.md and git log.
- docs/superpowers/plans/*.md -> docs/archive/plans/: agent pre-work
plans for d6c4d4f (2026-06-30), already marked implemented. The MPI plan
promised collective MPI-IO, which is not what shipped (MpiVol is
root-read + broadcast); its header says so and points at ROADMAP.md,
where collective I/O is an open item.
Nothing links to the old paths (research/*.md mention IMPROVEMENT_LOG.md
and ROADMAP.md in prose only).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The old file was a tracker for the mid-2026 agent-memory tracks, last
updated 2026-08-05, with an OpenClaw track and "what's next" items that
have since shipped (CI, fuzz target, WAL checksums, HNSW parallel build).
Now: releases v2.0.0-v2.7.0 and every PR merged since (#3-#21, merge
dates from git log), range-read milestones M0-M5, a one-paragraph summary
of the agent-memory work, and what is genuinely next: crates.io and PyPI
publishing (plus the broken Node package), SWMR writing, MPI collective
I/O, paged-metadata single-request reads, Blosc2/ZFP encoders, and the
open items of docs/known-issues.md. OpenClaw and ZeroClaw are listed only
as withdrawn. No dates are given for future work.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Explains each class compare.py assigns, how ref_bugs.py confirms a
ref-bug in every run and how ref.py corrects h5py's big-endian VL values
(ref_fix), and records the latest result (602 of 697 ok, 3 ref-bug,
2026-09-27) with a link to CONFORMANCE.md. Mentions --no-fetch.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The size table predated openUrl (it said so). Re-measured on tank at
9b5803f with `bash examples/wasm-viewer/build.sh`, then `wc -c` and
`gzip -9 -n -c`: the wasm is 1,384,607 B (378,485 gzipped), was 627,501
(191,639); the glue 40,711 (8,181), was 21,826 (4,487); remote.js 9,326
(3,448). 476,062 B of the wasm is the function-name section. opt-level z
(CARGO_PROFILE_WASM_RELEASE_OPT_LEVEL=z) is now 1% smaller gzipped than
the profile's s; 3 is larger. h5wasm rows unchanged.
Also adds the measured cost of listing a large group and reading one
dataset by URL (passes / requests / bytes, from CHANGELOG, 2026-09-27).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Every crate under crates/ now has a README (android, bench, cli, napi and
wasm had none), each saying what the crate is, its main types and
functions (names checked against the code), its cargo features with
defaults and which ones build C (checked with `cargo tree`), and links to
the top-level docs.
Corrections to the old stubs:
- clawhdf5-derive: the derive is `H5Type`, not `HDF5Type`, and it needs
clawhdf5-format as a dependency.
- clawhdf5-filters: deflate backends only, and no library crate depends
on it; the filter pipeline and every other codec are in -format.
- clawhdf5-gpu: vector distance compute, not I/O; not used by
HDF5Memory::search.
- clawhdf5-io: MpiVol is root-read + broadcast, not collective MPI-IO.
- clawhdf5-ann: from_hdf5/search(q, k) did not exist; load_from_hdf5 and
search(q, k, ef).
- clawhdf5-accel: checksum::crc32_simd did not exist; the SSE4 and wasm
backends are reported but run the scalar kernels.
- clawhdf5-gpu: the old example called l2_distances, which does not
exist (l2_search).
- clawhdf5-agent: it described a "vector store" with "GPU acceleration";
it now covers HDF5Memory, search options, WAL, signing, the graph.
- crates.io/docs.rs badges removed and `cargo install <crate>` replaced:
nothing is published; depend on git.
- fuzz: the opt-in CLAWHDF5_FUZZ_SECONDS smoke run in ci-test.sh.
- tools: the FileEditor interop tests that live in this crate.
- remote, py: license, other front ends, limits, File.mode/flush/chunks.
The Rust examples of the facade, format, filters, accel, ann, derive and
agent READMEs were compiled and run as tests (netcdf4, gpu and remote
compiled only) in a scratch crate; the CLI example was run.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
One line per document: user guides, evidence (CONFORMANCE, BENCHMARKS,
the conformance README), design docs, crate and package READMEs, and the
historical notes (roadmap, improvement logs, June plans, research briefs).
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
QUICKSTART used APIs that do not exist (file.dataset_names(),
File::attr, AttrValue::Str, memory.search(&q, 5), MemoryConfig::new with
a &str, consolidation without timestamps), `clawhdf5 = "2.0"` from
crates.io, and "3-45x faster than libhdf5". It now covers HDF5 in Rust
(write, read, strings, in-place append, remote, SWMR), Python (read, r+,
w, URLs), NetCDF-4, h5rs, agent memory and the CLI, every snippet
compiled and run (Python against a wheel built from the tree).
USE_CASES dropped claims with no source (the agent crate adds ~2MB,
IVF-PQ under 1.2 ms on modest hardware, an OpenClaw scenario, a .brain
layout and `clawhub publish` commands) and now covers the HDF5 cases
(no-C builds, threads, remote data, untrusted files, SWMR, in-place
edits), the agent cases with measured numbers, and when to use
something else.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Leads with what clawhdf5 is today for an HDF5 reader: the conformance
result (602 of 697, 0 mismatches, no panic/hang/crash; CONFORMANCE.md of
2026-09-28), the CVE corpus against h5dump and h5py, concurrent reads
against h5py threads and processes (BENCHMARKS.md, 2026-09-26, c5334b1)
and the libhdf5 comparison with its date and caveat; then a feature matrix
(supported / read only / not supported), install from git and maturin,
Rust and Python quick starts, remote files, the browser, SWMR, h5rs, a
short agent-memory section, the crate map and a documentation table.
Removed: the unverifiable "1850+ tests" badge and "~86K lines" footer,
the "What's new v2.2 -> v2.7" list (it is CHANGELOG.md), the agent
comparison table with other products, the Phase 1/2 roadmap checklist,
and the long agent sections (now docs/agent-memory.md). Fixed: crates
listed as C-free, SZIP / N-Bit / scale-offset as read-only, virtual
datasets as written too, Python 'r+' can create attributes (it cannot
create or delete objects). Every code snippet was compiled and run
against the workspace, the Python ones against a wheel built from it.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The agent-memory detail that lived only in README.md (architecture,
modules, performance and footprint tables, LongMemEval, feature flags and
settings, file schema, SQLite migration, research foundation) moves to its
own page, so the README can lead with the HDF5 library. Code examples are
updated to the current API (MemoryConfig::new takes a PathBuf,
HDF5Memory::search with SearchOptions, consolidation with timestamps) and
were compiled and run against the workspace; the CLI section was run
against the built `clawhdf5` binary. New: the CLI's search defaults to
0.7/0.3, not the library's 0.4/0.6; the /integrity group of signed stores.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Standing rules (no C by default, h5py must read what we write, one
float16 implementation, claims need evidence, OpenClaw/ZeroClaw
withdrawn, the known-issues rule) are gathered in one place; library and
agent-memory invariants are split; the CI section lists what
ci-test.sh and the conformance workflow run now instead of a dated
"two jobs, green" line. Adds the conformance, remote, wasm, Python and
benchmark workflows (idle load below 2, dated records), and warns that
scripts/run-benchmarks.sh is stale and overwrites BENCHMARKS.md. Crate
roles corrected (clawhdf5-io is I/O adapters; codecs live in
clawhdf5-format). Benchmark figures now live in BENCHMARKS.md only.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The committed CONFORMANCE.md counts bad_nbit_parms_walk.h5 as an
our-error because its six h5py reads agreed in that run; a rerun the
same day confirmed the over-read and counted it ref-bug. The ok count
(602 of 697) is the same either way.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
range-reads.md opens with a table of milestones M0-M5 and the PRs that
merged them (#17-#21), replacing a header left garbled by earlier
merges, and each milestone's status names its PR. swmr.md says the
reader is merged (PR #19) and the writer does not exist. openclaw.md
links the Node package's known-issues entry.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A "Current headline numbers" table gives each figure's newest dated
measurement with its machine, command and section. Sections a later run
replaced are marked superseded with a link to the newer one; stale
"still open" notes and cross-references now point at the fixes. No
measured value changed.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
An "Open issues" table at the top links each open entry; fixed entries
move to "Fixed (history)", newest first, keeping the date, PR, affected
releases and what users must do. Open entries re-checked against main
(9b5803f): the remaining audit gaps are gathered into one entry, the
range-read and remote limits no longer contradict themselves (Python and
the browser open URLs; SWMR reading is File::open_swmr), and the
nondeterministic damaged-chunk error, fixed in PR #19, has its own
history entry.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
