clawhdf5-migrate writes real agent stores; knowledge-graph fix; dated benchmark re-run #6

2026-09-25T12:48:06Z

osobh commented

2026-09-25 12:48:06 +00:00

Summary

Two items from the README claims audit, done as a multi-agent workflow (implement → adversarial review → fix; measure → trace every number → fix), plus a real performance bug the benchmark re-run found.

`clawhdf5-migrate` writes a real agent store (`a8fb758`)

Until now its output was a layout of its own (/chunks, /sessions, /entities, /relations, no /meta) that HDF5Memory::open rejects. So a migrated SQLite database couldn't be used as agent memory, contrary to the README.

Written through the agent's own API, so there's no second copy of the schema. Sessions, entities and relations carry over.
Precision: float16 by default, --f32 to opt out, same as clawhdf5-cli. The half-based conversion is gone.
Every source row is checked before anything is written. A wrong embedding length, an empty embedding, a dimension mismatch, or a float16 value out of range is an error naming the chunk, and an existing store is left untouched.
--incremental matches rows by content and follows the source's deleted flags.
Validation reads the result back through HDF5Memory::open_read_only, compares embeddings bit for bit, and checks that a migrated record is found by search.
Adversarial review: 2 blockers and 4 majors found and fixed, each with a regression test. 42 migrate tests, including h5py opening a migrated store.
Breaking: files written by older versions of the tool aren't agent stores; re-run the migration. The old layout's per-dataset SHA-256 attributes are gone.

Knowledge-graph traversal regression fixed (`1b3bbb0`)

bfs_neighbors and spreading_activation rebuilt an adjacency index over the whole graph on every call (1efd82c). It's now cached, and checked against a cheap fingerprint of the graph, so direct edits to the public fields still trigger a rebuild (tested).

Knowledge-graph bench	Before	After
BFS, 1K entities	155.1 µs	23.1 µs
Spreading activation, 100 entities	22.8 µs	10.1 µs

Benchmarks re-run, dated and traced (`dce5559`)

Every stale BENCHMARKS.md section was re-run on tank on 2026-09-24 (24 commands, run serially), with the command recorded for each section. A separate check traced every changed number back to the raw output.
Write path: the apparent regression isn't one. v2.3.0 on the same machine measures the same. float16 adds ~2 µs per save, and the int8 index adds nothing.
New multimodal_bench: cross-modal search at 1K and 10K records, which the README claimed but nothing measured.
footprint_bench now reports the stored precision and takes --f32.
Not re-run: consolidation efficiency's 100K row and its memory-reduction part.

Test plan

scripts/ci-test.sh locally: 14/14 (MSRV 1.92, h5py interop, zlib-ng, no-C check)
Migrate: 42 tests; the reviewer's reproduction script now fails cleanly on every case
Knowledge graph: staleness test (push, in-place rewire, pop + push at equal length); 54 knowledge tests pass
Write-path isolation: 4 store configurations, plus v2.3.0 built and run on the same machine
Gitea CI on this PR

🤖 Generated with Claude Code

## Summary Two items from the README claims audit, done as a multi-agent workflow (implement → adversarial review → fix; measure → trace every number → fix), plus a real performance bug the benchmark re-run found. ### `clawhdf5-migrate` writes a real agent store (`a8fb758`) Until now its output was a layout of its own (`/chunks`, `/sessions`, `/entities`, `/relations`, no `/meta`) that `HDF5Memory::open` rejects. So a migrated SQLite database couldn't be used as agent memory, contrary to the README. - **Written through the agent's own API**, so there's no second copy of the schema. Sessions, entities and relations carry over. - **Precision:** `float16` by default, `--f32` to opt out, same as `clawhdf5-cli`. The `half`-based conversion is gone. - **Every source row is checked before anything is written.** A wrong embedding length, an empty embedding, a dimension mismatch, or a `float16` value out of range is an error naming the chunk, and an existing store is left untouched. - **`--incremental`** matches rows by content and follows the source's deleted flags. - **Validation** reads the result back through `HDF5Memory::open_read_only`, compares embeddings bit for bit, and checks that a migrated record is found by search. - **Adversarial review:** 2 blockers and 4 majors found and fixed, each with a regression test. 42 migrate tests, including h5py opening a migrated store. - **Breaking:** files written by older versions of the tool aren't agent stores; re-run the migration. The old layout's per-dataset SHA-256 attributes are gone. ### Knowledge-graph traversal regression fixed (`1b3bbb0`) `bfs_neighbors` and `spreading_activation` rebuilt an adjacency index over the whole graph on every call (`1efd82c`). It's now cached, and checked against a cheap fingerprint of the graph, so direct edits to the public fields still trigger a rebuild (tested). | Knowledge-graph bench | Before | After | |---|---:|---:| | BFS, 1K entities | 155.1 µs | **23.1 µs** | | Spreading activation, 100 entities | 22.8 µs | 10.1 µs | ### Benchmarks re-run, dated and traced (`dce5559`) - **Every stale BENCHMARKS.md section** was re-run on tank on 2026-09-24 (24 commands, run serially), with the command recorded for each section. A separate check traced every changed number back to the raw output. - **Write path:** the apparent regression isn't one. v2.3.0 on the same machine measures the same. `float16` adds ~2 µs per save, and the int8 index adds nothing. - **New `multimodal_bench`:** cross-modal search at 1K and 10K records, which the README claimed but nothing measured. - **`footprint_bench`** now reports the stored precision and takes `--f32`. - **Not re-run:** consolidation efficiency's 100K row and its memory-reduction part. ## Test plan - [x] `scripts/ci-test.sh` locally: 14/14 (MSRV 1.92, h5py interop, zlib-ng, no-C check) - [x] Migrate: 42 tests; the reviewer's reproduction script now fails cleanly on every case - [x] Knowledge graph: staleness test (push, in-place rewire, pop + push at equal length); 54 knowledge tests pass - [x] Write-path isolation: 4 store configurations, plus v2.3.0 built and run on the same machine - [ ] Gitea CI on this PR 🤖 Generated with [Claude Code](https://claude.com/claude-code)

osobh added 3 commits 2026-09-25 12:48:07 +00:00

fix(migrate): write a real clawhdf5-agent store a8fb758489

clawhdf5-migrate wrote a layout of its own (/chunks, /sessions,
/entities, /relations, root attributes, no /meta or schema_version) that
HDF5Memory::open rejects, so a "migrated" SQLite database could not be
used as agent memory — contrary to the README.

It now writes through the agent's own API (HDF5Memory::create/open,
save_batch, the session cache and the knowledge graph), so there is no
second copy of the schema:

- sessions and entities/relations carry over; deleted rows become
  deleted records (or are left out with --skip-deleted);
- embeddings follow the library default (float16), --f32 opts out and
  --float16 is a hidden no-op, as in clawhdf5-cli; the `half`-based
  conversion is gone;
- every source row is checked before the output is created: a wrong
  embedding length, an empty embedding, a dimension that differs from
  an existing store's, or a float16 value beyond +-65504 is an error
  naming the chunk id, and an existing store is left untouched;
- --incremental opens the existing store, adds only rows it does not
  hold (matched by content) and follows the source's deleted flags;
- a source with no memory rows needs --embedding-dim;
- validation reads the result back with HDF5Memory::open_read_only,
  compares every field (embeddings bit for bit, round_to_f16 of the
  source for float16) and checks a migrated record is found by search.

clawhdf5-agent gains HDF5Memory::sessions()/sessions_mut(),
HDF5Memory::delete_batch (one save, all-or-nothing, no auto-compact),
SessionCache::add_at, and re-exports SessionCache/SessionEntry.

The old layout's per-dataset SHA-256 provenance attributes have no place
in the agent schema and are gone. An adversarial review found two
blockers (silent truncation of long embeddings; an --incremental
dimension check that could never fire) and four majors (a failed run
wiping the existing store, dim-0 stores, deleted-flag drift); all are
fixed with regression tests. 42 migrate tests, incl. h5py opening a
migrated store.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

perf(agent): cache the knowledge graph's adjacency index 1b3bbb054a

bfs_neighbors and spreading_activation built an adjacency index over the
whole graph on every call (1efd82c), so a 2-hop BFS over 1K entities
paid to index every entity and relation first: 155 us, 6.5x the 24 us
the README quoted. Found by the dated benchmark re-run.

The index is now cached on KnowledgeCache and checked against a
fingerprint of the graph on each use — one pass over entity ids and
relation endpoints, no allocation — so any change, including direct
edits of the public entities/relations Vecs (schema.rs's load path
pushes to them), still triggers a rebuild. A test edits the graph
directly in every way (push, in-place rewire, pop + push at equal
length) between traversals.

tank, 2026-09-24: BFS 1K entities 155.1 -> 23.1 us, 100 entities
17.5 -> 5.23 us, spreading activation 100 22.8 -> 10.1 us.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

bench: re-run every stale BENCHMARKS.md section, dated and traced

CI / test-arm64 (pull_request) Successful in 54s

Details

CI / test (pull_request) Successful in 5m6s

Details

dce5559ff2

Every undated or pre-September section re-run on one machine on one day
(tank, AMD Ryzen 7 7800X3D, 2026-09-24, commit 5c8323c), 24 commands run
serially with the load average checked before each, with the command
recorded for each section. A separate check traced every changed number
back to the raw output; its corrections are applied (e.g. the on-disk
~820 B/record is float16 plus always-deflated text on a synthetic corpus
of 40 distinct texts, not float16 alone).

Two apparent regressions were isolated rather than published:
- knowledge-graph traversal: a real bug, fixed in the previous commit;
- the write path: v2.3.0 built and run on the same machine measures the
  same as today, so the old 18 us / 6.17 ms figures (undated, other
  hardware) are not reproducible; float16 adds ~2 us per save and the
  int8 index nothing (both isolated by switching the bench's config).

Also:
- new multimodal_bench: cross-modal search at 1K/10K records, which the
  README claimed but nothing measured;
- footprint_bench reports whether it built float16 or f32 stores and
  takes --f32 (it kept printing "f32" after the default changed);
- README: performance tables, the "Why" table figures and the SQLite
  migration section (from the previous migrate commit);
- CHANGELOG for this branch.

Not re-run: consolidation_efficiency's 100K row and its memory-reduction
part (stopped for time), and cross_platform.sh.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

osobh merged commit c0f704c381 into main

2026-09-25 14:52:36 +00:00

osobh deleted branch feat/migrate-and-benchmarks

2026-09-25 14:52:39 +00:00

osobh referenced this pull request

2026-09-25 14:53:11 +00:00

Complete the consolidation benchmark: cheaper novelty scoring #7

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: quantumclaw/clawhdf5#6

clawhdf5-migrate writes real agent stores; knowledge-graph fix; dated benchmark re-run #6

Summary

clawhdf5-migrate writes a real agent store (a8fb758)

Knowledge-graph traversal regression fixed (1b3bbb0)

Benchmarks re-run, dated and traced (dce5559)

Test plan

`clawhdf5-migrate` writes a real agent store (`a8fb758`)

Knowledge-graph traversal regression fixed (`1b3bbb0`)

Benchmarks re-run, dated and traced (`dce5559`)