feat(agent): Ed25519-signed checkpoints
Makes the README's "cryptographically verifiable memory" true. With HDF5Memory::set_signing_key(key), every checkpoint stores a signed manifest of the store: a SHA-256 per memory record (text, embedding as stored, channel, timestamp, session, tags, deleted flag, activation) in a Merkle tree, plus hashes of the settings (and WAL mark), sessions and knowledge graph. The signature, public key and manifest hashes go in /meta; the per-record hashes in /integrity/record_hashes, so HDF5Memory::verify(path, &public_key) can say which records changed, not just that something did. A forged manifest fails the signature. Decisions, as agreed: - the key is set on the open store and never persisted; - a signed store refuses to checkpoint without its key (MemoryError::SigningKeyRequired); remove_signature() is the deliberate way back to unsigned; - checkpoints only: saves still in the WAL are not covered, and verify reports how many there are. The hashes cover exactly what the file persists, in the form the loader returns it (strings lose trailing NULs; an empty WAL mark is not written), so untouched stores verify across any number of reopen and checkpoint cycles. MemoryError becomes #[non_exhaustive] (it already gains variants in this unreleased version). CLI: keygen (owner-only key file), --signing-key / CLAWHDF5_SIGNING_KEY on writing commands (create signs immediately), verify --public-key (JSON; exit 2 if not valid), `signed` in create/stats output. Tests: reopen/checkpoint cycles with awkward strings (f16 and f32), refusal without the key, wrong and rotated keys, eight kinds of edit each detected and located, a forged manifest, unsigned stores, NULs in text, and an edit made in place with h5py that verify pinpoints. Cost on tank (search_harness --signing-study --full, 3 runs): ~20% of a checkpoint (+9 ms at 10K, +89-112 ms at 100K), verify 18.6 ms / 247 ms, 32 bytes per record in the file. New deps ed25519-dalek, sha2, rand_core: pure Rust, the no-C check passes. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
@@ -3,6 +3,9 @@
|
||||
## Unreleased
|
||||
|
||||
### Upgrade Notes
|
||||
- **Breaking:** `MemoryError` is now `#[non_exhaustive]` and gained
|
||||
`SigningKeyRequired`; a `match` on it needs a wildcard arm. Future variants
|
||||
will no longer be breaking.
|
||||
- **Breaking:** `clawhdf5-agent`'s `agent` feature is removed. It enabled
|
||||
nothing — the agent layer is always built — but the README and guides told
|
||||
people to pass it; drop `agent` from `features = [...]`.
|
||||
@@ -60,6 +63,29 @@
|
||||
`quantized_index = false`, or pass `create --f32-index` to the CLI, to opt
|
||||
out. The CLI's `--quantized-index` is still accepted but is now a no-op.
|
||||
|
||||
### Signing
|
||||
- `clawhdf5-agent`: **Ed25519-signed checkpoints** — the README's
|
||||
"cryptographically verifiable memory", now true. With
|
||||
`HDF5Memory::set_signing_key(key)`, every checkpoint stores a signed
|
||||
manifest: a SHA-256 per record (text, embedding as stored, channel,
|
||||
timestamp, session, tags, deleted flag, activation) in a Merkle tree, plus
|
||||
hashes of the settings (and WAL mark), sessions and knowledge graph, with
|
||||
the per-record hashes in `/integrity/record_hashes`.
|
||||
`HDF5Memory::verify(path, &public_key)` recomputes everything from the file
|
||||
and reports which part changed and which records (`changed_records`); a
|
||||
forged manifest fails the signature. The key is never persisted; a signed
|
||||
store refuses to checkpoint without it (`MemoryError::SigningKeyRequired`),
|
||||
and `remove_signature()` is the deliberate way back to unsigned. Saves still
|
||||
in the WAL are not covered (`wal_entries_unsigned`). Tests include every
|
||||
kind of edit, and an edit made with h5py in place, which verify pinpoints.
|
||||
Cost: ~20% of a checkpoint, 32 bytes per record (`BENCHMARKS.md`, "Signed
|
||||
checkpoints"). New dependencies `ed25519-dalek`, `sha2`, `rand_core` — pure
|
||||
Rust; the no-C check still passes.
|
||||
- `clawhdf5-cli`: `keygen --out <file>` (owner-only key file),
|
||||
`--signing-key <file>` / `CLAWHDF5_SIGNING_KEY` on writing commands
|
||||
(`create` signs immediately), `verify --public-key <hex|file>` (JSON report;
|
||||
exit status 2 if not valid), and `signed` in `create`/`stats` output.
|
||||
|
||||
### Migration
|
||||
- `clawhdf5-migrate`: writes through the agent's own API (`HDF5Memory::create`
|
||||
/ `open`, `save_batch`, the session cache and knowledge graph), so there is
|
||||
|
||||
Reference in New Issue
Block a user