osobhandClaude Opus 5.5
a8fb758489
clawhdf5-migrate wrote a layout of its own (/chunks, /sessions, /entities, /relations, root attributes, no /meta or schema_version) that HDF5Memory::open rejects, so a "migrated" SQLite database could not be used as agent memory — contrary to the README. It now writes through the agent's own API (HDF5Memory::create/open, save_batch, the session cache and the knowledge graph), so there is no second copy of the schema: - sessions and entities/relations carry over; deleted rows become deleted records (or are left out with --skip-deleted); - embeddings follow the library default (float16), --f32 opts out and --float16 is a hidden no-op, as in clawhdf5-cli; the `half`-based conversion is gone; - every source row is checked before the output is created: a wrong embedding length, an empty embedding, a dimension that differs from an existing store's, or a float16 value beyond +-65504 is an error naming the chunk id, and an existing store is left untouched; - --incremental opens the existing store, adds only rows it does not hold (matched by content) and follows the source's deleted flags; - a source with no memory rows needs --embedding-dim; - validation reads the result back with HDF5Memory::open_read_only, compares every field (embeddings bit for bit, round_to_f16 of the source for float16) and checks a migrated record is found by search. clawhdf5-agent gains HDF5Memory::sessions()/sessions_mut(), HDF5Memory::delete_batch (one save, all-or-nothing, no auto-compact), SessionCache::add_at, and re-exports SessionCache/SessionEntry. The old layout's per-dataset SHA-256 provenance attributes have no place in the agent schema and are gone. An adversarial review found two blockers (silent truncation of long embeddings; an --incremental dimension check that could never fire) and four majors (a failed run wiping the existing store, dim-0 stores, deleted-flag drift); all are fixed with regression tests. 42 migrate tests, incl. h5py opening a migrated store. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
25 lines
762 B
TOML
25 lines
762 B
TOML
[package]
|
|
name = "clawhdf5-migrate"
|
|
version = "2.7.0"
|
|
edition = "2024"
|
|
rust-version.workspace = true
|
|
description = "CLI to migrate SQLite agent memory databases to clawhdf5-agent stores"
|
|
license = "MIT"
|
|
repository = "https://git.redclaw.dev/quantumclaw/clawhdf5"
|
|
readme = "README.md"
|
|
keywords = ["sqlite", "hdf5", "migration", "agent", "memory"]
|
|
categories = ["command-line-utilities", "database"]
|
|
|
|
[[bin]]
|
|
name = "clawhdf5-migrate"
|
|
path = "src/main.rs"
|
|
|
|
[dependencies]
|
|
clawhdf5-agent = { path = "../clawhdf5-agent", version = "2.7.0" }
|
|
clawhdf5-format = { path = "../clawhdf5-format", version = "2.7.0" }
|
|
rusqlite = { version = "0.31", features = ["bundled"] }
|
|
clap = { version = "4", features = ["derive"] }
|
|
|
|
[dev-dependencies]
|
|
tempfile = { workspace = true }
|