osobhandClaude Opus 5.5
b55b24b7ba
Every crate under crates/ now has a README (android, bench, cli, napi and wasm had none), each saying what the crate is, its main types and functions (names checked against the code), its cargo features with defaults and which ones build C (checked with `cargo tree`), and links to the top-level docs. Corrections to the old stubs: - clawhdf5-derive: the derive is `H5Type`, not `HDF5Type`, and it needs clawhdf5-format as a dependency. - clawhdf5-filters: deflate backends only, and no library crate depends on it; the filter pipeline and every other codec are in -format. - clawhdf5-gpu: vector distance compute, not I/O; not used by HDF5Memory::search. - clawhdf5-io: MpiVol is root-read + broadcast, not collective MPI-IO. - clawhdf5-ann: from_hdf5/search(q, k) did not exist; load_from_hdf5 and search(q, k, ef). - clawhdf5-accel: checksum::crc32_simd did not exist; the SSE4 and wasm backends are reported but run the scalar kernels. - clawhdf5-gpu: the old example called l2_distances, which does not exist (l2_search). - clawhdf5-agent: it described a "vector store" with "GPU acceleration"; it now covers HDF5Memory, search options, WAL, signing, the graph. - crates.io/docs.rs badges removed and `cargo install <crate>` replaced: nothing is published; depend on git. - fuzz: the opt-in CLAWHDF5_FUZZ_SECONDS smoke run in ci-test.sh. - tools: the FileEditor interop tests that live in this crate. - remote, py: license, other front ends, limits, File.mode/flush/chunks. The Rust examples of the facade, format, filters, accel, ann, derive and agent READMEs were compiled and run as tests (netcdf4, gpu and remote compiled only) in a scratch crate; the CLI example was run. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
50 lines
2.3 KiB
Markdown
50 lines
2.3 KiB
Markdown
# clawhdf5-cli
|
|
|
|
The `clawhdf5` command: create, fill, search and inspect a
|
|
[`clawhdf5-agent`](../clawhdf5-agent/README.md) memory store from the
|
|
shell. Output is JSON. (For general HDF5 files use `h5rs` from
|
|
[`clawhdf5-tools`](../clawhdf5-tools/README.md).)
|
|
|
|
```bash
|
|
cargo install --path crates/clawhdf5-cli # installs `clawhdf5`; not on crates.io yet
|
|
# or: cargo run -p clawhdf5-cli -- --help
|
|
```
|
|
|
|
No C is compiled.
|
|
|
|
## Commands
|
|
|
|
The store is `--path FILE` (or `CLAWHDF5_PATH`) before the subcommand.
|
|
|
|
| Command | What |
|
|
|---|---|
|
|
| `create [--agent-id ID] [--dim N] [--wal] [--f32] [--f32-index]` | a new store (dimension 384 by default); float16 embeddings and an int8 index copy unless `--f32` / `--f32-index`. The WAL is off unless `--wal` (the library's default is on), so each save is checkpointed at once |
|
|
| `save [--json '{...}']` | save one entry, from `--json` or stdin: `{"chunk", "embedding", "source_channel", "timestamp", "session_id", "tags"}` |
|
|
| `search --embedding '[...]' [--query TEXT] [-k N] [--vector-weight W] [--keyword-weight W]` | hybrid search (defaults 5 results, weights 0.7 / 0.3) |
|
|
| `recall INDEX` | one entry by index |
|
|
| `stats` | counts and configuration |
|
|
| `flush-wal` | checkpoint the WAL into the `.h5` |
|
|
| `agents-md [--output FILE]` | generate an `AGENTS.md` from the store |
|
|
| `export` | every entry as JSON lines |
|
|
| `snapshot DEST` | a copy of the store's `.h5` file |
|
|
| `keygen --out FILE` | a new Ed25519 signing key (64 hex characters, created owner-only on Unix) |
|
|
| `verify --public-key HEX_OR_FILE` | check a signed store; exit status 2 if it does not verify |
|
|
|
|
`recall`, `stats`, `agents-md` and `export` open the store read-only
|
|
(no lock, nothing written), so they work while another process has it
|
|
open. `save`, `search` (which records activation boosts) and `flush-wal`
|
|
open it for writing and take the store's lock. With
|
|
`--signing-key FILE` (or `CLAWHDF5_SIGNING_KEY`) every checkpoint a command
|
|
makes is signed; a signed store refuses to checkpoint without the key.
|
|
|
|
```bash
|
|
clawhdf5 --path mem.h5 create --agent-id demo --dim 3
|
|
echo '{"chunk":"hello","embedding":[0.1,0.2,0.3],"source_channel":"cli","timestamp":0,"session_id":"s1","tags":""}' \
|
|
| clawhdf5 --path mem.h5 save
|
|
clawhdf5 --path mem.h5 search --embedding '[0.1,0.2,0.3]' --query hello -k 3
|
|
```
|
|
|
|
## License
|
|
|
|
MIT
|