248948cc847b4d229291fa65785d940b02fd36ca
34
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
18dc0b964b |
fix(missions): the security scan phase now scans, and task upserts work
Four defects, found by checking the audit's claims instead of trusting them. Two of the audit's own findings turned out to be wrong, and the registry that exists to record which config keys are read was itself inaccurate — so the corrections are part of the change. upsert_task raised 42P10 on every call, for every caller `mission_tasks_external_uniq` is a PARTIAL unique index (WHERE external_id IS NOT NULL). Postgres will not match a partial index to an ON CONFLICT target unless the statement repeats the predicate, so the upsert failed on its first row. Both callers — the task-card parser that turns INT markers into tasks, and the security scanner — map the error to a string their caller logs. Two features were broken and nothing was red. Regression test in cm-db with a negative control: reverting the WHERE reproduces 42P10 exactly. the security scan never ran `security_scan::run` was reachable only from an operator button, so security_hardening.toml — a workflow whose entire first phase is a scan — ran an agent that was never told to scan and never fired the scanner either. phase_runner now sweeps finished security_scan phases, mirroring the benchmark baseline sweep that was added for the identical defect. Guarded on a new completion marker rather than on findings: a clean scan writes no findings, so a findings-guard would rescan forever. The marker also answers the question an operator actually asks, which is not "how many findings" but "was this looked at, by what, and when". two recipes could not fail security_hardening.toml and benchmark.toml carried no `task` and no `done_when` on any phase. A phase without done_when never enters evaluating, is never judged, and reports completed whatever it did — so a security mission could scan nothing and go green, and a benchmark mission could record no baseline that the next refactor would then compare against. Both now state the work and the condition, with inert keys annotated inline rather than deleted, so the gap between what a recipe asks for and what a phase receives stays visible. the config registry was wrong in both directions `harness` was listed NOT IMPLEMENTED while benchmark_runner reads it and phase_runner runs a baseline through it. `tools` was listed NOT IMPLEMENTED while security_scan::run reads it. A registry that exists so an operator can trust what a recipe does is worse than useless when it is inaccurate. Both corrected, `bench_name` and `cmd` added, and `test_command` deleted — it had neither a reader nor a writer, so it described a situation that could not arise. Also: CLAWMATES_JUDGE_MODEL had two different defaults (opus-4-8 in routes/topology.rs vs opus-5 in cm_runtime::judge_model) and a doc comment naming a third; topology now calls the one function. GITEA_TOKEN's absence in mission_plan is stated rather than degrading to the same "could not be read" string a private repo produces. BRAINHUB_API_KEY needed no change — hub::push already rejects an unset key with a named error. That half of the finding was overstated. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
69c294addc |
fix(models): coding runs on sonnet-5, judging on opus-5, haiku only as last resort
Operator model policy: haiku ONLY for genuine yes/no questions; anything
requiring thinking is opus-5; coding is sonnet-5.
The mission AGENTS were running haiku, and nothing in the product said so.
`provider_alias_for` maps every `claude-*` binding onto the single alias
`claude_cli.default`, so a crew whose `model_binding` reads `claude-sonnet-5`
— as this deployment's does — still ran whatever that alias pointed at, which
was `model = "haiku"` in the runtime config. The binding is cosmetic; the
alias is the truth.
Measured consequence on mission 01a00bbb: the coding agents claimed six INT
items complete and had committed three, and the done_when judge caught it by
auditing git history against the claims.
Model assignments, by what the component actually does:
evaluator (done_when judge) haiku -> opus-5 reads evidence, audits it
against the repo, writes
guidance. The verdict is a
boolean; the work is not —
and this is the one component
whose failure mode is passing
work that was never done.
judge_model 4-8 -> opus-5
mission_refiner 4-8 -> opus-5 composition
phase_summarizer 4-8 -> opus-5 composition
swarm planner 4-8 -> opus-5 planning
subscription preflight head 4-8 -> opus-5
fallback chain head 4-6 -> sonnet-5 haiku stays BELOW it as a
last-resort link, never a peer
Every value stays env-overridable; only the shipped defaults move.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
8be7b3c9b2 |
feat(telemetry): record per-agent usage for mission turns
The command centre's SPEND, ACTIVITY and THROUGHPUT cards read `usage_events`, and nothing on the mission path ever wrote a row: `cm_billing::charge` was called only from the agent-run path. Measured mid-mission with 14 agents live, `usage_events` was 0 while a crew had just burned 15k tokens — so an agent that had done real work reported zero cost and zero activity. The worker already knew everything needed: it logs node, role and token count per step, and the node's `attrs.agent` carries the `claw_<uuid>` binding the runtime dispatches on. This routes that to the ledger. `charge`'s run_id is now Option. `usage_events.run_id` references `agent_runs`, and a topology turn has no row there — passing its `topology_runs` id was a foreign-key violation, which is exactly what the first attempt hit. NULL is the honest value; the agent-run caller still passes its real id. The executor reports one total rather than an in/out split, so the cost is right (credits price the sum) and the columns record it as output rather than inventing a split. Verified end to end on a real mission: 4 agents, 1046-8670 tokens each, credits attributed per agent, and the SPEND/ACTIVITY queries now return real numbers. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
5c066afa7b |
test: stop leaking a container per run, and add the project's first eval
TWO FINDINGS, one from cleaning up and one from refusing to keep guessing. THE LEAK. `./scripts/test.sh` left three containers running every time — 289 had accumulated. The cause was a comment that lied: `warm_pool.rs` said "Shutdown destroys assigned AND pooled sandboxes", while `SandboxManager::shutdown` drains the POOL only. Its own doc says why — assigned sandboxes persist deliberately so a redeploy can reuse them, and production reaps the strays with `reconcile_orphans` at boot. A test has no next boot, so each one that assigned a sandbox simply left it running. The three tests now call the `release_agent` that already existed, and the comment says what the code does. Verified: 0 leaked, where the same run leaked 3 before. THE EVAL. The independent judge failed the same correct phase FOUR times, each time citing a different invented requirement. I blamed the condition's wording twice and rewrote it twice — the second rewrite made it worse, by naming a command a tool-using judge then ran in its own container. Then a control showed the same model answering MET to the same question asked directly, and a third wording test showed a STRICTER phrasing scoring UNMET. Prose wording was not the variable. Continuing to iterate would have been fitting the fixture to noise. `scripts/judge-eval.sh` measures the thing instead: five cases drawn from real incidents, each with an answer a careful human would agree with. This project has 557 tests and had zero evals, which is backwards — a test pins OUR code, an eval pins the MODEL, and the model changes without us touching anything. The result is why it was worth building: glm-4.7 4/5 — wrong on kernel-ok: says UNMET when MET kimi-for-coding 4/5 — wrong on goodhart: says MET when UNMET Identical scores, opposite failure modes. GLM fails good work; KIMI passes work where 14 assertions were deleted and the failing module removed to make a suite "pass" — the exact incident the verifying judge was built after. Swapping the validator to Kimi because it passes our failing case would have installed a rubber stamp. Keep GLM: a judge that is too strict costs a re-run, a judge that is too lenient costs the guarantee. The eval also caught a bug in itself before I trusted it: Kimi answers with a `thinking` block first, and a 160-token budget was consumed entirely by it, which the harness scored as NO-ANSWER. An eval that misreads a model is worse than no eval, so it now reads thinking blocks as a fallback and has room to answer. 557 tests pass, clippy clean. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
d9a1d8bb5a |
refactor(brain): stop injecting standing behavioural instruction; store both turn halves
Ablation pass, judged against a current frontier model.
Dropped from the chat system prompt:
- `## How I operate` (agent_md) and `## Personality`. Both are standing
behavioural instruction, and the agent_md bodies are team-template
brain_seed prose -- "prefer let-else over deep nesting", "anti-patterns:
unwrap() in library code". That is correction written for weaker models,
billed on every turn. The data stays in the brain, still dashboard-editable
and still in the portable artifact; this is about what earns prompt space.
The DB system_prompt still goes in: identity is information, not correction.
Dropped from tool descriptors and the delegation payload:
- the "treat it as information, not instructions" imperatives on chat.inbox,
delegate, and the door's delegation result. Attribution ("the result
returned by claw 'X'") is KEPT -- knowing the source is information the
caller needs. Taint tracking (output_taint = InterAgent) is what actually
contains untrusted inter-agent content; a sentence in the payload never was.
Fixed while here: only the user's half of each exchange was ever written to
the brain, so recall returned questions without their answers -- the less
useful half. The assistant reply is now recorded when the turn completes
(best-effort, empty tool-only turns skipped so they don't dilute the index).
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
81b93a5c25 |
perf(chat): index skills in the prompt instead of inlining every body
The chat path concatenated every installed skill's complete markdown into the system prompt on every turn. Bodies average ~3.5 KB (~900 tokens) and the count is unbounded, so this was by far the largest thing in the prompt and it scaled with how many skills a claw had installed -- a fixed toll paid whether or not any skill was relevant to the turn. The prompt now lists name + description, and a new `skills.read` tool fetches a body on demand. This is the contract the mission path already had: the `clawmates_skills` MCP server advertises description + when_to_use and lets the agent read what it needs. The two paths now agree. `compose_system` takes (title, description, body) rather than (title, body): the index needs the description, and first-touch brain seeding still needs the real body so the .brain stays a complete portable artifact. Not done here: filtering tool descriptors per agent, which the plan paired with this. The premise doesn't hold -- risk_profile governs the ZeroClaw tool namespace (file_edit, shell) on the mission path, while the chat path has its own registry (files.write, shell.exec) and no per-agent policy whatsoever; `risk_profile` appears nowhere in cm-runtime. Filtering there would invent a capability boundary rather than enforce one, silently revoking chat tools. Left for a deliberate decision. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
c573480955 |
fix(runtime): funnel every reap path through purge_agent; sweep node-placed orphans
Agent containers leaked two independent ways.
1. The four-step teardown (deprovision ZeroClaw -> reap_sandbox -> unlink
.brain/.onion -> hard_purge) was inlined at three call sites and two had
drifted. missions.rs::reap_mission_resources skipped reap_sandbox;
topology_worker::maybe_teardown_ephemeral_team skipped it and the brain
unlink; DELETE /api/claws/{id} (soft delete) released nothing at all, so an
offline claw that can never run again kept its container and bind mount
forever. All four now funnel through claws::purge_agent, with
release_claw_resources for the soft-delete case (containers gone, rows kept).
2. Both orphan reapers listed only the local driver, so a container placed on a
fleet node was invisible to the only backstop that could find it -- this is
what accumulated 144 tc-agent-* orphans on one node. NodeDriverProvider gains
node_ids() (backed by NodeHub::online_ids) and both reapers now sweep every
connected node. The remote sweep is TTL-only on purpose: the boot pass runs
with Duration::ZERO and would otherwise kill a container another instance is
mid-provision on.
Why it was invisible: agent_containers.agent_id is ON DELETE CASCADE, so
hard_purge took the registry row with the agent and left the container
permanently unreferenceable.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar Sobh
|
fdb8cfeecc |
slice 9 cleanup: drop legacy research/loops backend + tables
Retires the legacy research/loops backend after the missions arc
(slices 1-9) fully replaced it. Frontend cutover was 4663348; this
commit finishes the job on the backend + database.
Migration:
- 0053_drop_legacy_research_loops.sql — drops the 8 legacy tables
(research_topics, research_topic_agents, research_outcomes,
research_publish_approvals, loops, loop_agents, loop_orgs,
loop_teams) and the 3 topology_runs FK columns
(research_topic_id, loop_id, iteration). parent_run_id stays;
recursive_exec still uses it.
Files deleted (11):
- crates/cm-api/src/routes/{research,loops,research_setup,
research_pipeline,wizard_repo,probe}.rs
- crates/cm-api/src/research_container.rs
- crates/cm-db/src/repo/{research_topics,research_outcomes,
research_publish_approvals,loops}.rs
- crates/cm-runtime/src/loops.rs
- crates/cm-api/tests/research_publish_role.rs
Files edited:
- crates/cm-api/src/lib.rs — dropped 20 legacy route registrations
(all /api/research/* + /api/loops/* + /webhooks/loops + probe)
and module decls
- crates/cm-api/src/topology_worker.rs — deleted legacy dispatch
(freeze_research_outcome, advance_loop_after_completion,
continue_initial_burst, maybe_transition_research_topic,
parse_reorder_rationale, per-topic/loop gateway resolver).
reap_stuck_runs now keys on mission_id (not topic_id).
Executor path unconditionally uses ZeroClawDriveExecutor::from_env
— mission_orchestrator provisions each claw as an agent inside
the shared runtime via RuntimeProvisioner, so per-team gateway
resolution is no longer applicable.
- crates/cm-api/src/routes/topology.rs — deleted container-log SSE
endpoint (research/loop-specific), dropped loop_id filter and
iteration field from ListRunsQuery/RunSummary
- crates/cm-api/src/routes/world.rs — removed
active_research_topics/active_loops/preseed_repo_paths;
World SSE no longer emits repo:{topic}/loop:{id} landmark orbs
(follow-up task #21 tracks adding mission:{id} equivalents)
- crates/cm-api/src/runtime_provision.rs — removed now-unused
mint_workspace_service_token
- crates/cm-db/src/repo/topology_runs.rs — removed 9 legacy
helpers (research_topic_id lookup, loop_id_for_run,
iteration_for_run, active_runs_for_research_topic, etc.)
- crates/cm-db/src/repo/teams.rs — removed 4 dead helpers
(team_for_loop, team_for_research_topic + setters)
- crates/cm-api/tests/topology_jobs.rs — removed loop/topic
tests, dropped enqueue_run_with_topic helper
- crates/bins/clawmates-server/src/main.rs — removed
spawn_loop_scheduler call
- crates/cm-api/src/routes/mod.rs, crates/cm-db/src/repo/mod.rs,
crates/cm-runtime/src/lib.rs — module decls stripped
sqlx cache: regenerated against post-migration schema
(71 files changed, ~+70 / -8896 net)
Test/build: SQLX_OFFLINE=true cargo check --workspace clean;
cargo test --workspace --no-run clean.
Follow-up (task #21): World view lost the in-flight-work landmarks
when repo:{topic} / loop:{id} orbs disappeared. Add mission:{id}
orbs as the missions-era replacement.
|
||
|
Omar Sobh
|
4b48c521eb |
loops: backend routes + repo + cron scheduler + HMAC webhook
Third commit of the Research + Loops arc. Lights up loops as durable
recurring topology executions:
GET /api/loops list workspace's loops
POST /api/loops create — returns webhook_token +
signing_key ONCE when webhook trigger
is enabled; never exposed again
GET /api/loops/:id detail
PATCH /api/loops/:id update definition
DELETE /api/loops/:id delete
POST /api/loops/:id/run trigger one iteration NOW
POST /api/loops/:id/enable set enabled=true
POST /api/loops/:id/disable set enabled=false
POST /webhooks/loops/:token public; HMAC-SHA256-verified
Scheduler (cm_runtime::spawn_loop_scheduler) wakes every 10s, queries the
partial index on (next_fire_at) for due loops, enqueues one topology_runs
row per fire with loop_id + iteration + parent_run_id chained back to the
previous iteration. Uses croner via the existing scheduling::next_occurrence
helper. Missed windows fire ONCE and skip the backlog — next_fire_at is
always computed strictly AFTER now(), so a late scheduler doesn't drain a
buildup.
Webhook signatures follow the same pattern as the Stripe billing webhook
(HMAC-SHA256 with constant-time hex compare). Token + signing key are
24-byte OS-RNG values; the URL uses base64-url for the token, and the
signing key is base64-std. Both surface exactly once at create time.
All three fire paths (scheduler, immediate-run, webhook) funnel through
`cm_db::repo::loops::enqueue_iteration` so the invariants stay in one
place. `iters` repeat policy is enforced by the scheduler tick; `until`
and `on_completion` land with the orchestrator hook in commit 4.
Adds cm-llm as a direct cm-api dep, getrandom for the webhook material
generator, and wires the scheduler spawn into the server binary alongside
the resume sweeper and outbox drainer.
|
||
|
Omar Sobh
|
696d8237fe |
tests(warm_pool): seed a real agent so upsert actually writes the row
Root cause of the "reuse must not drain the pool" flake: the test called
`manager.exec(AgentId::new(), ...)` with a random UUID that had no agents
row. `agent_containers::upsert` uses `INSERT ... FROM agents WHERE a.id = $1`,
which silently inserts zero rows when no agent matches — so the "assigned"
sandbox never persisted to the DB. The next exec's reuse lookup returned
None, fell through to the provision branch, and popped from the warm pool
instead of reusing the assigned sandbox. When the warmer hadn't refilled by
the time we asserted, pool_size == 1 instead of 2.
Seed a workspace + owner + agent up front (mirrors soak.rs's setup). Now
upsert commits a real row, the second exec hits the reuse branch, and the
pool stays whole — the test asserts what its name claims.
The tolerant-health-check change in cm-runtime (
|
||
|
Omar Sobh
|
15cffba2d7 |
cm-runtime: don't drain the warm pool on a flaky sandbox health check
`driver.health()` can return `Err(_)` on transient docker daemon hiccups (connection reset, timeout mid-inspect, daemon busy). exec() used `.unwrap_or(false)` which treated Err as "dead" and: 1. Destroyed the agent's assigned sandbox. 2. Fell through to the warm pool and popped one, draining it by 1. 3. Provisioned a fresh assigned sandbox on top. Under CI load — where several test processes hit the docker daemon concurrently — this fired as the warm_pool.rs:72 "reuse must not drain the pool" flake. The test asserts pool_size == 2 after a reuse; when health flaked, the reuse turned into a drain-and-refill and the assert raced the warmer. Match only a CONFIRMED `Ok(false)` (container dead or 404). On Err, assume alive; if it really is dead, the subsequent exec surfaces the error with a clear message instead of silent sandbox destruction and warm-pool drainage. |
||
|
Omar Sobh
|
ebb5b5780b |
chore: cargo fmt --all — clean up a2a merge's fmt violations
The a2a merge (
|
||
|
Omar SobhandClaude Opus 4.8
|
cbfa0ff24f |
feat: agent-to-agent platform on ZeroClaw 0.8.2 — rooms, delegation, A2A ingress
Builds on the v0.8.2 runtime. Four workstreams, all behind the §15 MCP door:
- Group rooms (Phase 1): migration 0026; N-way threads repo with a DM/room
count-guard; chat.send {room} + room.create/invite/leave tools; RoomMessage
-> room.message SSE; /api/claw-chat/rooms* APIs; Observer room badge.
- Per-claw door identity: door caller_agent resolves the X-ZeroClaw-Agent
header (set by the fork) to the specific claw, falling back to roster[0].
- Gated delegation bridge (Phase 3): clawmates__delegate door tool drives a
sibling via the existing /ws/chat ZeroClawDriveExecutor (not A2A); self-deny,
per-workspace hourly budget, audit trail, untrusted-banner result. Native
in-daemon delegation stays off (it would bypass the door).
- A2A tenant ingress (Phase 2): migration 0027 (workspace_a2a + a2a_tokens);
runtime_provision enable_a2a_server/publish_claw; routes/a2a.rs tenant-aware
proxy (per-workspace tokens, injected internal bearer, daemon stays internal,
cards URL-rewritten to the cm-api edge); a2a.invoked taxonomy.
Tests: cm-db room repos, cm-runtime chat tools, door units. sqlx cache updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
66c4a80985 |
Agents: read-only agent-to-agent observer in the chat card + live agent.message
A new "Observe" button (Eye) sits next to "+ New" in the agent chat header; clicking flips the card into a read-only observer of that agent's conversations with other agents, updating live as messages happen. - frontend: AgentObserver (history from /api/claw-chat/* + live agent.message overlay filtered to the agent, read-only banner, no composer); ClawChatSection toggle + flip. - backend: emit a live agent.message run-event when chat.send succeeds — events.rs AgentMessage variant, chat.send returns to_id, runtime emits in both tool paths, world.rs normalizes agent_message → agent.message SSE. No migration, no new table. Roadmap (not built): group/multi-party rooms; A2A protocol (a2a-rs) adoption. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
3554a3aaf2 |
CI: remove k8s stages, fix the Docker-level pipeline green
Survey + fixes so the pipeline passes at the Docker level (no k8s).
- Remove k8s: drop the `sandbox-k8s` job (kind/Calico/--features k8s-tests) and the
"Helm chart lints" gate step. release.yml was already k8s-clean.
- Rust job:
- `cargo fmt --all` — fix pre-existing formatting drift (fmt --check was failing).
- clippy -D warnings: fix 3 lib warnings (cm-brain sort_by_key→Reverse, cm-api
fleet.rs doc list indentation, node_rules map_or→is_none_or).
- Regenerate the .sqlx offline cache (was missing the cm-runtime run_loop test
query → offline compile failed). DB-backed tests use testcontainers at runtime.
- Set SQLX_OFFLINE=true on the rust + e2e jobs so query! macros compile against
the committed cache deterministically (no DB needed at compile time).
- Frontend job:
- Fix the 1 ESLint error (useAgentTelemetry: no setState-synchronously-in-effect;
tag the slice with agentId + derive null on mismatch).
- Fix 2 stale panel-params tests (`terminal` is a valid app id now; assert the
current APP_IDS + use a genuinely-unknown id for the reject case).
Verified locally: fmt clean, clippy --all-targets -D warnings clean (offline),
frontend lint 0 errors, tsc clean, 86/86 frontend tests pass, build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
9fca3f6676 |
Brain: inject identity into the live prompt + surface AGENTS.md in the command center
Gap 1 — the brain's identity sections were stored but UI-only. cm-runtime/brain.rs
`compose_system` now folds the brain's AGENTS.md ("## How I operate") + personality
into the live system prompt (after the Postgres-authoritative base, before skills +
memory; falls back to the brain's soul_md when the base is empty). Mirrors the
OpenClaw/ZeroClaw render order.
Mapping — expose `agent_md` on `GET /api/claws/{id}/brain` (ClawBrainResponse) and
render it as a collapsible "HOW I OPERATE · AGENTS.md" card in the command center's
BRAIN column (RawBrain gains agent_md; richBrain passes it through). So the section
that's now in the prompt is also visible in the UI.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
10c89f5157 |
Node-placed agent terminal: container PTY on the agent's node + WebRTC, shared node-local drives
Completes "agent on a node" (single-node): when an agent's placement points at a fleet node, its terminal container runs there and the browser reaches it over a direct WebRTC DataChannel (LAN speed), sharing a node-local volume with the sandbox. gw-04-local agents are byte-identical to before. - cm-sandbox/docker.rs: empty drive subpath → mount the whole volume at the target (volume_options None), so a per-agent node-local volume auto-creates at ~/drives. - cm-api/fleet.rs: NodeHub.open_pty/webrtc_offer carry optional container+session (injected only when Some); node-terminal caller passes None (host shell unchanged). - cm-runtime/terminals.rs: TerminalManager gains node_provider + placement (mirrors SandboxManager, draining-aware); node_local_drive_mount(agent) = clawmates_agent_<id> at ~/drives; placement_for() ensures + locates the container; attach uses driver_for(node) (local byte-identical). - cm-runtime/sandboxes.rs: a node-placed agent sandbox mounts the same per-agent volume → shares files with the terminal on that node. - cm-api/routes/terminal.rs: ticket response gains `node`; ws() bridges node-placed agents through the NodeHub relay (WebRTC + fallback) execing into the container; local path unchanged. server main wires with_node_provider. - frontend: agentTerminalConnector mints the ticket then picks WebRTC (node-placed, ⚡ direct / relayed badge) vs WS (local); webrtcConnector generalized to be endpoint-agnostic (node terminal reuses it). Known follow-up: terminal (uid 65532) and sandbox (uid 10001) share the volume but differ in uid — cross-container writes need an aligned uid/gid (group-writable). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
c94784bab2 |
Fleet: actionable executions — rules engine + metrics-aware placement (Phase 2)
Turn the Beszel-tapped metrics into a self-managing loop. - migration node_rules (workspace/node-scoped: metric op threshold, for_seconds, action JSONB, last_fired). - cm-db: repo/node_rules.rs (CRUD + list_enabled); node_metrics::eval_all merges Beszel + heartbeat scalars per node + a headroom() heuristic; nodes::status_of; heartbeat now PRESERVES a `draining` status across heartbeats (so a cordon sticks). - cm-api: node_rules.rs evaluator (spawn_evaluator, 20s) — when a metric condition holds for the rule's window it fires drain / undrain / alert (in-memory sustained + cooldown tracking, modeled on the node sweeper); routes/beszel.rs rules CRUD (GET/POST/PATCH/DELETE /api/fleet/rules); spawned in clawmates-server. - cm-runtime: placement_node() is metrics-aware — a `draining` node stops receiving new agent sandboxes (falls back to local), so the drain rule is actionable. - frontend: FleetRules section in the Local view — build rules (node · metric · op · threshold · duration → action), toggle/delete, with fired-history. The loop: hot/overloaded node → rule drains it → placement avoids it → recovers → undrain rule brings it back. Deployed; node_rules migration applied. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
fb59378aa2 |
Fleet P2b: run agent sandboxes on connected nodes (RemoteDriver + placement)
Agents can now provision their sandbox on a connected fleet node instead of the gateway host. Local stays the strict default, so existing agents are byte-for- byte unaffected until explicitly placed elsewhere. Security parity: the daemon links the REAL cm-sandbox DockerDriver and runs the typed container ops (sb_provision/sb_exec/sb_destroy/sb_health/sb_list) through it — identical hardening (cap-drop ALL, seccomp, no-net, read-only, non-root) to local sandboxes. cm-sandbox spec types are now Serialize/Deserialize so the spec crosses the channel. - cm-api: RemoteDriver (impl SandboxDriver over the node channel) + HubDriverProvider (impl cm_runtime::NodeDriverProvider, hands out a driver only for connected nodes via a sync online set) + NodeHub.call/is_connected. AppState.with_node_hub so the hub is shared with the placement provider. - cm-runtime SandboxManager: driver_for(node_id) routes by the recorded agent_containers.node_id (local default = existing driver, identical path); placement_node() reads the workspace setting and falls back to local if the node is offline; exec/release route accordingly. NodeDriverProvider trait. - DB: 0020_workspace_placement + repo (for_agent/get/set/clear). - main.rs: build the NodeHub first; inject HubDriverProvider into the agent manager + share the hub with AppState. - API+UI: GET/PUT /api/fleet/placement + a "Run agents on: Local / <node>" selector in the Fleet overview. Note: a node must be able to pull the agent image (the daemon docker-pulls it); interactive PTY for agent containers on remote nodes is not wired (Terminal app stays local) — the in-dashboard node shell already covers host access. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
e9ce368ec1 |
Scaling Phase 1: multi-tenant onboarding + replica-safe coordination
Decouples "many users" + "many server replicas" from "many machines" so the platform is tenant-isolated and horizontally safe on the current single node. - Per-signup workspaces (cm-auth): a new hosted-identity sign-in provisions and owns its own workspace instead of joining the first. Config-gated by auth.per_signup_workspace (default off); concurrent first-logins serialized by a per-subject advisory lock so no duplicate workspaces. - Terminal tickets in Postgres (migration 0016, hashed, single-use): any replica can redeem a ticket minted by another. Drops the in-process ticket map. - Container registry in Postgres (migration 0017, agent_containers): Terminal and Sandbox managers resolve an agent's container through a shared registry, so a 2nd replica reuses it instead of spawning a duplicate. node_id recorded as 'local' (Phase 2 hook). Boot reconcile removes only true orphans, so terminals now survive a redeploy (tmux sessions resume). - Per-workspace quotas (cm-api/quota.rs): plan-tier caps on agents + live containers, enforced at agent create + terminal spin-up (reconnects allowed), returned as HTTP 402. New GET /api/quota surfaces usage vs limits. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
e61724ff82 |
Agent computer: terminal (tmux + drives + tabs), Obsidian vault, UI polish
Terminal app (xterm ⇄ WebSocket ⇄ per-agent themed container):
- zsh + oh-my-zsh + powerlevel10k image (agent-terminal), runs as uid 65532 to
share read-write ownership of the file-drive volume with the server.
- Interactive PTY in cm-sandbox (bollard exec tty/attach + resize) + a
TerminalManager; ticket-authed WS bridge routed straight to the backend via a
Traefik PathRegexp(/ws) rule. MOTD greets the user by name.
- tmux resumable sessions; multi-tab (one tmux session per tab, same container),
drag-to-reorder, rename, and a Save that persists named tabs to the server
(terminal_tabs, migration 0014) so they survive logout / a new device.
- Files drives mounted per-agent (subpath) at ~/drives/{documents,received,
shared}; a reconciler keeps the Files app's index in sync with terminal writes.
Storage moved to a shared `filedata` volume (CLAWMATES_STORAGE__DATA_DIR).
Obsidian vault (a markdown "second brain" per agent):
- New `vault` FileDrive (migration 0015) mounted into the terminal at ~/obsidian;
a file-content read route; a purple Obsidian tile + a vault viewer app.
Computer UI:
- Draggable computer-panel width (min = phone preset) keeping the size presets.
- Green Terminal glyph, "Claw Chat" → "Chat", colored gradient-outline app icons.
- Agent page: avatar↔activity-grid spacing + larger, uniform section fonts with
colored section-tinted tag chips.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
34f744734b |
Large World graph, agent platform, brain stack & dashboard rebuild
Frontend - Large World: collapse org/company/team tiers into one expandable React Flow hierarchy (WorldFlow) with per-click expand, persisted node positions, a compact tree sidebar, wrench multi-select delete across levels, and a sized right slide-out (phone/tablet/full) showing an agent summary + drill button. - Agent page: GitHub-style animated contribution grid (VitalsCard), collapsible System Prompt + Personality cards, restructured anatomy cards, bigger avatar with name/title header row, Markdown/JSON-aware rendering, brain registry + history, avatar generate/upload. - User-icon menu (Infrastructure/Brains/Tools/Profile/Credits) + ToolPanel; Master Planner deploy wizard (Specialists/Swarm/Scheduled/Triggered); Team Runs view; reap-progress modal; dashboard is the single live interface. Backend - cm-brain crate (.brain as the agent definition) + brain apply/history. - Hard-purge reap (FK-ordered) + sandbox release + SSE batch-delete. - Swarm self-verifying loop, mode-aware planner, web.search tool, webhooks (migration 0013), org/company/team delete endpoints, scheduler sweeps. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
148705769f |
Reaping: sandbox orphan reaper + DB expiry/retention sweeper + volume-init retry
Closes the cleanup gaps found in review (latent today; bites under load/crashes).
Sandbox containers (cm-sandbox / cm-runtime):
- label every sandbox `clawmates.sandbox={agent|browser}` at create
- SandboxDriver::list_managed(kind) (Docker label filter + K8s label selector)
- SandboxManager::reconcile_orphans(ttl) + spawn_reaper: removes engine
containers no live handle owns (ZERO = all)
- boot reconciliation (every pre-existing sandbox is an orphan from a dead
process) + periodic reaper (5m interval / 10m TTL)
- SIGTERM graceful drain: serve().with_graceful_shutdown → shutdown() both
managers so a redeploy can't leak; destroy errors now logged not swallowed
DB expiry/retention (cm-db cleanup.rs + cm-api cleanup_sweeper, hourly):
- expire auth_sessions + oauth_states past expires_at (security)
- prune sent outbox(7d), run_events(14d), routine_runs(30d), terminal
topology_runs(90d), consumed execution_grants(7d)
Compose: volume-init restart "no" → on-failure:5 (retry instead of wedging boot).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
3402a3b56d |
Door governor: runtime-agent judge (Kimi-as-judge on the subscription)
The §15 governor can now be judged by a ZeroClaw runtime agent instead of a server-side registry/API model. ZeroClawDriveExecutor::judge drives an agent with the governor prompt and parses ALLOW/DENY (fail-open). mcp_door routes to it when CLAWMATES_JUDGE_MODEL=runtime:<alias>. This unblocks Kimi-as-judge with NO Kimi Platform key: Kimi runs on the membership via kimi_cli, so CLAWMATES_JUDGE_MODEL=runtime:judge_kimi makes the coding agent the governor. (Same path works for any subscription-only model.) cm-runtime re-exports judge_model. clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
e8486ee57c |
Door delivery: outbox drainer + SMTP transport (inert until configured)
The §15 door's email_send queues to `outbox` but nothing delivered it. Add a real transport: cm-db outbox repo (list_queued/mark_sent/mark_failed) + a cm-runtime drainer — an EmailSender trait (testable), a lettre STARTTLS LettreSender, drain_once (queued -> sent/failed), and spawn_drainer wired into the server beside the scheduler/sweeper/topology-worker. Config-gated: inert (logs "outbox delivery DISABLED") until CLAWMATES_SMTP_* is set, so it ships safely before credentials exist. The agent never holds the SMTP credential — it only writes to outbox through the gated door; the server owns the transport. NOTE: live delivery is still credential-blocked — Migadu's API can't send (SMTP-only) and the admin token is invalid; no SMTP creds exist. The transport is built + tested (drain_marks_sent_and_failed via a mock sender); set CLAWMATES_SMTP_* to go live with zero further code. clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
8dee01c77b |
Durable topology jobs (1/4): job-table schema + repo
Evolve topology_runs into a durable-job table (migration 0009): status state machine (queued/running/completed/failed/cancelled), kind, input graph, per-step checkpoint, error, last_event_id, timestamps. comparison becomes nullable (the result blob, absent until completion). Back-compat: existing rows default to completed/compare. cm-db repo gains the durable-job ops: enqueue_run, claim_next_queued (CAS via FOR UPDATE SKIP LOCKED), checkpoint, complete, fail, requeue_stale (resume sweep), and status(). Regenerated .sqlx cache. Also fix two pre-existing test RuntimeConfig literals missing the providers field (from the registry work). Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
2ec8832ef6 |
llm: named-provider registry — judges & topology nodes on GLM/Kimi
Adds a multi-provider registry so judges and topology execution can run on
providers beyond the default. cm-config gains [[llm.providers]] (name, base_url,
api_key_env) — all OpenAI-compatible (GLM, Kimi/Moonshot). The server builds an
Arc<dyn LlmProvider> per entry (OpenAiCompatProvider) keyed by name; a missing
key is skipped with a warning, not a boot failure. cm-runtime RuntimeConfig
carries a ProviderRegistry; Runtime::resolve_provider("<name>:<model>") selects a
registry provider (else the default). The door governor (Runtime::judge) and the
topology compare endpoint both resolve through it, so CLAWMATES_JUDGE_MODEL and
CLAWMATES_TOPOLOGY_EXEC_MODEL accept "glm:glm-4.6" / "kimi:kimi-k2".
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
325038e806 |
judges use claude-opus-4-8; everything else stays on sonnet-4-6
Both LLM-as-judge sites — the door governor (Runtime::judge) and the topology comparison scorer (JudgeScorer) — now use the judge model (default claude-opus-4-8, override CLAWMATES_JUDGE_MODEL). Execution/reasoning turns keep using the configured default model (claude-sonnet-4-6). Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
e0e150084d |
door: live governor agent (LLM veto) — self-governing autonomy
Adds Runtime::judge (the configured model returns ALLOW/DENY + reason, fail-open) and wires it into the door policy behind CLAWMATES_DOOR_GOVERNOR: a governor agent judges each outbound action and can veto exfiltration / spam / secret-leakage, atop the deterministic rules. Realizes the self-governing- topology path — authority decided by an agent, not a human, still audited. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
19e9943f79 |
door: richer auto-policy (c) + broker-backed tools (b)
(c) policy_decide is now async with real governance: kill switch + per-workspace hourly rate cap (audit_log count) + email recipient-domain allowlist + a governor extension point. Still allow-all by default (autonomous). (b) the door now supports broker-executed tools: for a broker tool it mints an auto-approved approval + single-use grant (agent->session->run->approval-> decide), then executes via the runtime so the broker consumes the grant and reveals the credential — the agent never holds it. Exposes slack_post. cm-runtime gains tool_broker_executed + tool_preview accessors. 4 door unit tests + clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
87f612016e |
1B: Clawmates §15 MCP door — autonomous gated egress for tool-free agents
Tool-free ZeroClaw agents get one actuator: an MCP server (POST /mcp, JSON-RPC
2.0, protocol 2024-11-05) that fronts the existing §15 machinery. The human
approver is replaced by an automated policy (default allow-all → agents are
autonomous; CLAWMATES_DOOR_POLICY=deny is a kill switch), but the governed parts
stay: every action is journaled to the append-only audit log, actions execute
through the runtime's gated-tool path, and broker credential-custody is wired in
for v2 tools. Synchronous execution returns the real result inline.
- crates/cm-api/src/mcp_door.rs: initialize/tools.list/tools.call handler; auth
(bearer -> workspace), classify effects, policy auto-decide, execute, audit.
v1 exposes email_send (-> outbox); slack/pay (broker+grant chain) is next.
- crates/cm-runtime: Runtime::{tool_descriptor_json, tool_gate_category,
execute_door_tool} — door-facing entry that builds ToolContext and consumes a
grant for runtime-executed gated tools.
- deploy/clawmates-runtime: agents now carry mcp_bundles=["clawmates_door"];
[[mcp.servers]] points at the door (bearer injected at deploy, not committed).
Validated live on gw-04: tools/call email_send -> isError:false + outbox row +
"agent|door.executed" audit, no human. 4 door unit tests + clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
a35c82d860 |
feat(api): POST /api/topologies/compare (provider-backed)
Run a task across topologies via the API and return a leaderboard + quality/cost Pareto front. The handler builds a ProviderExecutor + JudgeScorer over the runtime's configured provider/model, then calls cm_orchestrator::compare. - cm-runtime: expose provider()/model()/max_tokens() accessors on Runtime. - cm-api: depend on cm-orchestrator (provider feature); add the compare route. - Integration test runs a 2-topology comparison through the real server (scripted provider) → 200 with results + leaderboard. 4 topology tests green; offline build + clippy clean. The topology endpoints (catalog/classify/build/compare) ship to gw-04 with the upcoming ReactFlow UI in one server+frontend redeploy. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
5407111a89 |
Live Anthropic validation — and the platform-breaking bug it caught
Running the opt-in live suite (CM_LIVE_LLM=1 + ANTHROPIC_API_KEY) against
the real API immediately surfaced a launch blocker: Anthropic (and
OpenAI) restrict tool names to ^[a-zA-Z0-9_-]{1,128}$ — our ENTIRE
registry uses dotted names (clock.now, email.send, shell.exec, ...).
The scripted provider never enforced the pattern, so every real-model
deployment would have 400'd on the first tool call.
- Fix at the provider boundary, where it belongs: wire_tool_name /
internal_tool_name codec (dots <-> __) applied in BOTH HTTP providers
at all three sites (tools list, assistant tool_use echo, inbound
tool_use decode). Internal naming (DB step rows, scenarios, UI traces)
unchanged. Offline unit test round-trips every registry name through
the wire pattern
- New live tests, all passing against api.anthropic.com (Haiku 4.5):
- provider tool ROUND TRIP: real ToolUse arrives, ToolResult ships
back exactly as a checkpoint would reassemble it, model completes,
real usage events on the wire
- full runtime loop: real model calls clock.now, run completes, REAL
token usage metered, credits decremented
- the #1-risk validation: a real model's email.send intercepted ->
suspended -> approved -> checkpoint RESUMED against the live API ->
completed -> outbox exactly 1 (checkpoint/resume fidelity end to end)
- Stray TC_OPENAI_COMPAT_* envs renamed to CM_OPENAI_COMPAT_*
No credentials stored anywhere; the key was passed via env only.
163 Rust tests (+5 live, key-gated).
Co-Authored-By: Claude Fable 5 <[email protected]>
|
||
|
Omar SobhandClaude Fable 5
|
add4f79fed |
Rebrand: TeamClaw -> Clawmates (clawmates.work)
Full-depth rename per the approved plan; the 'claw' product vocabulary (claws, /claws routes, clawId, Claw Chat) stays — it is now the brand. - Display brand: Clawmates (manifest, titles, hero, login/rail logo 'clawmates'); default host app.clawmates.work; registry ghcr.io/clawmates - Crates tc-* -> cm-* (16 crates + all imports); binaries clawmates-server/broker/bundler; images clawmates/*; env prefix CLAWMATES_* (+ CM_TEST_DATABASE_URL / CM_LIVE_LLM); config clawmates.toml; helm chart deploy/helm/clawmates with clawmates-* resources; db names clawmates*; sockets /run/clawmates; cookie cm_session; kind cluster clawmates-test; seccomp node profile clawmates-agent-profile.json - All 9 Playwright brand assertions updated in lockstep; historical spec document left untouched as the only remaining 'TeamClaw' - Local env migrated: dev pg clawmates-dev-pg/clawmates_dev, shared test server clawmates-test-pg, kind cluster recreated with image + profile, compose images rebuilt under clawmates/* Verified end to end: 161 Rust + 68 frontend tests, 29 Playwright journeys, 4 live kind tests, helm/install/LOC/placeholder gates, and the clean-room install rehearsal serving the clawmates login page from a signed bundle of the rebuilt images. Co-Authored-By: Claude Fable 5 <[email protected]> |