Decouples "many users" + "many server replicas" from "many machines" so the
platform is tenant-isolated and horizontally safe on the current single node.
- Per-signup workspaces (cm-auth): a new hosted-identity sign-in provisions and
owns its own workspace instead of joining the first. Config-gated by
auth.per_signup_workspace (default off); concurrent first-logins serialized by
a per-subject advisory lock so no duplicate workspaces.
- Terminal tickets in Postgres (migration 0016, hashed, single-use): any replica
can redeem a ticket minted by another. Drops the in-process ticket map.
- Container registry in Postgres (migration 0017, agent_containers): Terminal
and Sandbox managers resolve an agent's container through a shared registry,
so a 2nd replica reuses it instead of spawning a duplicate. node_id recorded
as 'local' (Phase 2 hook). Boot reconcile removes only true orphans, so
terminals now survive a redeploy (tmux sessions resume).
- Per-workspace quotas (cm-api/quota.rs): plan-tier caps on agents + live
containers, enforced at agent create + terminal spin-up (reconnects allowed),
returned as HTTP 402. New GET /api/quota surfaces usage vs limits.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Terminal app (xterm ⇄ WebSocket ⇄ per-agent themed container):
- zsh + oh-my-zsh + powerlevel10k image (agent-terminal), runs as uid 65532 to
share read-write ownership of the file-drive volume with the server.
- Interactive PTY in cm-sandbox (bollard exec tty/attach + resize) + a
TerminalManager; ticket-authed WS bridge routed straight to the backend via a
Traefik PathRegexp(/ws) rule. MOTD greets the user by name.
- tmux resumable sessions; multi-tab (one tmux session per tab, same container),
drag-to-reorder, rename, and a Save that persists named tabs to the server
(terminal_tabs, migration 0014) so they survive logout / a new device.
- Files drives mounted per-agent (subpath) at ~/drives/{documents,received,
shared}; a reconciler keeps the Files app's index in sync with terminal writes.
Storage moved to a shared `filedata` volume (CLAWMATES_STORAGE__DATA_DIR).
Obsidian vault (a markdown "second brain" per agent):
- New `vault` FileDrive (migration 0015) mounted into the terminal at ~/obsidian;
a file-content read route; a purple Obsidian tile + a vault viewer app.
Computer UI:
- Draggable computer-panel width (min = phone preset) keeping the size presets.
- Green Terminal glyph, "Claw Chat" → "Chat", colored gradient-outline app icons.
- Agent page: avatar↔activity-grid spacing + larger, uniform section fonts with
colored section-tinted tag chips.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
A team = a baseline topology staffed with real claws. Migration 0010 (teams +
team_members node→claw bindings) + cm-db repo/teams.rs. runtime_provision.rs
turns a claw into a live runtime agent claw_<id> via the synced gateway config
API (#7468): create agent + bind model_provider (mapped from chosen model) +
risk_profile=toolfree + clawmates_door bundle — atomic, immediately drivable.
routes/teams.rs: POST /api/teams (create claws + provision + build(kind,roles)
+ bind node.attrs["agent"]=claw_<id> + persist), GET /api/teams[/{id}],
POST /api/teams/{id}/run (enqueue a durable run of the team graph — reuses the
topology worker + SSE). v1 persona = topology role via the prompt builder; the
claw's system_prompt stays its chat identity.
Spike confirmed: runtime agent provisioning works; IDENTITY.md persona works
for API models (Gemini/Groq), masked by CLI models (Claude/Kimi Code). 16
cm-api tests + provision unit tests pass, clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The §15 governor can now be judged by a ZeroClaw runtime agent instead of a
server-side registry/API model. ZeroClawDriveExecutor::judge drives an agent
with the governor prompt and parses ALLOW/DENY (fail-open). mcp_door routes to
it when CLAWMATES_JUDGE_MODEL=runtime:<alias>.
This unblocks Kimi-as-judge with NO Kimi Platform key: Kimi runs on the
membership via kimi_cli, so CLAWMATES_JUDGE_MODEL=runtime:judge_kimi makes the
coding agent the governor. (Same path works for any subscription-only model.)
cm-runtime re-exports judge_model. clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The §15 door's email_send queues to `outbox` but nothing delivered it. Add a
real transport: cm-db outbox repo (list_queued/mark_sent/mark_failed) + a
cm-runtime drainer — an EmailSender trait (testable), a lettre STARTTLS
LettreSender, drain_once (queued -> sent/failed), and spawn_drainer wired into
the server beside the scheduler/sweeper/topology-worker.
Config-gated: inert (logs "outbox delivery DISABLED") until CLAWMATES_SMTP_*
is set, so it ships safely before credentials exist. The agent never holds the
SMTP credential — it only writes to outbox through the gated door; the server
owns the transport.
NOTE: live delivery is still credential-blocked — Migadu's API can't send
(SMTP-only) and the admin token is invalid; no SMTP creds exist. The transport
is built + tested (drain_marks_sent_and_failed via a mock sender); set
CLAWMATES_SMTP_* to go live with zero further code. clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
#2 GLM judge (closes#114): registry NamedProvider gains a `format` field;
build_provider_registry builds an AnthropicProvider for format="anthropic".
GLM's coding/OpenAI endpoint is ToS-throttled for raw SDK, but its Anthropic
endpoint (api.z.ai/api/anthropic) accepts raw API calls (verified x-api-key
-> glm-4.7), so CLAWMATES_JUDGE_MODEL=glm:glm-4.7 routes the door governor /
topology judge through GLM with no runtime-routing. (Kimi-as-judge still needs
a Platform key — coding key is agent-only.)
#3 run-control: POST /api/topology-runs/{id}/cancel (workspace-scoped,
queued/running only); the worker honors it at the step boundary (checks
current_status in the checkpoint callback) and won't clobber a cancel with
`failed`. Frontend Run tab gains a Cancel button and clickable recent runs
that deep-link into a live/replayed stream (SSE replays from checkpoint).
cm-* tests (incl. new cancel test) + clippy + frontend lint/typecheck green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Backend: GET /api/topology-runs/{id}/events streams Server-Sent Events by
tailing the durable per-step checkpoint the worker already writes — a `step`
event per newly-completed step (replayed on connect so reload/reconnect
re-attaches), then a terminal `done` event with the final output/error. Each
step carries its index as the SSE id, so the browser's Last-Event-ID resumes
without duplicates on reconnect. No new table, no worker change — reuses the
checkpoint; avoids run_events' agent_runs FK.
Frontend: the Run tab now opens an EventSource (through the same-origin proxy,
which adds the bearer) instead of polling — appending steps as they stream and
finalizing on `done`. One streaming connection, lower latency, auto-resume.
cm-api builds + clippy clean; frontend lint + typecheck + next build clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
POST /api/topologies/run now ENQUEUES a durable job and returns 202
{run_id, status:queued} instead of executing inside the HTTP request — the
prerequisite for long-horizon runs (no client/proxy/LB timeout, survives
restarts).
topology_worker: a spawned loop that requeues stale running jobs, claims the
next queued one (CAS via FOR UPDATE SKIP LOCKED), drives it through
execute_resumable, and checkpoints RunProgress after every step; on crash the
stale sweep requeues it and the next claim resumes from the last checkpoint.
Wired into server startup beside the scheduler + resume sweeper.
GET /api/topology-runs/{id} now reports lifecycle status/kind/error/checkpoint
+ the result blob (kept the `comparison` field name for back-compat with the
compare UI; null until completed). list_runs includes status + kind.
Tests: durable lifecycle (enqueue→claim→checkpoint→complete) + stale-requeue
resume, both green; p0 endpoints (compare path) unchanged. 13 + 2 tests pass,
clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Split execute() into a thin wrapper over new execute_resumable(), which starts
from a prior RunProgress checkpoint (completed step count + outputs + records +
metrics) and invokes an async on_step callback after each newly completed step.
The step plan is re-derived from the graph (planners are deterministic), so only
completed outputs need persisting; the callback owns persistence, keeping the
orchestrator storage-agnostic. RunProgress + the journal types (StepRecord,
RunMetrics, StepPhase, GatedAction) gain Deserialize for JSONB round-trip.
New test: resume-from-checkpoint runs only the remaining steps and reproduces
the full run's output. 14 tests pass, clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Evolve topology_runs into a durable-job table (migration 0009): status state
machine (queued/running/completed/failed/cancelled), kind, input graph,
per-step checkpoint, error, last_event_id, timestamps. comparison becomes
nullable (the result blob, absent until completion). Back-compat: existing
rows default to completed/compare.
cm-db repo gains the durable-job ops: enqueue_run, claim_next_queued (CAS via
FOR UPDATE SKIP LOCKED), checkpoint, complete, fail, requeue_stale (resume
sweep), and status(). Regenerated .sqlx cache. Also fix two pre-existing test
RuntimeConfig literals missing the providers field (from the registry work).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
TurnRequest gains an optional `agent` sourced from the graph node's
attrs["agent"]. The ZeroClaw executor binds a node to that alias directly
when present, falling back to the role→alias map otherwise. This lets a
single POST /api/topologies/run specify a different model per role
(heterogeneous topologies) entirely in the graph JSON — no server
ZEROCLAW_AGENT_MAP change or recreate per configuration, which makes
quota-frugal model×role sweeps practical.
cm-orchestrator 13 + cm-api topology_exec 4 tests pass, clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Adds a multi-provider registry so judges and topology execution can run on
providers beyond the default. cm-config gains [[llm.providers]] (name, base_url,
api_key_env) — all OpenAI-compatible (GLM, Kimi/Moonshot). The server builds an
Arc<dyn LlmProvider> per entry (OpenAiCompatProvider) keyed by name; a missing
key is skipped with a warning, not a boot failure. cm-runtime RuntimeConfig
carries a ProviderRegistry; Runtime::resolve_provider("<name>:<model>") selects a
registry provider (else the default). The door governor (Runtime::judge) and the
topology compare endpoint both resolve through it, so CLAWMATES_JUDGE_MODEL and
CLAWMATES_TOPOLOGY_EXEC_MODEL accept "glm:glm-4.6" / "kimi:kimi-k2".
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Both LLM-as-judge sites — the door governor (Runtime::judge) and the topology
comparison scorer (JudgeScorer) — now use the judge model (default
claude-opus-4-8, override CLAWMATES_JUDGE_MODEL). Execution/reasoning turns keep
using the configured default model (claude-sonnet-4-6).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Adds Runtime::judge (the configured model returns ALLOW/DENY + reason,
fail-open) and wires it into the door policy behind CLAWMATES_DOOR_GOVERNOR: a
governor agent judges each outbound action and can veto exfiltration / spam /
secret-leakage, atop the deterministic rules. Realizes the self-governing-
topology path — authority decided by an agent, not a human, still audited.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
(c) policy_decide is now async with real governance: kill switch +
per-workspace hourly rate cap (audit_log count) + email recipient-domain
allowlist + a governor extension point. Still allow-all by default (autonomous).
(b) the door now supports broker-executed tools: for a broker tool it mints an
auto-approved approval + single-use grant (agent->session->run->approval->
decide), then executes via the runtime so the broker consumes the grant and
reveals the credential — the agent never holds it. Exposes slack_post.
cm-runtime gains tool_broker_executed + tool_preview accessors.
4 door unit tests + clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Tool-free ZeroClaw agents get one actuator: an MCP server (POST /mcp, JSON-RPC
2.0, protocol 2024-11-05) that fronts the existing §15 machinery. The human
approver is replaced by an automated policy (default allow-all → agents are
autonomous; CLAWMATES_DOOR_POLICY=deny is a kill switch), but the governed parts
stay: every action is journaled to the append-only audit log, actions execute
through the runtime's gated-tool path, and broker credential-custody is wired in
for v2 tools. Synchronous execution returns the real result inline.
- crates/cm-api/src/mcp_door.rs: initialize/tools.list/tools.call handler; auth
(bearer -> workspace), classify effects, policy auto-decide, execute, audit.
v1 exposes email_send (-> outbox); slack/pay (broker+grant chain) is next.
- crates/cm-runtime: Runtime::{tool_descriptor_json, tool_gate_category,
execute_door_tool} — door-facing entry that builds ToolContext and consumes a
grant for runtime-executed gated tools.
- deploy/clawmates-runtime: agents now carry mcp_bundles=["clawmates_door"];
[[mcp.servers]] points at the door (bearer injected at deploy, not committed).
Validated live on gw-04: tools/call email_send -> isError:false + outbox row +
"agent|door.executed" audit, no human. 4 door unit tests + clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
ZeroClaw pairing codes are single-use, so a static ZEROCLAW_PAIRING_CODE only
works for the first run. Add ZEROCLAW_TOKEN: pair once out-of-band, set the
durable bearer, and runs are repeatable. PAIRING_CODE stays as a fallback; one
of the two is required.
Validated live on gw-04: a single-node pipeline driven through the deployed
authed POST /api/topologies/run drove a real ZeroClaw role-agent over /ws/chat
and returned a RunRecord with real output + token metering, persisted to
topology_runs.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
cm-orchestrator owns the topology graph; each turn now drives a real ZeroClaw
role-agent in a container via the proven gateway drive recipe, instead of a
tool-free cm-llm call.
- crates/cm-api/src/topology_exec.rs: ZeroClawDriveExecutor impl TurnExecutor —
pair (POST /pair + X-Pairing-Code, token cached) -> ws /ws/chat?agent=<alias>
-> send {type:message,content} -> drain chunk/done/approval_request/error.
approval_request is recorded as a BLOCKED GatedAction, never auto-approved (§15).
Role->alias via ZEROCLAW_AGENT_MAP, fallback ZEROCLAW_DEFAULT_AGENT (scout).
- POST /api/topologies/run {task,graph} -> execute() -> RunRecord, persisted
best-effort to the existing topology_runs table (no migration). compare stays
tool-free. from_env() is read in-handler so cm-api still boots unset.
- deploy/clawmates-runtime: example config now declares a tool-free multi-agent
role-cast; README documents the ZEROCLAW_* knobs + run endpoint.
tokio-tungstenite 0.26 (already in lock) + dev axum `ws` for the hermetic test.
3 lib tests green, clippy clean, SQLX_OFFLINE build clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Save each comparison and let users reload past ones.
- migration 0008: topology_runs (workspace-scoped; full comparison as JSONB).
- cm-db repo::topology_runs (insert / list_recent / get) + regenerated .sqlx.
- cm-api: compare persists best-effort (never loses the LLM result on a DB
hiccup); GET /api/topology-runs (recent) + GET /api/topology-runs/{id}.
Integration test asserts persist → list → get.
- frontend: "Recent comparisons" list on the Compare tab; click to reload a
saved run. e2e p8 green (39 suite); offline build + clippy clean.
Server self-migrates at boot (cm_db::MIGRATOR), so 0008 applies on deploy.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Run a task across topologies via the API and return a leaderboard + quality/cost
Pareto front. The handler builds a ProviderExecutor + JudgeScorer over the
runtime's configured provider/model, then calls cm_orchestrator::compare.
- cm-runtime: expose provider()/model()/max_tokens() accessors on Runtime.
- cm-api: depend on cm-orchestrator (provider feature); add the compare route.
- Integration test runs a 2-topology comparison through the real server
(scripted provider) → 200 with results + leaderboard. 4 topology tests green;
offline build + clippy clean.
The topology endpoints (catalog/classify/build/compare) ship to gw-04 with the
upcoming ReactFlow UI in one server+frontend redeploy.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Stateless, auth-gated routes backing the topology builder UI:
- GET /api/topologies — catalog of the 12 kinds + descriptions + role mix
- POST /api/topologies/classify — infer a kind + metrics from a posted graph
- POST /api/topologies/build — build a canonical graph from {kind, roles}
Add Serialize to cm-topology Classification/GraphMetrics; add ApiError::BadRequest
(400) for invalid build input; add cm-topology dep. 3 integration tests
(catalog/build/auth) green against Postgres; offline build + clippy clean.
No DB or provider yet — running/comparing topologies is a later provider-backed
endpoint. Server redeploy will batch with the ReactFlow UI that consumes these.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
cm-topology::build(kind, roles) instantiates a canonical graph for any of the 12
kinds from a role list (the template catalog + the search generator).
cm-orchestrator::evolve searches a (kinds × team-size) grid using the comparison
machinery as fitness: build a candidate per cell, run the task, score it, and
keep a MAP-Elites-style archive of per-cell elites + a quality/cost Pareto front
and the global best. evolve_all() covers every kind at full size. This is the
bridge toward Autonomous Organizational Evolution on a safe substrate — every
candidate still executes via safe turns (§15 invariant holds).
Demoed in topology_bench (auto-picks the best topology + Pareto kinds).
cm-topology 20 tests; cm-orchestrator 17 (--features provider); clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Milestone-A second capability: chain whole topology runs in sequence, threading
each stage's final output into the next stage's task (e.g. swarm brainstorm →
hierarchical execute → debate review). Each stage is a full safe topology run,
so §15 holds at every step. WorkflowRecord aggregates per-stage RunRecords +
totals. Demonstrated in the benchmark example. 14 tests; clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Every TopologyKind now runs, mapped to five execution patterns:
- hierarchical ← hub_spoke, star_moe, market
- pipeline ← ring
- swarm ← flat, holacratic
- mesh (new) ← blackboard (two peer-exchange rounds + aggregate)
- debate (new) (propose → critique → revise → judge)
execute()'s match is now exhaustive (adding a kind upstream forces an executor),
so the Unsupported error is gone. Benchmark spans all five distinct patterns.
14 tests with --features provider; clippy clean. Doc updated.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
`cargo run -p cm-orchestrator --example topology_bench --features provider`
runs one task across hierarchical/pipeline/swarm topologies and prints a
leaderboard + quality/cost Pareto front. Offline-deterministic via the scripted
provider; set ANTHROPIC_API_KEY to run against a real model. Demonstrates
milestone B end-to-end and seeds the reproducible paper harness.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Make Scorer async and add JudgeScorer (behind the `provider` feature): asks a
cm-llm model to rate a run's output 0-100 vs the task and normalizes to [0,1],
giving the comparison harness real quality numbers. Robust integer parsing
(handles "Score: 92/100", clamps >100); provider errors score 0.0.
11 tests with --features provider (judge incl. parse + scripted-provider score);
core stays 7. Clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
compare(graphs, task, executor, scorer) runs the same task across a set of
topologies on the same executor, scores each, and returns a Comparison:
- per-topology results (quality, tokens, turns, blocked approvals, output),
- a leaderboard (quality desc),
- a quality/cost Pareto front (on_pareto flags),
- best_quality and best_value (quality-per-token) picks.
This is the "which patterns yield better results" engine and the structured
output the paper's benchmark tables consume. Generic over TurnExecutor +
Scorer (pluggable LLM-judge later); pure core, no new deps. 9 tests, clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Add a real TurnExecutor (behind the `provider` feature) that runs each
topology turn as a single tool-free LLM call via any cm-llm provider
(Anthropic / OpenAI-compat / scripted). Topologies now execute real model
calls and produce real metrics (tokens), feeding the comparison harness.
- ProviderExecutor builds a per-role system prompt + threads upstream context
into the user message; collects TextDelta → output, Usage → tokens.
- Tool-free reasoning turns take no sandbox-leaving actions (gated = []); §15
remains satisfied. Tool-using turns will route through a cm-runtime adapter.
- Core crate stays dependency-light; cm-llm/futures are optional (feature).
- Tests: pipeline + hierarchical run over the deterministic scripted provider
(7 tests with --features provider). Clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A pure async control-flow engine that executes a task across a TopologyGraph
by sequencing safe agent turns. Safety by construction: the engine can only
invoke turns via a generic TurnExecutor — it performs no side effects itself,
so §15 gating (inside each turn) is inherited and switching topology cannot
escalate authority.
- TurnExecutor trait + TurnRequest/TurnOutcome (real impl will wrap
cm-runtime::Runtime; tests use a scripted Echo executor).
- Pure planners (plan.rs): hierarchical (delegate down / synthesize up),
pipeline (topo-ordered threading), swarm (parallel attempts + aggregate).
- RunRecord journal (per-step + RunMetrics: tokens, gated actions, approvals
granted/blocked, turns) — feeds the Phase 4 comparison harness/paper.
- Unsupported kinds return an error (no panic). 5 tests, clippy clean.
Next (Phase 2b): a real TurnExecutor adapter over cm-runtime::send_message.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
On a user's first login the authed shell fires several API calls at once;
each ran the JIT-provision path and raced to INSERT the same new user row,
tripping the partial unique index on auth_subject. The losing requests
500'd and the post-login SSR errored out. Use INSERT ... ON CONFLICT
(auth_subject) DO UPDATE ... RETURNING so concurrent callers converge on
the row the winner created. Regenerated the .sqlx offline query cache.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
- GET /api/team/orgchart — each member grouped with the claws they manage
(agents.managed_by); GET /api/team/leaderboard — every claw ranked by
its real usage_events rollup (credits/tokens/runs, zeros included).
Both tested against real Postgres.
- Stripe Buy-credits (the Slack/Clerk integration pattern): [billing]
config (stripe keys + price + webhook secret + credits_per_pack);
POST /api/credits/checkout opens a real Checkout Session; POST
/api/billing/stripe verifies Stripe's t=,v1= HMAC (constant-time) and
grants one credit lot, idempotent on the session id; GET
/api/billing/config gates the button (honest degradation when unset).
Offline tests: signed grant + replay no-double-grant + forged-sig 400 +
config flag. Live checkout creation deferred to a CM_LIVE_STRIPE test.
- /apps global page support: clawId now optional on connect + directory;
absent => workspace-wide connection (app_connections.agent_id NULL) via
new connections::list_for_workspace.
- ApiError gains a From<sqlx::Error> so inline queries use ? cleanly.
cm-api 10 test files incl. team_tabs (2) + stripe_billing (3); clippy clean.
Co-Authored-By: Claude Fable 5 <[email protected]>
Made the Docker Compose route turn-key for a real self-host, then stood
the whole stack up and drove a live chat through it.
- First-owner bootstrap (cm-auth::bootstrap_owner): a fresh local-auth
install has no users and no signup route, so the initial Owner +
workspace are provisioned ONCE from CLAWMATES_BOOTSTRAP_* env on first
boot — idempotent, never clobbers an existing install (keys on 'any
workspace exists'). Two real-Postgres tests (creates + signs in;
second call is a no-op). Wired into server boot, guarded on a
non-empty password
- deploy/compose/README.md: full production bring-up — services, the
security topology, every config knob, Anthropic vs local-LLM, the
broker-key backup, ops, and TLS/SSE proxy notes
- .env.example fleshed out (bootstrap, LLM, auth mode, OTLP); compose
uses optional env_file so only the knobs you set are injected (unset
options never override clawmates.toml with empty strings)
- volume-init one-shot chowns the broker's named volumes so the non-root
scratch broker can write its socket + generated master key
Deployed locally and verified end to end: all 5 containers healthy,
broker generated its key, server bootstrapped owner@…, login + /api/user/me
work, and a real message streamed a live Anthropic response through the
gateway. Captured screenshots of login, workspace home, chat, and the
Computer panel.
166 Rust tests (+2 bootstrap).
Co-Authored-By: Claude Fable 5 <[email protected]>
Last open item from the roadmap + post-1.0 list. Run against the live
Clerk instance closing-seasnail-39.clerk.accounts.dev.
- Backend (crates/cm-auth/tests/live_clerk.rs, CM_LIVE_CLERK=1): pulls
REAL discovery + JWKS from the live instance, mints a REAL session JWT
via Clerk's Backend API (create user -> open session -> session token),
and runs it through AuthService::authenticate — verify + JIT provision
(keyed on the real sub), duplicate-subject suppression, tamper
rejection against the live JWKS. Decodes the instance domain from the
publishable key; cleans up the test user after. PASSING
- Frontend: built with AUTH_MODE=clerk + real keys, next start serves
Clerk's <SignIn /> at /login wired to the instance (instance domain +
data-clerk attributes present in the HTML). Both halves confirmed
end to end against production Clerk
- docs/clerk.md: documented the smoke procedure for both halves
166 Rust tests (+6 live, key-gated). Keys used via env only, never
stored — rotate them (they passed through chat).
Co-Authored-By: Claude Fable 5 <[email protected]>
- scripts/netpol-cluster.sh: a kind cluster with Calico (default CNI
disabled) — the only way to PROVE the §15 default-deny NetworkPolicy,
since kindnet accepts the object but never enforces it. New live test
on that cluster: outbound connect to 1.1.1.1 dropped, DNS egress
dropped, while API-server exec keeps working (not pod network).
Kernel-level enforcement of the sandbox egress claim, demonstrated
- K8sDriver::connect_with_context: pin a kubeconfig context instead of
ambient. The whole k8s suite now pins its cluster explicitly — the
netpol cluster's creation had silently switched the current context
and stranded the seccomp test on the wrong cluster (fixed and made
impossible to recur)
- CI: netpol-cluster up + calico egress test in the sandbox-k8s job
- Visual-regression lock (plan P6): @visual Playwright spec with
animation-disabled, masked-dynamic-region screenshots of login,
workspace home, chat welcome, computer panel, credits; darwin
baselines committed (5 PNGs); CI excludes @visual until linux
baselines are generated there. Full local suite: 33 journeys
165 Rust tests + 5 live kind tests (2 clusters) + 33 journeys.
Co-Authored-By: Claude Fable 5 <[email protected]>
Running the opt-in live suite (CM_LIVE_LLM=1 + ANTHROPIC_API_KEY) against
the real API immediately surfaced a launch blocker: Anthropic (and
OpenAI) restrict tool names to ^[a-zA-Z0-9_-]{1,128}$ — our ENTIRE
registry uses dotted names (clock.now, email.send, shell.exec, ...).
The scripted provider never enforced the pattern, so every real-model
deployment would have 400'd on the first tool call.
- Fix at the provider boundary, where it belongs: wire_tool_name /
internal_tool_name codec (dots <-> __) applied in BOTH HTTP providers
at all three sites (tools list, assistant tool_use echo, inbound
tool_use decode). Internal naming (DB step rows, scenarios, UI traces)
unchanged. Offline unit test round-trips every registry name through
the wire pattern
- New live tests, all passing against api.anthropic.com (Haiku 4.5):
- provider tool ROUND TRIP: real ToolUse arrives, ToolResult ships
back exactly as a checkpoint would reassemble it, model completes,
real usage events on the wire
- full runtime loop: real model calls clock.now, run completes, REAL
token usage metered, credits decremented
- the #1-risk validation: a real model's email.send intercepted ->
suspended -> approved -> checkpoint RESUMED against the live API ->
completed -> outbox exactly 1 (checkpoint/resume fidelity end to end)
- Stray TC_OPENAI_COMPAT_* envs renamed to CM_OPENAI_COMPAT_*
No credentials stored anywhere; the key was passed via env only.
163 Rust tests (+5 live, key-gated).
Co-Authored-By: Claude Fable 5 <[email protected]>
Full-depth rename per the approved plan; the 'claw' product vocabulary
(claws, /claws routes, clawId, Claw Chat) stays — it is now the brand.
- Display brand: Clawmates (manifest, titles, hero, login/rail logo
'clawmates'); default host app.clawmates.work; registry
ghcr.io/clawmates
- Crates tc-* -> cm-* (16 crates + all imports); binaries
clawmates-server/broker/bundler; images clawmates/*; env prefix
CLAWMATES_* (+ CM_TEST_DATABASE_URL / CM_LIVE_LLM); config
clawmates.toml; helm chart deploy/helm/clawmates with clawmates-*
resources; db names clawmates*; sockets /run/clawmates; cookie
cm_session; kind cluster clawmates-test; seccomp node profile
clawmates-agent-profile.json
- All 9 Playwright brand assertions updated in lockstep; historical
spec document left untouched as the only remaining 'TeamClaw'
- Local env migrated: dev pg clawmates-dev-pg/clawmates_dev, shared
test server clawmates-test-pg, kind cluster recreated with image +
profile, compose images rebuilt under clawmates/*
Verified end to end: 161 Rust + 68 frontend tests, 29 Playwright
journeys, 4 live kind tests, helm/install/LOC/placeholder gates, and
the clean-room install rehearsal serving the clawmates login page from
a signed bundle of the rebuilt images.
Co-Authored-By: Claude Fable 5 <[email protected]>
- tc-telemetry: fmt subscriber always; with [telemetry] otlp_endpoint
set, spans batch-export over OTLP/HTTP. Tested against a REAL OTLP
receiver decoding the actual protobuf (official proto types): the
emitted span and service.name arrive on the wire. No endpoint = no
export = no network (air-gap stance). tower-http TraceLayer gives
every API request a span
- The broker finally has its own image (images/broker.Dockerfile,
9.5MB from scratch) — the Helm chart referenced one that never
existed — and the compose deployment now RUNS the broker, sharing a
socket volume with the server (the unix-socket equivalent of the K8s
sidecar). Compose secret flows were silently dead before this
- server.Dockerfile fixes surfaced by the rehearsal: the workspace
build needs tools/ (bundler joined the workspace) and
images/seccomp/ (include_str! profile) in the build context
- scripts/rehearse-install.sh (plan: clean-VM rehearsal): assembles a
REAL signed bundle from the built images (server/frontend/broker/
postgres/socket-proxy), runs the customer path — offline verify,
docker load, compose up — and asserts /healthz plus the served login
page before teardown. Passing locally; wired as a release.yml step,
which also builds/ships the broker + socket-proxy images now
161 Rust tests + 29 journeys; clean-room rehearsal green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- K8sDriver.with_localhost_seccomp(profile): sandbox pods run under the
STRICT allowlist instead of the runtime default. Proven live on kind:
the harness installs the profile onto the node, a pod runs ordinary
work as uid 10001, and unshare is kernel-denied inside the pod — the
same probe the Docker suite uses, now passing on both targets
- Helm: sandbox.seccomp=localhost renders a DaemonSet that installs the
chart-shipped profile into /var/lib/kubelet/seccomp on every node
(ConfigMap + hostPath); ci/check-helm.sh enforces the chart copy stays
byte-identical to images/seccomp/agent-profile.json and asserts the
hardened render (DaemonSet + profile + HPA)
- server HPA (autoscaling/v2, CPU target) behind
server.autoscaling.enabled
- SandboxManager.warm(n): a background warmer keeps n pre-provisioned
sandboxes ready so an agent's first exec skips container startup;
unhealthy pool entries are discarded, reuse never drains the pool,
shutdown destroys assigned AND pooled. [sandbox] warm_pool config
(default 0). Real-Docker test: prefill -> assign -> refill -> reuse ->
clean shutdown
160 Rust tests + 4 live kind tests.
Co-Authored-By: Claude Fable 5 <[email protected]>
- The compose server NEVER sees the raw Docker socket (plan risk #5):
tecnativa/docker-socket-proxy on an isolated engine_net with exactly
CONTAINERS/POST/EXEC/DELETE/VERSION allowed; server reaches it via
DOCKER_HOST. DockerDriver honors DOCKER_HOST (connect_to). Proven by a
REAL proxy test: full sandbox lifecycle works through the allowlist
while /networks, /secrets, and /images all 403 — the blast-radius cap
if the server is ever owned. (This also fixes compose deployments,
where sandbox provisioning previously had no engine access at all.)
- Gateway load test (plan P6): 40 concurrent SSE streams against one
server — every run completes with the full §13 event vocabulary,
every journal strictly monotonic, every resumeFrom=0 replay byte-equal
to its live stream
- release.yml: SBOMs (syft, spdx-json) for all four images shipped
INSIDE the signed bundle; final verification now runs in a
--network none container — proving the customer's verify path needs
no internet, not just claiming it
159 Rust tests.
Co-Authored-By: Claude Fable 5 <[email protected]>
- tc-auth JwtVerifier: OIDC discovery -> JWKS, RS256 with the issuer
pinned, 5s leeway (the crate's default 60s would double the life of
Clerk's 60s session tokens), key cache with one refresh on unknown kid
(Clerk rotates). Serves auth.mode = clerk AND generic oidc — a Clerk
instance IS an OIDC issuer, so one verifier covers both
- AuthService.authenticate dispatches: JWT-shaped bearers take the
hosted-identity path, everything else stays a local opaque session.
External users JIT-provision keyed by the stable sub claim
(users.auth_subject, unique partial index in migration 0007); an
existing local account with the same email is LINKED, not duplicated;
role tracks the issuer claim every request (org:admin -> Owner)
- Config auth.mode = "clerk" (requires issuer_url; validated), server
pins the issuer at boot, Helm values/configmap accept mode=clerk
- Tests with REAL crypto, no mocks: fresh RSA keypairs, a live local
issuer publishing real discovery + JWKS docs, Clerk-shaped tokens —
JIT + role mapping, repeat-subject no-dup, expired refused (leeway
regression), wrong-key forgery refused, foreign issuer refused, and
the full router round trip with Authorization: Bearer <session JWT>
- docs/clerk.md: dashboard session-token customization (email + org
role claims), config, @clerk/nextjs getToken() wiring, what CI proves
157 Rust + 63 frontend tests + 29 journeys. Air-gapped installs keep
local auth — Clerk is a cloud-only alternative, not a replacement.
Co-Authored-By: Claude Fable 5 <[email protected]>
- SandboxSpec gains an egress flag (default false — the kernel suite
still proves zero-network for agent sandboxes). Egress-enabled
containers exist ONLY for the browser: no credentials, no broker
route, bridge network with host-gateway alias for local test pages
- images/agent-browser: Alpine Chromium, uid 10001, setuid bits
stripped — same non-root hardening as agent-base
- browser.goto tool: headless chromium --dump-dom in the agent's
browser container; HTML stripped to readable text (4k cap) and
returned with output_taint=web; viewport screenshot captured,
base64'd out of the container, stored in the blob store
- Taint semantics tightened: the step that PRODUCED untrusted output
now carries its own taint (recorded before the step row), not just
later steps — chat.inbox test updated to the stricter §15 reading
- GET /api/claws/{id}/browser/viewport.png serves the latest capture;
BrowserApp polls it and renders the live viewport (spec §7.1),
keeping the empty state until the agent has browsed
- Proven end to end with REAL Chromium against a REAL local page:
content 'Revenue up 14 percent' returned tainted web; the gated
email.send that follows carries 'web' in its approval taint_sources
(untrusted content can never quietly reach outward); screenshot
verified by PNG magic bytes
152 Rust tests + 63 frontend + 27 Playwright journeys.
Co-Authored-By: Claude Fable 5 <[email protected]>
- SandboxManager (tc-runtime): one container per agent, provisioned
lazily on first use, reused for the manager's lifetime, replaced
transparently if dead, destroyed on shutdown
- shell.exec tool: sh -lc inside the agent's sandbox; stdout/stderr/
exit_code return to the model as the step output. No external effects
declared — the sandbox boundary (uid 10001, no caps, seccomp
allowlist, read-only rootfs, zero egress) is the §15 control here,
not an approval gate
- RuntimeConfig.sandboxes (+ with_sandboxes builder); [sandbox] config
{image, enabled}; the server connects the Docker driver at boot and
tolerates an absent engine (shell.exec reports it per-call)
- Tests with the REAL DockerDriver: a scripted run executes two
commands — output proves uid 10001 from inside, and /home/agent state
written by the first call is read by the second (same sandbox); a
deployment without a sandbox runtime records honest error steps and
the run still completes
151 Rust tests + 27 Playwright journeys.
Co-Authored-By: Claude Fable 5 <[email protected]>
- images/seccomp/agent-profile.json is now a TRUE ALLOWLIST: Docker's
default profile (vendored from moby v27.5.1, defaultAction ERRNO) with
18 syscalls an agent never needs stripped from the allow groups
(unshare, ptrace, bpf, mount family, setns, module loading,
perf_event_open, process_vm_*, reboot, quotactl, ...); arch map trimmed
to x86_64 + aarch64. All 6 Docker kernel assertions still green.
- K8sDriver (tc-sandbox feature 'k8s', kube-rs): one hardened pod per
sandbox — runAsUser 10001, cap-drop ALL, no-new-privs via
allowPrivilegeEscalation=false, RuntimeDefault seccomp, read-only
rootfs with emptyDir /tmp + /home/agent, resource limits, no service
account token — in a PSS-restricted namespace carrying a default-deny
NetworkPolicy (applied server-side apply, idempotent). Exec via the API
server attach channel with exit codes parsed from v1.Status.
- Live suite (feature 'k8s-tests') against a REAL kind cluster: uid /
CapEff==0 / NoNewPrivs / rootfs probes from inside pods, PSS label +
deny-all policy asserted via the API, lifecycle. Honest limits in the
rustdoc: Localhost seccomp profile and CNI-enforced egress are
per-cluster provisioning (kindnet does not enforce NetworkPolicy).
- rustls 0.23 process provider pinned to ring at driver connect.
- CI: dedicated sandbox-k8s job (helm/kind-action) running the suite.
149 Rust tests + 3 live kind tests; clippy clean including the k8s feature.
Co-Authored-By: Claude Fable 5 <[email protected]>
- S3BlobStore (object_store, path-style) behind the same BlobStore trait,
tested against a REAL MinIO container (round trip, overwrite, NotFound
on get and delete, nested keys); [storage] backend=local|s3 config with
validation + server-side selection (S3 creds via env overlay)
- Helm chart: server pod with the secret broker as a SIDECAR sharing a
private emptyDir unix socket (no network hop carries credentials),
frontend, optional local PVC vs S3, OIDC/oauth values, unbuffered-SSE
ingress annotations, NetworkPolicies (frontend->server only), hardened
securityContexts; ci/check-helm.sh lints AND asserts the rendered
topology properties
- deploy/airgapped/install.sh: offline signature+checksum verification via
the bundled teamclaw-bundler BEFORE any docker load; --verify-only mode;
ci/test-install.sh rehearses clean/tampered/wrong-key paths with the
real binary
- CI: helm gate + installer rehearsal wired in
149 Rust tests; helm lint + rendered assertions green; installer
verify-path rehearsal green.
Co-Authored-By: Claude Fable 5 <[email protected]>
- Concurrency soak (exit criterion): 12 concurrent gated runs, every
decision attempted twice concurrently, explicit resumes racing the
durable sweeper — exactly one execution per approval, grants consumed
at most once, every decision audited, zero stuck runs, zero unaudited
executions. (Testkit pool raised to 20 connections; the 5-connection
pool starved the storm.)
- axe a11y sweep (exit criterion): serious+critical violations fail CI on
login, shell, chat, computer home, settings app, all global pages, and
the wizard. Two real violations found and fixed: aria-label on a plain
div (wizard progress -> role=group) and a button directly inside a <dl>
(settings -> plain bordered list).
- tools/bundler (exit criterion): keygen / assemble / verify CLI — copies
artifacts, writes manifest.json + sha256 checksums.txt + a detached
ed25519 signature; verification is fully offline (keyless signing is
internet-dependent and disqualified). Tests: round trip, tampered
artifact caught by hash, tampered checksum list caught by signature,
wrong key refused, missing artifact reported.
147 Rust + 63 frontend tests + 27 Playwright journeys (incl. 4 a11y).
Co-Authored-By: Claude Fable 5 <[email protected]>
