573956e84071ac5e70a55b0e437f47a39cd3d563
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar Sobh
|
21ac35c8d4 |
research: spawn per-topic ZeroClaw team container on start (commit 1/3)
Commit 1 of the path-B (real per-topic isolation) plan. The
container spawns and its coordinates persist — nothing talks to
it yet; commit 2 wires ZeroClawDriveExecutor to prefer the topic's
URL when populated. This split keeps each landing verifiable.
Backend
- Migration 0038: research_topics gets zeroclaw_container_name +
zeroclaw_gateway_url columns. Both nullable so a topic can exist
before a spawn and teardown just NULLs them out.
- cm-db: ResearchTopic struct extended; get/list SELECTs updated;
new set_zeroclaw_container(id, workspace_id, name, url) helper
used both for spawn (Some/Some) and teardown (None/None).
- cm-api: bollard added as a workspace dep (matches cm-sandbox's
version). New research_container module:
· connect() → uses DOCKER_HOST when set (prod's socket-proxy
at tcp://socket-proxy:2375) else the local socket. Same
pattern cm-sandbox already uses.
· container_name_for(topic_id) → "research-<uuid>-team"
(deterministic so a re-start reattaches to the same
container instead of orphaning it).
· inherited_env() → propagates ZEROCLAW_*, OPENAI_*,
ANTHROPIC_*, GEMINI_*, GROQ_* from the parent server env
(provider config + tokens), stripping the server's own
ZEROCLAW_GATEWAY_URL/WORKSPACE so the team runtime doesn't
loop back on itself. Appends ZEROCLAW_GATEWAY_PORT=42617
and ZEROCLAW_WORKSPACE=/zeroclaw-data/workspace for the
team's own listener.
· spawn(docker, topic_id, repo_host_path, state_host_path):
- inspect: if the container already exists, start it if
stopped and return its coordinates (idempotent restart).
- else create with:
image = CLAWMATES_RESEARCH_TEAM_IMAGE or
clawmates-runtime:latest
cmd = [daemon, --host, 0.0.0.0]
env = inherited_env()
mounts = repo_host_path → /workspace/repo (rw)
state_host_path → /zeroclaw-data (rw)
network = CLAWMATES_RESEARCH_TEAM_NETWORK or
clawmates_core
labels = clawmates.role=research-team,
clawmates.research.topic_id=<uuid>
- creates state_host_path first so bind doesn't ENOENT.
· stop(docker, name) → stop + remove. Idempotent on 404/304.
- start_topic wires spawn after the clone completes:
· state root = CLAWMATES_RESEARCH_WORKSPACE_ROOT / <topic> /
state
· on success, persists (name, url) on the topic row so commit
2 can look them up when constructing the executor
· every failure (docker connect, docker create/start, DB
persist) is best-effort: logs and continues. A missing team
container leaves the topic pointing at the workspace-wide
gateway URL (env), preserving prior behavior.
Deploy prerequisites (not in this commit)
- The compose stack's clawmates_server service needs bind-mounts
of CLAWMATES_RESEARCH_WORKSPACE_ROOT (e.g.
/var/lib/clawmates-research:/var/lib/clawmates-research) so
paths the server writes to are visible on the host and the
spawned team container mounts the same underlying data.
- socket-proxy's ACL must allow POST + DELETE on /containers
(already the case in prod per the audited compose file).
|
||
|
Omar Sobh
|
4b48c521eb |
loops: backend routes + repo + cron scheduler + HMAC webhook
Third commit of the Research + Loops arc. Lights up loops as durable
recurring topology executions:
GET /api/loops list workspace's loops
POST /api/loops create — returns webhook_token +
signing_key ONCE when webhook trigger
is enabled; never exposed again
GET /api/loops/:id detail
PATCH /api/loops/:id update definition
DELETE /api/loops/:id delete
POST /api/loops/:id/run trigger one iteration NOW
POST /api/loops/:id/enable set enabled=true
POST /api/loops/:id/disable set enabled=false
POST /webhooks/loops/:token public; HMAC-SHA256-verified
Scheduler (cm_runtime::spawn_loop_scheduler) wakes every 10s, queries the
partial index on (next_fire_at) for due loops, enqueues one topology_runs
row per fire with loop_id + iteration + parent_run_id chained back to the
previous iteration. Uses croner via the existing scheduling::next_occurrence
helper. Missed windows fire ONCE and skip the backlog — next_fire_at is
always computed strictly AFTER now(), so a late scheduler doesn't drain a
buildup.
Webhook signatures follow the same pattern as the Stripe billing webhook
(HMAC-SHA256 with constant-time hex compare). Token + signing key are
24-byte OS-RNG values; the URL uses base64-url for the token, and the
signing key is base64-std. Both surface exactly once at create time.
All three fire paths (scheduler, immediate-run, webhook) funnel through
`cm_db::repo::loops::enqueue_iteration` so the invariants stay in one
place. `iters` repeat policy is enforced by the scheduler tick; `until`
and `on_completion` land with the orchestrator hook in commit 4.
Adds cm-llm as a direct cm-api dep, getrandom for the webhook material
generator, and wires the scheduler spawn into the server binary alongside
the resume sweeper and outbox drainer.
|
||
|
Omar Sobh
|
fb047879c2 |
research: backend routes + repo + wizard refine (behind /api/research)
Second commit of the Research + Loops arc. Builds on 0030 by lighting up
the container CRUD, the agent-slot attach/detach, the standby→processing
transition, and the wizard's one-shot LLM refine.
GET /api/research list workspace's topics
POST /api/research create (accepts wizard output)
GET /api/research/:id detail (topic + attached agents)
PATCH /api/research/:id update non-status fields
POST /api/research/:id/agents attach agent (idempotent)
DELETE /api/research/:id/agents/:agent detach
POST /api/research/:id/start standby → processing
POST /api/research/wizard/refine one-shot LLM refine
The refine endpoint accumulates the workspace's default LLM provider's
stream into a JSON object (`{title, description}`) with the tight system
prompt at the top of the module. Same provider routing as agent runs
(via runtime.provider()), so a workspace already using GLM/Kimi gets it
for free.
State machine's remaining transitions (processing → reviewing on last
run_completed; reviewing → publishing via approvals gate) land with the
orchestrator hookup + approvals extension. Publish approval and loops
are separate commits still to come.
Adds cm-llm as a direct cm-api dep (previously only pulled transitively
via cm-runtime) so the refine endpoint can build a ChatRequest. Uses
sqlx::query! for compile-time verification; .sqlx cache generated on
morpheus against a fresh migrated DB.
|
||
|
Omar SobhandClaude Opus 4.8
|
fb59378aa2 |
Fleet P2b: run agent sandboxes on connected nodes (RemoteDriver + placement)
Agents can now provision their sandbox on a connected fleet node instead of the gateway host. Local stays the strict default, so existing agents are byte-for- byte unaffected until explicitly placed elsewhere. Security parity: the daemon links the REAL cm-sandbox DockerDriver and runs the typed container ops (sb_provision/sb_exec/sb_destroy/sb_health/sb_list) through it — identical hardening (cap-drop ALL, seccomp, no-net, read-only, non-root) to local sandboxes. cm-sandbox spec types are now Serialize/Deserialize so the spec crosses the channel. - cm-api: RemoteDriver (impl SandboxDriver over the node channel) + HubDriverProvider (impl cm_runtime::NodeDriverProvider, hands out a driver only for connected nodes via a sync online set) + NodeHub.call/is_connected. AppState.with_node_hub so the hub is shared with the placement provider. - cm-runtime SandboxManager: driver_for(node_id) routes by the recorded agent_containers.node_id (local default = existing driver, identical path); placement_node() reads the workspace setting and falls back to local if the node is offline; exec/release route accordingly. NodeDriverProvider trait. - DB: 0020_workspace_placement + repo (for_agent/get/set/clear). - main.rs: build the NodeHub first; inject HubDriverProvider into the agent manager + share the hub with AppState. - API+UI: GET/PUT /api/fleet/placement + a "Run agents on: Local / <node>" selector in the Fleet overview. Note: a node must be able to pull the agent image (the daemon docker-pulls it); interactive PTY for agent containers on remote nodes is not wired (Terminal app stays local) — the in-dashboard node shell already covers host access. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
f5f96508eb |
Fleet P2a: in-dashboard remote terminal (PTY over the WSS channel)
You can now open a real shell on any connected node from the dashboard — the
daemon spawns a host PTY and streams it over the existing outbound control
channel (no inbound port, no Tailscale brokering needed).
Daemon:
- portable-pty host shell sessions: pty_open/pty_in/pty_resize/pty_close ops; a
reader thread streams base64 pty_out frames. Outbound frames now funnel through
one mpsc channel so PTY output and heartbeats interleave.
cm-api NodeHub:
- per-connection pty_sinks + sid multiplexing; open_terminal/terminal_input/
terminal_resize/terminal_close; in-memory single-use terminal tickets (the
browser WS can't carry a bearer, and the session is instance-local anyway).
- routes/nodes.rs: POST /api/nodes/{id}/terminal/ticket + GET .../terminal/ws
(bridges browser xterm <-> node PTY: binary = keystrokes, text = resize).
Frontend:
- NodeTerminal xterm modal (reuses the agent Terminal's xterm setup); a Terminal
button on each online node card opens a shell.
This proves the bidirectional streaming-over-channel mechanism the RemoteDriver
will reuse. Remaining P2: RemoteDriver + placement (run agent workloads on nodes).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
e61724ff82 |
Agent computer: terminal (tmux + drives + tabs), Obsidian vault, UI polish
Terminal app (xterm ⇄ WebSocket ⇄ per-agent themed container):
- zsh + oh-my-zsh + powerlevel10k image (agent-terminal), runs as uid 65532 to
share read-write ownership of the file-drive volume with the server.
- Interactive PTY in cm-sandbox (bollard exec tty/attach + resize) + a
TerminalManager; ticket-authed WS bridge routed straight to the backend via a
Traefik PathRegexp(/ws) rule. MOTD greets the user by name.
- tmux resumable sessions; multi-tab (one tmux session per tab, same container),
drag-to-reorder, rename, and a Save that persists named tabs to the server
(terminal_tabs, migration 0014) so they survive logout / a new device.
- Files drives mounted per-agent (subpath) at ~/drives/{documents,received,
shared}; a reconciler keeps the Files app's index in sync with terminal writes.
Storage moved to a shared `filedata` volume (CLAWMATES_STORAGE__DATA_DIR).
Obsidian vault (a markdown "second brain" per agent):
- New `vault` FileDrive (migration 0015) mounted into the terminal at ~/obsidian;
a file-content read route; a purple Obsidian tile + a vault viewer app.
Computer UI:
- Draggable computer-panel width (min = phone preset) keeping the size presets.
- Green Terminal glyph, "Claw Chat" → "Chat", colored gradient-outline app icons.
- Agent page: avatar↔activity-grid spacing + larger, uniform section fonts with
colored section-tinted tag chips.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
34f744734b |
Large World graph, agent platform, brain stack & dashboard rebuild
Frontend - Large World: collapse org/company/team tiers into one expandable React Flow hierarchy (WorldFlow) with per-click expand, persisted node positions, a compact tree sidebar, wrench multi-select delete across levels, and a sized right slide-out (phone/tablet/full) showing an agent summary + drill button. - Agent page: GitHub-style animated contribution grid (VitalsCard), collapsible System Prompt + Personality cards, restructured anatomy cards, bigger avatar with name/title header row, Markdown/JSON-aware rendering, brain registry + history, avatar generate/upload. - User-icon menu (Infrastructure/Brains/Tools/Profile/Credits) + ToolPanel; Master Planner deploy wizard (Specialists/Swarm/Scheduled/Triggered); Team Runs view; reap-progress modal; dashboard is the single live interface. Backend - cm-brain crate (.brain as the agent definition) + brain apply/history. - Hard-purge reap (FK-ordered) + sandbox release + SSE batch-delete. - Swarm self-verifying loop, mode-aware planner, web.search tool, webhooks (migration 0013), org/company/team delete endpoints, scheduler sweeps. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
6d77a0acc1 |
Topology P-zc 1A: ZeroClawDriveExecutor — real role-agents over /ws/chat
cm-orchestrator owns the topology graph; each turn now drives a real ZeroClaw
role-agent in a container via the proven gateway drive recipe, instead of a
tool-free cm-llm call.
- crates/cm-api/src/topology_exec.rs: ZeroClawDriveExecutor impl TurnExecutor —
pair (POST /pair + X-Pairing-Code, token cached) -> ws /ws/chat?agent=<alias>
-> send {type:message,content} -> drain chunk/done/approval_request/error.
approval_request is recorded as a BLOCKED GatedAction, never auto-approved (§15).
Role->alias via ZEROCLAW_AGENT_MAP, fallback ZEROCLAW_DEFAULT_AGENT (scout).
- POST /api/topologies/run {task,graph} -> execute() -> RunRecord, persisted
best-effort to the existing topology_runs table (no migration). compare stays
tool-free. from_env() is read in-handler so cm-api still boots unset.
- deploy/clawmates-runtime: example config now declares a tool-free multi-agent
role-cast; README documents the ZEROCLAW_* knobs + run endpoint.
tokio-tungstenite 0.26 (already in lock) + dev axum `ws` for the hermetic test.
3 lib tests green, clippy clean, SQLX_OFFLINE build clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
a35c82d860 |
feat(api): POST /api/topologies/compare (provider-backed)
Run a task across topologies via the API and return a leaderboard + quality/cost Pareto front. The handler builds a ProviderExecutor + JudgeScorer over the runtime's configured provider/model, then calls cm_orchestrator::compare. - cm-runtime: expose provider()/model()/max_tokens() accessors on Runtime. - cm-api: depend on cm-orchestrator (provider feature); add the compare route. - Integration test runs a 2-topology comparison through the real server (scripted provider) → 200 with results + leaderboard. 4 topology tests green; offline build + clippy clean. The topology endpoints (catalog/classify/build/compare) ship to gw-04 with the upcoming ReactFlow UI in one server+frontend redeploy. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
fa7e5dec5f |
feat(api): topology endpoints (catalog / classify / build)
Stateless, auth-gated routes backing the topology builder UI:
- GET /api/topologies — catalog of the 12 kinds + descriptions + role mix
- POST /api/topologies/classify — infer a kind + metrics from a posted graph
- POST /api/topologies/build — build a canonical graph from {kind, roles}
Add Serialize to cm-topology Classification/GraphMetrics; add ApiError::BadRequest
(400) for invalid build input; add cm-topology dep. 3 integration tests
(catalog/build/auth) green against Postgres; offline build + clippy clean.
No DB or provider yet — running/comparing topologies is a later provider-backed
endpoint. Server redeploy will batch with the ReactFlow UI that consumes these.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Fable 5
|
f3f08a8edd |
R4 backend: team org-chart + leaderboard, Stripe credits, workspace apps
- GET /api/team/orgchart — each member grouped with the claws they manage (agents.managed_by); GET /api/team/leaderboard — every claw ranked by its real usage_events rollup (credits/tokens/runs, zeros included). Both tested against real Postgres. - Stripe Buy-credits (the Slack/Clerk integration pattern): [billing] config (stripe keys + price + webhook secret + credits_per_pack); POST /api/credits/checkout opens a real Checkout Session; POST /api/billing/stripe verifies Stripe's t=,v1= HMAC (constant-time) and grants one credit lot, idempotent on the session id; GET /api/billing/config gates the button (honest degradation when unset). Offline tests: signed grant + replay no-double-grant + forged-sig 400 + config flag. Live checkout creation deferred to a CM_LIVE_STRIPE test. - /apps global page support: clawId now optional on connect + directory; absent => workspace-wide connection (app_connections.agent_id NULL) via new connections::list_for_workspace. - ApiError gains a From<sqlx::Error> so inline queries use ? cleanly. cm-api 10 test files incl. team_tabs (2) + stripe_billing (3); clippy clean. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
add4f79fed |
Rebrand: TeamClaw -> Clawmates (clawmates.work)
Full-depth rename per the approved plan; the 'claw' product vocabulary (claws, /claws routes, clawId, Claw Chat) stays — it is now the brand. - Display brand: Clawmates (manifest, titles, hero, login/rail logo 'clawmates'); default host app.clawmates.work; registry ghcr.io/clawmates - Crates tc-* -> cm-* (16 crates + all imports); binaries clawmates-server/broker/bundler; images clawmates/*; env prefix CLAWMATES_* (+ CM_TEST_DATABASE_URL / CM_LIVE_LLM); config clawmates.toml; helm chart deploy/helm/clawmates with clawmates-* resources; db names clawmates*; sockets /run/clawmates; cookie cm_session; kind cluster clawmates-test; seccomp node profile clawmates-agent-profile.json - All 9 Playwright brand assertions updated in lockstep; historical spec document left untouched as the only remaining 'TeamClaw' - Local env migrated: dev pg clawmates-dev-pg/clawmates_dev, shared test server clawmates-test-pg, kind cluster recreated with image + profile, compose images rebuilt under clawmates/* Verified end to end: 161 Rust + 68 frontend tests, 29 Playwright journeys, 4 live kind tests, helm/install/LOC/placeholder gates, and the clean-room install rehearsal serving the clawmates login page from a signed bundle of the rebuilt images. Co-Authored-By: Claude Fable 5 <[email protected]> |