Omar SobhandClaude Opus 5 794f2124bc
deploy / test (push) Successful in 5m18s
deploy / build (push) Successful in 5m54s
feat(microvm): Claude Code 2.1.276 rootfs pins, and a VM run that says what it ran
Every rootfs on the fleet had sat on Claude Code 2.1.223–2.1.226 since August
while the container tier moved to 2.1.276, and nothing recorded either. GLM
and Kimi exist only as microVM backends, so "have we upgraded GLM and Kimi"
is this change and the rebuild it drives.

Pins. All four agent-* images pin 2.1.276 — as separate ARGs, since Docker has
no include and each file has to stay reproducible alone — and
scripts/fc-build-rootfs.sh refuses to build if they disagree, naming the odd
one out. They had already drifted (claude 226, the rest 223) under comments
saying "same version on purpose". Between 2.1.226 and 2.1.276, 2.1.265 and
2.1.275 each broke every turn on ANTHROPIC_BASE_URL endpoints, which is how
glm and kimi reach `claude` inside a VM; the container-tier verification never
exercised that path, so the VM runs on those backends are the real test.

Provenance. `VmOutcome` carries the rootfs the node reported booting and the
guest's own `claude --version`; `launch_microvm_phase` persists both as
`checkpoint.vm` beside `records` (the two readers parse only `records`) and
names them in its log line. "Which image and CLI did this mission run on" is
a query now.

Independence. `evaluator` derived the implementer family from a constant
`"anthropic"`, true while every backend was Claude on Anthropic. With glm and
kimi rootfs it made a glm mission judged by glm:glm-5.3 read as
`independent = true` — the one claim that path exists to make honestly.
`implementer_family(missions.backend)` mirrors `microvm_credential_for`; the
subscription judge is now independent exactly when the agent did NOT run on
Anthropic.

Harness. `verify-mission-delivery.sh glm|kimi` run the microvm scenario on
each backend and add the proof the mission itself cannot give: the placed
node's journal must show the VM dialling that provider's host, never being
denied it, and dialling nothing else but the forge — a model's self-report is
measured worthless here. `assert_cli_version` reads checkpoint.vm. The stale
scratch-repo default (dead since the 09-14 wipe) is the re-synced id.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
2026-09-18 20:16:06 -05:00

Clawmates

Deploy agents at any scale — a single claw, a team, a company, or a whole org — and run a mission across the organizational topology that fits it.

Clawmates is a multi-agent platform where every unit of work is a topology: a graph of role-slots bound to real AI agents ("claws"). The same model nests recursively — a team is a topology of claws, a company is a topology of teams, an org is a topology of companies — so you compose and run agentic systems from one agent up to an entire organization. A safety invariant runs through all of it: authority is topology-invariant — no choice of structure can let an agent exceed its sandbox (spec §15).

Live at clawmates.work.


Features

  • The deploy ladder — single → team → company → org. Pick a scale; each rung instantiates a baseline topology and binds it to real, individually-chattable claws. Higher rungs compose the rung below: a company is staffed with teams, an org with companies.
  • Missions. The primary user-facing unit: a scoped multi-team workload with team templates, live progress, bulk operations, and a canvas view. Missions are hard-required to include a team template and can span research + development teams.
  • Recursive execution. Running a parent runs each child's whole sub-topology, all the way down to the leaf claws — on a durable, crash-resumable runner (checkpointed per step, with cancellation).
  • INFRA tier via Herdr. Every fleet node runs a persistent clawmates-node daemon (herdr) reachable from the platform: mission wizard picks a target runtime, fleet_herdr dispatches on launch, and a Live Pane surfaces each node's herdr TUI via xterm.js.
  • 12 organizational topologies. Hierarchical, pipeline, swarm, mesh, debate, hub-spoke, star-MoE, market, ring, flat, holacratic, blackboard — over five execution patterns.
  • Multi-topology comparison + evolution. Run one task across many topologies and get a quality/cost Pareto front; a MAP-Elites search evolves better (kind × team-size) configurations using the comparison harness as fitness.
  • Recursive zoom canvas. One view for every tier: click a node to drill down (org→company→team→claw), breadcrumb to zoom back up.
  • §15 safety by construction. Agents run tool-free in network-isolated sandboxes; every sandbox-leaving action is a gated, human-approvable "door" tool. A secret broker holds credentials that never reach agent code, and an allow-listed Docker socket caps blast radius.
  • Heterogeneous models. Bind any node to a different backend; supported providers include Claude (default for refine), GLM, Kimi, Groq. Configured per-node in the wizard.
  • Level-Up. Per-claw and per-team improvement proposals with an inbox + review drawer.
  • Beszel + Tailscale integration. First-class routes to the fleet's monitoring hub and mesh.
  • Self-hostable. A single-node Docker Compose deployment runs the whole platform with the same network-segmented security model as the Kubernetes path; a separate rolling-deploy path serves clawmates.work from gw-04 against the fleet registry.

Architecture

A Rust workspace (the platform) + a Next.js app (the web UI).

Backend — Rust workspace (crates/)

Crate Role
cm-domain Shared types: ids, roles, workspaces, users
cm-topology Topology data model: 12-kind taxonomy, graph, classifier, per-kind builders + heuristics
cm-orchestrator Execution engine: async control-flow over a generic TurnExecutor; planners, comparison harness, MAP-Elites evolution
cm-runtime The §15-safe per-tenant agent runtime
cm-brain Shared LLM planning + reasoning primitives used by orchestrator and refine
cm-api REST/SSE API + streaming gateway + recursive tier execution + the MCP "door" + missions + fleet_herdr
cm-db Postgres persistence (sqlx, offline-checked)
cm-llm Provider abstraction over the model backends
cm-secrets / clawmates-broker The secret broker — credentials never leave it
cm-sandbox / cm-safety Sandbox provisioning + the §15 approval/gating model
cm-tools Tool contract + registry surfaced through the door
cm-testkit Shared test utilities (scripted providers, fixture builders)
cm-auth, cm-billing, cm-files, cm-scheduler, cm-config, cm-telemetry Supporting services

Binaries (crates/bins/)

Bin Role
clawmates-server The single server binary (API + gateway + runtime + scheduler)
clawmates-broker Out-of-process secret broker over a private unix socket
clawmates-node The herdr daemon: runs on every fleet node, dispatches missions to that node, exposes a TUI streamed into the Live Pane

Frontend (frontend/)

Next.js 16, React 19, Tailwind v4. Two-tier rail (structure + context), the recursive zoom canvas, and the deploy wizards. Missions surface (canvas + list + wizard + live pane + team tab + live events), Herdr sessions UI, Level-Up inbox + review drawer. Talks to the backend through a same-origin /api proxy that swaps the session for a bearer token and streams SSE.

Data plane

Postgres, with the server self-migrating on boot. Migration series 00010057+; slice-9 cleanup (0053) retired the legacy research/loops path after missions replaced it.


Quickstart

The fastest path is the single-node Docker Compose deployment.

cd deploy/compose
cp .env.example .env
# Edit .env: set POSTGRES_PASSWORD and CLAWMATES_BOOTSTRAP_OWNER_PASSWORD.

docker compose build      # or load a signed release bundle (deploy/airgapped/install.sh)
docker compose up -d

The server self-migrates and, if CLAWMATES_BOOTSTRAP_OWNER_PASSWORD is set, provisions the first Owner + workspace on first boot. Then:

Backends: openai_compat by default (point [llm].base_url at vLLM/Ollama/llama.cpp); set provider = "anthropic" + ANTHROPIC_API_KEY to use Claude. Auth is local by default or clerk at runtime. See deploy/compose/README.md for all knobs and the broker master-key backup step.

Broker master key. The broker's master key lives in the broker_key named volume and is generated on first boot. Back this up before running the stack for anything real — losing it un-decrypts every stored secret.

Local development:

# Backend (needs a Postgres; sqlx is offline-checked against .sqlx/)
cargo build
cargo test
cargo clippy --all-targets

# Regenerate the sqlx cache after changing any query:
#   DATABASE_URL=… cargo sqlx prepare --workspace

# Frontend
cd frontend
npm install
npm run dev        # also: npm run lint / npm run typecheck / npm run build

Run the reproducible topology benchmark (offline-deterministic; real models with ANTHROPIC_API_KEY):

cargo run -p cm-orchestrator --example topology_bench --features provider

Production deployment (clawmates.work on gw-04)

The public site runs a different path than the airgapped compose. Gitea Actions builds and pushes broker, server, and frontend images to the fleet registry at 100.94.185.103:5000/clawmates/<svc>:latest. gw-04 runs a systemd-timer-driven rolling deploy:

  • Deploy script: deploy/gw-04/clawmates-deploy.sh — polls the registry, drift-checks each service's running image ID against :latest, and calls docker compose up -d <svc> on drift. Portable across docker compose v2 and legacy docker-compose v1.
  • Timer + unit: installed alongside the script under /etc/systemd/system/.
  • Logs: /var/log/clawmates-deploy.log.
  • Compose file: references registry-prefixed images directly — no retag bridging.

gw-04-specific gotchas (bit us on 2026-07-09 and 2026-07-12):

  • clawmates-runtime on gw-04 is not compose-managed — it's a standalone docker run invocation. Provider env (ANTHROPIC_API_KEY, ZEROCLAW_providers__*) must be set on that container.
  • The server container runs as UID 65532 (distroless nonroot). Any bind-mount host path must be chown 65532:65532 before boot or the server can't write.
  • Per-team ZeroClaw containers inherit ZEROCLAW_providers__* from the server; those envs must live on the compose server block, not just on shared runtime.

CI budgets

  • Hard limit: 1500 lines per source file. CI fails.
  • Soft limit: 1100 lines. CI warns — split before it hurts.
  • Enforced by ci/check-loc.sh.

Common split pattern: extract sub-components (MissionLivePane, AutoProvisionCard) into their own file when the parent creeps past the soft limit.


Roadmap

Shipped

  • Pure-Rust topology engine — model → classify → build (all 12 kinds) → execute → compare (Pareto) → workflow → LLM-judge → evolve.
  • Topologies UI: catalog browser, builder/visualizer, multi-topology comparison with a Pareto scatter.
  • Durable topology runs — crash-resumable, checkpointed per step, cancellable, with live SSE.
  • The full deploy ladder — single → team → company → org, with recursive execution down to the leaf claws and a recursive zoom canvas + two-tier navigation.
  • Missions (slices 19) — multi-team model, hard-required team templates, canvas + wizard + list, auto-refresh + Team tab + Live events tab, add/edit/delete toolbar, bulk delete, security-scan + benchmark trigger buttons, per-mission repo checkout on launch, LevelUpInbox mounted.
  • Herdr (phases 03)clawmates-node daemon (systemd/launchd persistence), fleet_herdr dispatch module + node daemon ops, missions runtime_kind + target_node schema, wizard runtime picker with on_launch auto-dispatch, Live Pane (xterm.js → node's herdr TUI), INFRA-tier Herdr sessions surface.
  • Refine on Opus 4.8 — before/after diff view, accept/cancel/restore controls.
  • Level-Up — per-claw/per-team improvement proposals, inbox + review drawer.
  • §15 safety: tool-free sandboxes, the gated MCP "door" (with real email/Slack delivery), the secret broker, allow-listed Docker socket.
  • Self-host: single-node Docker Compose with full network segmentation.
  • Prod path on gw-04: registry-driven rolling deploy via systemd timer.

Next

  • Team / company templates as first-class saved catalogs (compose orgs from reusable building blocks).
  • Per-leaf nested checkpoint resume (today the recursive runner resumes at parent-node granularity).
  • Persona injection into runtime turns (beyond role-driven prompting).
  • Richer per-tier dashboards (company coordination, org portfolio/governance metrics).
  • Group lifecycle management (delete/edit a deployed team/company/org; deprovision its agents) — bulk delete shipped for missions, extending to teams/companies/orgs next.

Research

  • The accompanying paper, Large Dynamic Agentic Topologies (papers/dynamic-agentic-topologies.md): the topology-invariant authority result, the comparison/Pareto methodology, and benchmark results.

Safety

The network segmentation is the security model. Agent sandboxes run with no network at all; only the browser container has egress. The secret broker is reachable only over a private socket and credentials never enter agent code. The server reaches Docker through an allow-listed socket proxy that can manage sandbox containers and nothing else. Every sandbox-leaving action is gated behind a human approval. No topology — and no switch between topologies — can bypass any of this.

S
Description
No description provided
Readme
43 MiB
Languages
Rust 57.9%
TypeScript 24.8%
HTML 8.8%
JavaScript 4.3%
Shell 2.9%
Other 1.3%