docs: bring the README up to date with two months of shipped work
deploy / test (push) Successful in 5m25s
deploy / build (push) Successful in 1m9s

Last rewritten 2026-07-21. Verified against the repo and prod before writing:
- adds what shipped since: the single dashboard, recipes + team templates,
  container and microVM tiers, the independent judge (Kimi fallback, quota
  watchdog, offline npm), tool gates + shadow task-permission/taint, skills
  door, project memory + self_audit, continuous research/podcast, cm-decide,
  fleet placement, A2A
- fixes wrong facts: cm-brain is the .brain facade (not LLM planning);
  cm-decide and fcagent were missing; migrations are 0001–0087 in migrations/;
  the deploy path is CI (test → build → push) plus the gw-04 timer
- the line-count budget is NOT enforced in CI (12 files exceed 1500) — said so
- safety section no longer claims every agent is tool-free and network-less:
  container-tier missions have public egress, and the gate is a guardrail,
  not a boundary

Every relative link and repo path in the file was checked to exist.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
Omar Sobh
2026-09-23 10:04:40 -05:00
co-authored by Claude Opus 5.5
parent 318a6791b9
commit 1e293f2840
+129 -95
View File
@@ -6,9 +6,9 @@ the organizational *topology* that fits it.**
Clawmates is a multi-agent platform where every unit of work is a **topology**: a graph of role-slots Clawmates is a multi-agent platform where every unit of work is a **topology**: a graph of role-slots
bound to real AI agents ("claws"). The same model nests recursively — a team is a topology of claws, a bound to real AI agents ("claws"). The same model nests recursively — a team is a topology of claws, a
company is a topology of teams, an org is a topology of companies — so you compose and run agentic company is a topology of teams, an org is a topology of companies — so you compose and run agentic
systems from one agent up to an entire organization. A safety invariant runs through all of it: systems from one agent up to an entire organization. The work itself runs as **missions**: recipes of
**authority is topology-invariant** — no choice of structure can let an agent exceed its sandbox (spec phases, staffed by team templates, executed in isolated containers or Firecracker microVMs, and checked
§15). by an independent judge before anything is called done.
Live at **[clawmates.work](https://clawmates.work)**. Live at **[clawmates.work](https://clawmates.work)**.
@@ -16,34 +16,50 @@ Live at **[clawmates.work](https://clawmates.work)**.
## Features ## Features
- **The deploy ladder — single → team → company → org.** Pick a scale; each rung instantiates a baseline - **One dashboard.** The workspace home is a single live dashboard with a tier rail — **World, Agents,
topology and binds it to real, individually-chattable claws. Higher rungs *compose* the rung below: Missions, Repos, Infra, Podcast**. World is a live Gource-style graph of what agents are doing; the
a company is staffed with teams, an org with companies. org → company → team structure is one expandable graph; each agent opens a slide-out "computer"
- **Missions.** The primary user-facing unit: a scoped multi-team workload with team templates, live (chat, terminal, files, brain).
progress, bulk operations, and a canvas view. Missions are hard-required to include a team template - **Missions.** A mission is a **recipe** (`templates/workflows/*.toml`) of ordered phases — research,
and can span research + development teams. coding, security scan, benchmark — each with a task brief and a `done_when` completion condition, staffed
- **Recursive execution.** Running a parent runs each child's whole sub-topology, all the way down to by a **team template** (`templates/teams/*.toml`). Seven recipes ship: `research_only`,
the leaf claws — on a **durable, crash-resumable** runner (checkpointed per step, with cancellation). `research_and_code`, `security_hardening`, `refactor`, `benchmark`, `continuous_research`, `self_audit`.
- **INFRA tier via Herdr.** Every fleet node runs a persistent `clawmates-node` daemon (herdr) reachable - **Two execution tiers.** *Container tier*: each mission gets its own ZeroClaw runtime container on the
from the platform: mission wizard picks a target runtime, `fleet_herdr` dispatches on launch, and gateway, running Claude Code (`claude_cli`) with fallback to Kimi and GLM. *MicroVM tier*: phases run
a Live Pane surfaces each node's herdr TUI via xterm.js. in Firecracker microVMs on fleet nodes (`clawmates-node` + the `fcagent` guest), with per-backend
- **12 organizational topologies.** Hierarchical, pipeline, swarm, mesh, debate, hub-spoke, star-MoE, egress (Claude, GLM, Kimi, or a local model over vsock).
market, ring, flat, holacratic, blackboard — over five execution patterns. - **An independent judge.** Every conditioned phase is judged by a model from a *different* provider
- **Multi-topology comparison + evolution.** Run one task across many topologies and get a quality/cost family than the agents (GLM by default, Kimi as automatic fallback), which runs its own allow-listed
**Pareto front**; a MAP-Elites search evolves better (kind × team-size) configurations using the checks — tests, `rg`, git — against a copy of the work, commits to a verification plan before reading
comparison harness as fitness. the evidence, and installs npm dependencies offline from the lockfile. A phase that fails is retried
- **Recursive zoom canvas.** One view for every tier: click a node to drill down (org→company→team→claw), with the judge's guidance, up to `max_iterations`. Measured with `scripts/judge-eval.sh` (15 known-answer
breadcrumb to zoom back up. cases).
- **§15 safety by construction.** Agents run tool-free in network-isolated sandboxes; every - **Judge quota watchdog.** Polls the GLM and Kimi usage APIs, warns at 80% of any window, and switches
sandbox-leaving action is a gated, human-approvable "door" tool. A secret broker holds credentials that judging to the fallback at 95% (`GET /api/judge/quota`).
never reach agent code, and an allow-listed Docker socket caps blast radius. - **Delivery to git.** Each phase's work is committed and pushed to a mission branch on the repo's forge;
- **Heterogeneous models.** Bind any node to a different backend; supported providers include Claude `continuous_research` auto-merges additive-only changes into the vault.
(default for refine), GLM, Kimi, Groq. Configured per-node in the wizard. - **Tool gates on every agent call.** A `PreToolUse` gate (both tiers) refuses destructive and exfiltrating
- **Level-Up.** Per-claw and per-team improvement proposals with an inbox + review drawer. commands, protects its own hook files, enforces per-role policy (e.g. a read-only verifier), and records
- **Beszel + Tailscale integration.** First-class routes to the fleet's monitoring hub and mesh. task-permission and argument-provenance ("taint") violations in shadow mode. A stop gate keeps a
- **Self-hostable.** A single-node Docker Compose deployment runs the whole platform with the same microVM agent working until its `done_when_check` passes.
network-segmented security model as the Kubernetes path; a separate rolling-deploy path serves - **Skills.** A catalog of skills bound to team roles, delivered to mission agents through the MCP
clawmates.work from gw-04 against the fleet registry. skills door; per-mission skill triage and skill-use measurement.
- **Project memory.** Each repository keeps a `.brain` (ClawhDF5) of every judge verdict; missions recall
relevant past verdicts, and the `self_audit` recipe audits the whole record for failure patterns.
- **Continuous research + podcast.** Harvests new arXiv papers into an Obsidian vault, triages them,
writes an analysis and a two-host script, and renders audio with ElevenLabs.
- **Decision tier (`cm-decide`).** Cheap calibrated classifiers (Jev) for gut-check decisions: the §15
door governor (allow / deny / hold for approval), skill triage, paper triage, memory rerank.
- **Master Planner.** The "+" deploy is a chat that proposes and scaffolds a team for a goal
(specialists, swarm, scheduled, triggered).
- **12 organizational topologies + evolution.** Hierarchical, pipeline, swarm, mesh, debate, hub-spoke,
star-MoE, market, ring, flat, holacratic, blackboard; multi-topology comparison with a quality/cost
**Pareto front**, and MAP-Elites evolution over (kind × team size).
- **Fleet.** Multi-user workspaces with quotas; a node pool (`clawmates-node` over Tailscale) with
capacity-aware placement, drain, per-node tool versions and one-click updates, and Beszel metrics
feeding a rules engine.
- **Agent-to-agent comms.** Chat rooms, a gated delegation bridge, per-claw door identity and A2A ingress.
- **Self-hostable.** A single-node Docker Compose deployment runs the whole platform.
--- ---
@@ -58,12 +74,13 @@ A Rust workspace (the platform) + a Next.js app (the web UI).
| `cm-domain` | Shared types: ids, roles, workspaces, users | | `cm-domain` | Shared types: ids, roles, workspaces, users |
| `cm-topology` | Topology data model: 12-kind taxonomy, graph, classifier, per-kind builders + heuristics | | `cm-topology` | Topology data model: 12-kind taxonomy, graph, classifier, per-kind builders + heuristics |
| `cm-orchestrator` | Execution engine: async control-flow over a generic `TurnExecutor`; planners, comparison harness, MAP-Elites evolution | | `cm-orchestrator` | Execution engine: async control-flow over a generic `TurnExecutor`; planners, comparison harness, MAP-Elites evolution |
| `cm-runtime` | The §15-safe per-tenant agent runtime | | `cm-runtime` | The §15-safe per-tenant agent runtime and the LLM provider registry |
| `cm-brain` | Shared LLM planning + reasoning primitives used by orchestrator and refine | | `cm-brain` | Facade over the canonical `.brain` (ClawhDF5 brain-pack): one HDF5 file per agent or repo holding definition + memory |
| `cm-api` | REST/SSE API + streaming gateway + recursive tier execution + the MCP "door" + missions + fleet_herdr | | `cm-decide` | Typed, calibrated decisions (Jev classifiers) for the platform's code to branch on |
| `cm-api` | REST/SSE API, missions + phase runner, judge, tool gates, delivery, the MCP doors, fleet, podcast |
| `cm-db` | Postgres persistence (sqlx, offline-checked) | | `cm-db` | Postgres persistence (sqlx, offline-checked) |
| `cm-llm` | Provider abstraction over the model backends | | `cm-llm` | Provider abstraction (Anthropic-format and OpenAI-compatible backends) |
| `cm-secrets` / `clawmates-broker` | The secret broker — credentials never leave it | | `cm-secrets` | Secret storage behind the broker |
| `cm-sandbox` / `cm-safety` | Sandbox provisioning + the §15 approval/gating model | | `cm-sandbox` / `cm-safety` | Sandbox provisioning + the §15 approval/gating model |
| `cm-tools` | Tool contract + registry surfaced through the door | | `cm-tools` | Tool contract + registry surfaced through the door |
| `cm-testkit` | Shared test utilities (scripted providers, fixture builders) | | `cm-testkit` | Shared test utilities (scripted providers, fixture builders) |
@@ -73,21 +90,26 @@ A Rust workspace (the platform) + a Next.js app (the web UI).
| Bin | Role | | Bin | Role |
|---|---| |---|---|
| `clawmates-server` | The single server binary (API + gateway + runtime + scheduler) | | `clawmates-server` | The single server binary (API + gateway + runtime + background workers) |
| `clawmates-broker` | Out-of-process secret broker over a private unix socket | | `clawmates-broker` | Out-of-process secret broker over a private unix socket |
| `clawmates-node` | The **herdr** daemon: runs on every fleet node, dispatches missions to that node, exposes a TUI streamed into the Live Pane | | `clawmates-node` | Fleet node daemon: registers with the server, runs microVMs, terminals and tool updates on its node |
| `fcagent` | PID 1 inside each Firecracker microVM; answers the host over vsock |
### Frontend (`frontend/`) ### Frontend (`frontend/`)
Next.js 16, React 19, Tailwind v4. Two-tier rail (structure + context), the recursive zoom canvas, and Next.js 16, React 19, Tailwind v4. The dashboard at `/` (tier rail, World graph, agent computer),
the deploy wizards. Missions surface (canvas + list + wizard + live pane + team tab + live events), plus agent, team, skills, approvals and team-management pages. Talks to the backend through a
Herdr sessions UI, Level-Up inbox + review drawer. Talks to the backend through a same-origin `/api` same-origin `/api` proxy that swaps the session for a bearer token and streams SSE.
proxy that swaps the session for a bearer token and streams SSE.
### Content (`templates/`)
`templates/teams/` (12 team templates: roles, prompts, brain seeds, skill bindings) and
`templates/workflows/` (7 recipes), loaded at boot. How mature each one is — which have been run and
what they delivered — is tracked in [`docs/TEMPLATE-MATURITY.md`](docs/TEMPLATE-MATURITY.md).
### Data plane ### Data plane
Postgres, with the server self-migrating on boot. Migration series `00010057+`; slice-9 cleanup Postgres, with the server self-migrating on boot from `migrations/` (`0001``0087`).
(`0053`) retired the legacy research/loops path after missions replaced it.
--- ---
@@ -111,7 +133,8 @@ Owner + workspace on first boot. Then:
- **API health** → http://localhost:8080/healthz - **API health** → http://localhost:8080/healthz
Backends: `openai_compat` by default (point `[llm].base_url` at vLLM/Ollama/llama.cpp); set Backends: `openai_compat` by default (point `[llm].base_url` at vLLM/Ollama/llama.cpp); set
`provider = "anthropic"` + `ANTHROPIC_API_KEY` to use Claude. Auth is `local` by default or `clerk` at `provider = "anthropic"` + `ANTHROPIC_API_KEY` to use Claude. Extra named providers (GLM, Kimi, a local
model) go in `[[llm.providers]]` and are selected as `name:model`. Auth is `local` by default or `clerk` at
runtime. See [`deploy/compose/README.md`](deploy/compose/README.md) for all knobs and the broker runtime. See [`deploy/compose/README.md`](deploy/compose/README.md) for all knobs and the broker
master-key backup step. master-key backup step.
@@ -142,40 +165,45 @@ Run the reproducible topology benchmark (offline-deterministic; real models with
cargo run -p cm-orchestrator --example topology_bench --features provider cargo run -p cm-orchestrator --example topology_bench --features provider
``` ```
**End-to-end verification against a deployment:**
```bash
scripts/verify-mission-delivery.sh <scenario> # launches real missions, asserts delivery, gates, judge
scripts/judge-eval.sh # the judge's 15 known-answer cases (JUDGE=kimi to compare)
```
--- ---
## Production deployment (clawmates.work on gw-04) ## Production deployment (clawmates.work on gw-04)
The public site runs a different path than the airgapped compose. Gitea Actions builds and pushes - **CI:** a push to `main` runs [`.gitea/workflows/deploy.yml`](.gitea/workflows/deploy.yml) on the
`broker`, `server`, and `frontend` images to the fleet registry at gw-04 Gitea runner: `cargo test --workspace`, then builds the images and pushes
`100.94.185.103:5000/clawmates/<svc>:latest`. gw-04 runs a systemd-timer-driven rolling deploy: `main-<sha>` and `:latest` to the fleet registry at `100.94.185.103:5000`. The workflow then waits
up to 5 minutes for prod to report the new commit.
- **Roll-out:** [`deploy/gw-04/clawmates-deploy.sh`](deploy/gw-04/clawmates-deploy.sh), run every minute
by `clawmates-deploy.timer`, drift-checks each running image against `:latest` and recreates the
service on drift. Logs: `/var/log/clawmates-deploy.log`.
- **Host config** lives outside git on gw-04: `/opt/clawmates/.env` and `/opt/clawmates/clawmates.toml`
(provider registry, including the judge's GLM and Kimi providers).
- **Deploy script:** [`deploy/gw-04/clawmates-deploy.sh`](deploy/gw-04/clawmates-deploy.sh) — polls the **gw-04-specific gotchas:**
registry, drift-checks each service's running image ID against `:latest`, and calls
`docker compose up -d <svc>` on drift. Portable across `docker compose` v2 and legacy `docker-compose` v1.
- **Timer + unit:** installed alongside the script under `/etc/systemd/system/`.
- **Logs:** `/var/log/clawmates-deploy.log`.
- **Compose file:** references registry-prefixed images directly — no retag bridging.
**gw-04-specific gotchas (bit us on 2026-07-09 and 2026-07-12):**
- `clawmates-runtime` on gw-04 is **not compose-managed** — it's a standalone `docker run` invocation. - `clawmates-runtime` on gw-04 is **not compose-managed** — it's a standalone `docker run` invocation.
Provider env (`ANTHROPIC_API_KEY`, `ZEROCLAW_providers__*`) must be set on that container. Provider env (`ZEROCLAW_providers__*`) must be set on that container.
- The server container runs as **UID 65532** (distroless nonroot). Any bind-mount host path must be - The server container runs as **UID 65532** (distroless nonroot). Any bind-mount host path must be
`chown 65532:65532` before boot or the server can't write. `chown 65532:65532` before boot or the server can't write.
- Per-team ZeroClaw containers inherit `ZEROCLAW_providers__*` from the server; those envs must live - Per-mission ZeroClaw containers get their provider keys forwarded from the server; those envs must live
on the compose `server` block, not just on shared runtime. on the compose `server` block, not just on the shared runtime.
- The judge's container (`clawmates-runtime` on `clawmates_core`) has **no route to the internet** by
design; dependencies it needs are installed offline.
--- ---
## CI budgets ## Code size budget
- **Hard limit:** 1500 lines per source file. CI fails. [`ci/check-loc.sh`](ci/check-loc.sh) defines a soft limit of 1100 lines and a hard limit of 1500 per
- **Soft limit:** 1100 lines. CI warns — split before it hurts. source file. It is **not currently run in CI**, and 12 files exceed the hard limit (the largest,
- Enforced by [`ci/check-loc.sh`](ci/check-loc.sh). `crates/cm-api/src/phase_runner.rs`, is ~3,450 lines). Split files when you touch them.
Common split pattern: extract sub-components (`MissionLivePane`, `AutoProvisionCard`) into their own
file when the parent creeps past the soft limit.
--- ---
@@ -183,32 +211,26 @@ file when the parent creeps past the soft limit.
**Shipped** **Shipped**
- ✅ Pure-Rust topology engine — model → classify → build (all 12 kinds) → execute → compare (Pareto) → - ✅ Pure-Rust topology engine — model → classify → build (all 12 kinds) → execute → compare (Pareto) →
workflow → LLM-judge → evolve. evolve; durable, crash-resumable topology runs with live SSE.
- ✅ Topologies UI: catalog browser, builder/visualizer, multi-topology comparison with a Pareto scatter. - ✅ The deploy ladder (single → team → company → org) and the single live dashboard.
-Durable topology runs — crash-resumable, checkpointed per step, cancellable, with live SSE. -Missions as recipes of judged phases, on container and microVM tiers, delivered to git.
-**The full deploy ladder** — single → team → company → org, with recursive execution down to the -Independent cross-provider judge with its own checks, verification plans, offline npm installs,
leaf claws and a recursive zoom canvas + two-tier navigation. Kimi fallback and a quota watchdog.
-**Missions (slices 19)** — multi-team model, hard-required team templates, canvas + wizard + list, -`PreToolUse` gate on both tiers (floor rules, role policy, protected hook files) with task-permission
auto-refresh + Team tab + Live events tab, add/edit/delete toolbar, bulk delete, security-scan + and argument-provenance (taint) in shadow; microVM stop gate.
benchmark trigger buttons, per-mission repo checkout on launch, LevelUpInbox mounted. - ✅ Skills catalog + MCP skills door; skill triage and skill-use measurement.
-**Herdr (phases 03)**`clawmates-node` daemon (systemd/launchd persistence), `fleet_herdr` -Per-repo project memory (`.brain`) and the `self_audit` recipe over it.
dispatch module + node daemon ops, missions `runtime_kind` + `target_node` schema, wizard runtime - ✅ Continuous research → vault → podcast, end to end.
picker with `on_launch` auto-dispatch, Live Pane (xterm.js → node's herdr TUI), INFRA-tier Herdr - ✅ Decision tier (`cm-decide`): calibrated door governor with a held band for approval.
sessions surface. - ✅ Multi-user workspaces, node pool with capacity-aware placement, Beszel metrics + rules.
-**Refine on Opus 4.8** — before/after diff view, accept/cancel/restore controls. -Self-host via Docker Compose; CI → registry → timer-driven roll-out on gw-04.
-**Level-Up** — per-claw/per-team improvement proposals, inbox + review drawer.
- ✅ §15 safety: tool-free sandboxes, the gated MCP "door" (with real email/Slack delivery), the secret
broker, allow-listed Docker socket.
- ✅ Self-host: single-node Docker Compose with full network segmentation.
- ✅ Prod path on gw-04: registry-driven rolling deploy via systemd timer.
**Next** **Next**
- Team / company templates as first-class saved catalogs (compose orgs from reusable building blocks). - Enforce task permission and argument provenance (both still in shadow, gathering evidence).
- Per-leaf nested checkpoint resume (today the recursive runner resumes at parent-node granularity). - Evidence the remaining team templates (4 of 12 still need a target stack: mobile, gpu, threejs, and
- Persona injection into runtime turns (beyond role-driven prompting). `insight_research`).
- Richer per-tier dashboards (company coordination, org portfolio/governance metrics). - A dedicated judge key, so no other consumer of a shared provider plan can starve the judge.
- Group lifecycle management (delete/edit a deployed team/company/org; deprovision its agents) — - Wire `ci/check-loc.sh` into CI once the oversized files are split.
bulk delete shipped for missions, extending to teams/companies/orgs next.
**Research** **Research**
- The accompanying paper, *Large Dynamic Agentic Topologies* (`papers/dynamic-agentic-topologies.md`): - The accompanying paper, *Large Dynamic Agentic Topologies* (`papers/dynamic-agentic-topologies.md`):
@@ -218,8 +240,20 @@ file when the parent creeps past the soft limit.
## Safety ## Safety
The network segmentation **is** the security model. Agent sandboxes run with no network at all; only the The design goal is **authority that no topology can widen** (spec §15): a structure, or a switch between
browser container has egress. The secret broker is reachable only over a private socket and credentials structures, never gives an agent more reach than its sandbox. What that means today, per tier:
never enter agent code. The server reaches Docker through an allow-listed socket proxy that can manage
sandbox containers and nothing else. Every sandbox-leaving action is gated behind a human approval. No - **Chat / §15 agents** run tool-free; every action that leaves the sandbox goes through the MCP door,
topology — and no switch between topologies — can bypass any of this. where a calibrated governor allows, denies, or **holds for human approval**.
- **Mission agents** do run tools (Bash, file edits) inside their own container or microVM, behind the
`PreToolUse` gate. MicroVMs have no NIC and reach only an allow-listed set of hosts through a proxy.
**Container-tier missions currently have open public-internet egress** (tailnet and host SSH are
blocked); see [`docs/MISSION-EGRESS.md`](docs/MISSION-EGRESS.md) and
[`docs/TASK-PERMISSION-AND-TAINT.md`](docs/TASK-PERMISSION-AND-TAINT.md) for the measurements and the
controls being built on top.
- Credentials are held by the secret broker behind a private socket. A mission container gets only a
narrowly scoped skills token, never a full session. The server reaches Docker through an allow-listed
socket proxy.
- The gate is a guardrail against accidents and obvious exfiltration, not a boundary against a
determined agent (indirection defeats string matching). The boundaries are the VM, the network policy
and the broker.