Plan §12 proposed deleting the ~1,250 lines of `system_prompt` prose across
the 23 team templates as instruction-shaped injection. Tracing the two
prompt paths shows that would be strictly harmful:
- Missions never see it. `topology_exec::build_prompt` synthesizes its own
one-line system text from the role slot, so the prose costs zero mission
tokens and deleting it saves zero.
- Chat depends on it. `mission_orchestrator` copies it into
`agents.system_prompt`, which is the base prompt
`cm_runtime::brain::compose_system` augments for a claw's chat turns.
Deleting it leaves every mission-minted claw with no identity in chat.
The prose is also mostly information (domain standards, wire discipline)
rather than instructions restating general competence, which is the kind
ablation keeps. Comment left at the one injection site so this isn't
re-derived.
Co-Authored-By: Claude Opus 5 <[email protected]>
Three guardrails around which credential pays for what.
1. Boot announces the mission-runtime auth mode, and warns when subscription
auth is configured on a deployment with more than one user. A consumer
subscription credential may only run the account holder's own work, and
that condition is otherwise invisible -- it holds today and quietly stops
holding the first time someone else signs up. Adds users::count_all
(dynamic query, so the offline cache needs no regeneration).
2. Reject an ANTHROPIC_API_KEY shaped like a subscription OAuth token
(sk-ant-oat...) at boot rather than failing on the first model call far
from the mistake. Both credentials start sk-ant-, so the confusion is easy
to make and hard to spot.
3. provider_alias_for's GLM/Kimi -> anthropic.default fallback was documented
as deliberate but was silent in effect: a user picking "kimi" in the UI got
an agent spending the Anthropic key, with nothing saying so. It now logs
the substitution, and is_exact_provider_match() lets callers tell a real
family match from a substitution so a UI can say which model will actually
run. Behaviour is unchanged -- only the silence is.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude Code resolves credentials in a fixed priority order and ranks
ANTHROPIC_API_KEY ABOVE the subscription's CLAUDE_CODE_OAUTH_TOKEN.
mission_runtime forwarded that key into every per-mission container
unconditionally, so on a runtime authenticated with `claude /login` the key
would silently win: `claude` still works, agents still run, and every mission
bills the API while appearing to use the subscription. There is no error to
observe -- the only symptom is the invoice.
CLAWMATES_RUNTIME_AUTH = subscription | api_key now gates the forward list.
In subscription mode ANTHROPIC_API_KEY is withheld; Gemini/Groq/OpenAI still
forward in both modes since they have no subscription equivalent. The mode is
logged per container so it is visible in the deploy log rather than inferred.
Default is api_key -- today's behaviour exactly. An unset or misspelled value
falls back to it too, because defaulting to subscription on a typo would strip
the key and leave missions with no credential at all.
forwarded_provider_keys() is the single source for the list, called by both
ensure_container and the tests, so the two cannot drift -- the failure mode
here is invisible, which is precisely when duplicated knowledge is worst.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Three related bodies of work.
Container reaping: every teardown path now funnels through purge_agent, and
the orphan sweepers see node-placed containers instead of only the local
engine (the cause of 144 accumulated orphans on one node).
Prompt ablation, judged against a current frontier model: skills are indexed
and fetched on demand rather than inlined at ~900 tokens each; standing
behavioural instruction is no longer injected; the INT-XX marker contract is
stated where mission turns actually see it. Dead scaffolding and fabricated
capability cards removed.
Missions as workflows (W0/W2/W3 + goal UI): the workflow registry is wired so
phase config reaches the database at all; phases can carry a `done_when`
condition judged after each pass, iterate with the verdict's reason as
guidance, and surface every verdict in the UI. Evaluation is fail-closed --
an unparseable or missing verdict means not done.
Still open: model-authored plans (W1), plan viewer and planner workflow mode
(W5.1/5.5), mission scheduling (W4).
Two defects in the goal-condition work, both found while tracing a
research->coding mission end to end.
1. Setting a condition silently dropped the recipe's phase config.
phases_for_create treated a caller-supplied config as a wholesale replacement.
The wizard sends {done_when, max_iterations} as the entire config, so every
other recipe key was discarded. Harmless for research_and_code, where nothing
reads `produces` or `default_topology` -- but a conditioned security_hardening
phase lost its `tools` list, which security_scan.rs DOES read, so the scan
would run with nothing configured and report clean. A green security scan that
scanned nothing is the worst possible failure mode for that feature.
The recipe is now the base and the caller's keys override individually.
Shallow merge is deliberate: phase config is a flat settings bag, and a caller
sending `tools: [...]` means to replace the list, not union it. A non-object
override still replaces outright rather than silently picking a side.
2. One condition was applied to every phase.
The wizard had a single mission-level "Done when" that got copied onto all
phases. For research->coding that is actively wrong: "cargo test reported 0
failures" cannot hold while the research phase is running, so research would
burn all its passes and give up before coding ever started. Conditions are now
per phase, keyed by order_idx, with a per-kind placeholder that demonstrates
the rule that actually governs whether a condition works -- it must be
provable from what the agents wrote, because the checker cannot run commands.
Phases with no condition are sent unchanged, so they keep the recipe's
settings and finish in one pass exactly as before.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Makes the completion evaluator usable and observable.
- GET /api/missions/{id}/phases/{phase_id}/evaluations returns every verdict
for a phase, newest pass first, scoped like the summary endpoint.
- MissionPhase gains done_when / max_iterations / iteration, so the phase card
can show what the phase is working toward and which pass it is on.
- PhaseStatus gains 'evaluating' (amber) -- the state between "runs finished"
and "phase done" that only conditioned phases enter.
- New PhaseGoalStrip renders on the phase card, and renders NOTHING for phases
without a condition so unconditioned missions look exactly as before. It
polls only while the phase is running or being judged.
- Mission wizard step 2 gains the condition + a max-passes field.
Two deliberate emphases in the UI:
The evaluator's `reason` is the most prominent element, because it is both the
explanation of why a phase iterated and the literal text handed back to the
agents as guidance -- it is what tells an operator whether the condition is
written well.
The hint copy states the constraint that actually governs whether a condition
works: the judge cannot run commands, it only reads what the agents wrote, so
the condition has to be provable from their output. "cargo test reported 0
failures" works; "the code is well factored" does not. Getting this wrong is
the difference between a phase that converges and one that burns every pass.
An evaluator error is rendered distinctly from a negative verdict, so a judge
outage doesn't read as a judgement on the work.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
A phase used to complete when its topology_runs reached a terminal state --
purely structural. It marked itself done whether the agents produced the
artifact or wrote nothing at all, and it ran exactly once: execute_resumable's
skip(start) is resume, not repeat, and the only re-run path was a human
hitting the retry endpoint.
A phase can now carry `done_when`, a completion condition judged after each
pass against the evidence the agents actually surfaced. Not met and passes
remain -> the phase goes back to pending with iteration bumped, and the
verdict's reason is appended to the next pass's task text. That feedback is
what makes iteration converge rather than repeat -- the same mechanism /goal
uses, and that swarm.rs already uses for rejected work.
The evaluator runs on the SUBSCRIPTION model. CLAWMATES_EVALUATOR_MODEL
defaults to judge_model(), and a `runtime:<alias>` spec routes through
ZeroClawDriveExecutor -- a container agent on claude_cli, i.e. Claude Code on
the OAuth subscription, needing no platform API key. Same routing the door
governor uses.
Two deliberate departures from the governor's contract, both required:
- FAIL-CLOSED. Runtime::judge is fail-open and reads a verdict by
!contains("DENY"), so a model explaining why it *would* deny reads as
approval and an empty reply reads as approval. For completion that is
backwards: unsure must mean not done. The contract is swarm.rs's strict
JSON {"met","reason"} with .unwrap_or(false). Six tests cover the closed
paths -- prose, empty, missing field, non-boolean, transport error.
- judge_raw returns the raw reply; judge collapses to a bool too early to
carry a structured verdict.
Iteration scoping is the subtle part and has its own test: on pass 2 the
phase's own iteration is 1 but pass 1's completed run is still in the table,
so "are this phase's runs all finished?" must ask about the CURRENT pass or
that stale row closes out pass 2 the instant it is enqueued.
Evidence comes from phase_summarizer::collect_evidence, extracted from the
existing collect_material so the evaluator and the summary card cannot
disagree about what a phase produced.
done_when/max_iterations are promoted from phase config into columns (the
sweep filters on them every tick) and max_iterations is clamped to 20 at
insert -- the UI limits it too, but a runaway loop must not be one crafted
request away.
A phase with no condition completes exactly as before; that regression guard
is the first test in the file.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
workflow_registry.rs had zero call sites -- lib.rs declared the module and
nothing ever called load() or get(). So templates/workflows/*.toml was never
read, and because the client's TEMPLATE_PRESETS carries only {kind, order_idx}
with no config, PhaseSpec.config defaulted to Value::Null and every
wizard-created mission stored a null mission_phases.config.
Every per-phase setting was therefore inert. `loop = "until_no_more_int_items"`
and `commit_policy = "on_green_tests"` described a scheduler that does not
exist AND had no path to the database. benchmark_runner and security_scan
already read phase_config(); they were reading from null.
- Mission create derives phases from the recipe when none are sent, and
backfills config per phase (matched on kind+order_idx, then kind) when the
caller sends shape without config. An explicit config always wins.
- phases_for_create takes Option<&WorkflowRecipe> rather than reaching for the
global, because the registry resolves its directory relative to the process
cwd -- which under cargo test is the crate root, not the repo root.
- GET /api/workflows serves the catalog; the wizard fetches it and falls back
to TEMPLATE_PRESETS. Adding a TOML now adds a template with no FE change.
- load() runs at boot so a malformed recipe appears in the boot log instead of
silently producing a mission with no phase config.
Also fixes a latent bug in all five recipes: `default_team_template` was
written below the first [[phases]] block, and TOML scopes a bare key after a
table header INTO that table -- so it parsed as
phases[last].config.default_team_template and the real field was always None.
Invisible while the registry was dead code. Moved above the phases, with a
test asserting it neither returns None nor leaks into a phase config.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
TopologyKind describes twelve distinct intents, but the orchestrator
implements five planners and mapped the kinds onto them inside plan_steps.
So Market never auctions, StarMoe never routes to experts, Ring never cycles
and Holacratic never self-organizes -- each silently runs as whichever pattern
it collapses to, while kind::description() and the UI catalog kept promising
the distinct behaviour.
Rather than delete variants that appear in persisted rows, the collapse is now
named: ExecutionPattern + TopologyKind::execution_pattern() in cm-topology,
with plan_steps dispatching on the pattern instead of re-listing the mapping.
One source of truth, and the two cannot drift.
GET /api/topologies now reports `executes_as` and `distinct_at_execution` so a
UI can stop offering aliases as if they behaved differently.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
default_risk_profile_for_role decides whether a claw gets file edits, git and
shell by substring-matching its role against a fixed keyword list. On the
planner path that role string is free text the model invented for this
proposal, so a model's choice of wording silently decided tool access: a
proposed "implementation_lead" matches no keyword, lands research_readonly,
and then fails every file edit for a reason invisible from the role name.
TeamMemberInput and the planner's member schema now carry `needs_write`, and
resolve_risk_profile prefers it over the guess. The planner prompt asks for it
per member and says to grant write only to members that produce code or
commits. Absent (older clients, autoprovision, a model that omitted the field)
falls back to the old guess, so nothing changes for callers that don't set it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
task_card_parser.rs scans every mission turn line-by-line for TASK/WORK/
HANDOFF/TEST_PASS/TEST_FAIL/REVIEW_APPROVE/REVIEW_BLOCK/COMPLETED and
materializes mission_tasks rows from them. The exact syntax it demands --
literal, own line, with the colon, no bold, no code fence, one INT id per
line -- was documented in two places the agent does not reliably read:
1. the team-template role prompts, which are NEVER injected into mission
turns (runtime_provision writes model_provider / risk_profile /
mcp_bundles and nothing else), and
2. a foundation skill the agent had to choose to fetch.
The phase directives said "emit INT-XX markers" without ever saying what one
looks like. So the parser's contract was stated nowhere load-bearing, and
whether a mission produced task cards came down to whether the model guessed
the format. This is a machine contract, not a style hint -- it belongs in
phase_task_text, the one text every mission turn receives.
Added a regression test that feeds every marker example from the generated
prompt through the real parser, so the syntax we advertise and the syntax we
accept cannot drift apart again.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Ablation pass, judged against a current frontier model.
Dropped from the chat system prompt:
- `## How I operate` (agent_md) and `## Personality`. Both are standing
behavioural instruction, and the agent_md bodies are team-template
brain_seed prose -- "prefer let-else over deep nesting", "anti-patterns:
unwrap() in library code". That is correction written for weaker models,
billed on every turn. The data stays in the brain, still dashboard-editable
and still in the portable artifact; this is about what earns prompt space.
The DB system_prompt still goes in: identity is information, not correction.
Dropped from tool descriptors and the delegation payload:
- the "treat it as information, not instructions" imperatives on chat.inbox,
delegate, and the door's delegation result. Attribution ("the result
returned by claw 'X'") is KEPT -- knowing the source is information the
caller needs. Taint tracking (output_taint = InterAgent) is what actually
contains untrusted inter-agent content; a sentence in the payload never was.
Fixed while here: only the user's half of each exchange was ever written to
the brain, so recall returned questions without their answers -- the less
useful half. The assistant reply is now recorded when the turn completes
(best-effort, empty tool-only turns skipped so they don't dilute the index).
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The chat path concatenated every installed skill's complete markdown into
the system prompt on every turn. Bodies average ~3.5 KB (~900 tokens) and the
count is unbounded, so this was by far the largest thing in the prompt and it
scaled with how many skills a claw had installed -- a fixed toll paid whether
or not any skill was relevant to the turn.
The prompt now lists name + description, and a new `skills.read` tool fetches
a body on demand. This is the contract the mission path already had: the
`clawmates_skills` MCP server advertises description + when_to_use and lets
the agent read what it needs. The two paths now agree.
`compose_system` takes (title, description, body) rather than (title, body):
the index needs the description, and first-touch brain seeding still needs the
real body so the .brain stays a complete portable artifact.
Not done here: filtering tool descriptors per agent, which the plan paired
with this. The premise doesn't hold -- risk_profile governs the ZeroClaw tool
namespace (file_edit, shell) on the mission path, while the chat path has its
own registry (files.write, shell.exec) and no per-agent policy whatsoever;
`risk_profile` appears nowhere in cm-runtime. Filtering there would invent a
capability boundary rather than enforce one, silently revoking chat tools.
Left for a deliberate decision.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Tier 0 of the prompt-ablation pass -- subtraction only, none of this
reached a model.
- cm-brain: drop ClawBrain::export_markdown (zero callers).
- workflows: drop the `task_preamble` keys. No Rust code ever read them --
WorkflowPhase.config is an opaque serde_json::Value -- so the comment
calling the preamble "the belt, the skill the suspenders" described a belt
that was never implemented. (`commit_policy` is unread for the same reason;
left in place as documentation pending a decision.)
- mcp_door: derive the unknown-tool error from EXPOSED_TOOLS. The literal had
drifted to naming one of the three tools the door exposes.
- Dashboard.tsx: drop TEAM_TEMPLATES/COMPANY_TEMPLATES, defined and never
referenced, and disconnected from the real templates/teams/*.toml.
The substantive one: GET /api/claws/{id}/compartments returned hardcoded
strings for tools/capabilities/safety, identical for every claw. Every card
read "Network: none" and "Shell . blocked" regardless of the claw's real
risk_profile -- which is the actual capability boundary, so the card was
most wrong exactly where it mattered, on a coding_readwrite claw that does
have shell. Now derived from the claw's effective risk_profile (its team's
setting, else the same role-derived default the provisioner applies), with
the allowlists mirroring [risk_profiles.*] in the runtime config.
Note: cm-topology/src/heuristics.rs was slated for deletion here as unused.
It is not -- routes/topology.rs:43 serves it and p0_endpoints.rs:302 asserts
it. Left alone.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Agent containers leaked two independent ways.
1. The four-step teardown (deprovision ZeroClaw -> reap_sandbox -> unlink
.brain/.onion -> hard_purge) was inlined at three call sites and two had
drifted. missions.rs::reap_mission_resources skipped reap_sandbox;
topology_worker::maybe_teardown_ephemeral_team skipped it and the brain
unlink; DELETE /api/claws/{id} (soft delete) released nothing at all, so an
offline claw that can never run again kept its container and bind mount
forever. All four now funnel through claws::purge_agent, with
release_claw_resources for the soft-delete case (containers gone, rows kept).
2. Both orphan reapers listed only the local driver, so a container placed on a
fleet node was invisible to the only backstop that could find it -- this is
what accumulated 144 tc-agent-* orphans on one node. NodeDriverProvider gains
node_ids() (backed by NodeHub::online_ids) and both reapers now sweep every
connected node. The remote sweep is TTL-only on purpose: the boot pass runs
with Duration::ZERO and would otherwise kill a container another instance is
mid-provision on.
Why it was invisible: agent_containers.agent_id is ON DELETE CASCADE, so
hard_purge took the registry row with the agent and left the container
permanently unreferenceable.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Deploys were verifying green and then silently reverting minutes later.
Cause: gw-04 does not deploy from this script's recreate at all.
`clawmates-deploy.timer` runs every 60s, pulls
`$REGISTRY/clawmates/<svc>:latest`, and rolls the stack onto it whenever
the running image differs — so the local `docker tag` + `--force-recreate`
this script did was reverted within the minute. Its own log shows it:
server drift: running=<the new image> target=<the old :latest>
rolling: server frontend
The registry's `:latest` is therefore the only thing that decides what
prod runs — and `docker push …:latest` does NOT reliably move it here.
When the manifest already exists under another tag (the `main-<sha>` we
push immediately before), the push reports a digest but `:latest` keeps
resolving to the old image. Pushing a brand-new tag works, so it is
specific to overwriting an existing one.
Writing the manifest to the tag over the registry HTTP API does move it
(GET the main-<sha> manifest, PUT that body to :latest → 201), after
which the timer converges prod on its own. So:
- repoint :latest via manifest PUT from the build host, failing loudly on
a non-2xx instead of assuming the push landed
- roll gw-04 immediately rather than waiting up to 60s for the timer
- verify against the resolved :latest (what compose and the timer both
deploy from) instead of a main-<sha> tag that is never pulled there
Note for future debugging: image IDs differ per host for the same tag
(buildx OCI index — tank holds the index digest, gw-04 the resolved
platform image), so the trustworthy check is grepping the deployed binary
for a string only the new code contains.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The mission page made its own output unreadable. Reviewing a research
brief meant scrolling a 300px <pre> nested inside a 260px run box nested
inside the page scroller (plus a 4th scroll region for the description) —
and the text was capped at 6,000 chars server-side with no way to fetch
the rest, so a 53kB brief showed ~11% of itself and silently dropped the
remainder. Eight flat tabs (overview/phases/tasks/team/live/artifacts/
benchmarks/pane) mixed lifecycle, work items, people, telemetry, outputs
and infra at one level, so nothing indicated where the deliverable lived.
Reader:
- GET /api/missions/{id}/documents lists every agent output (titles +
sizes, no bodies); GET .../documents/{run_id}/{index} returns one in
full. Scoped to the mission so a run id from elsewhere can't be read.
- MissionOutputReader: rail (documents grouped by phase) · document ·
outline (headings, click to jump). Exactly one scroll container per
column, never nested. Copy + download .md.
- MarkdownBlock gains fenced code blocks (agent output is full of ```rust,
previously mangled into paragraphs), h4-h6, heading anchors, and an
outlineOf() helper.
Information architecture:
- Three primary tabs with shallow sub-views: RUN (phases/tasks/live) ·
OUTPUT (documents/artifacts/benchmarks) · SETUP (overview/team/pane).
- PhaseRunsList shows a short excerpt with no inner scrollbar and points
at the reader for the full text.
- The header description is clipped, not scrollable; its full text now
has a home in Setup → Overview.
Missions list:
- /api/missions returns MissionListItem — Mission flattened plus
phases_total/phases_done/current_phase, so the JSON stays a strict
superset. Cards render a progress bar and "Coding · 1/2" instead of a
bare status dot.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
`docker-compose pull server frontend` pulls `:latest`, and the registry
served a STALE manifest for that mutable tag: a deploy pushed
`main-9bc5f6a` correctly, but the gateway's `pull :latest` reported
"image is up to date" and left the previous image running. The verify
step caught it (running 9f2349 = main-0a647c0, expected bbf19f7e), so
the deploy failed loudly rather than silently — but it still could not
ship.
Immutable tags always resolve correctly, so pull `main-<sha>` and retag
it to `:latest` locally on the gateway, then recreate with `--no-deps`
and no compose pull. `:latest` is now just a local alias satisfying the
compose file's image reference; the sha tag is the source of truth.
Also switch the recreate to `--no-deps` (compose v1 has no
`--no-recreate-deps`) so a server/frontend deploy stops recreating
postgres.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
`inject_node_agents` wrote the claw alias as a top-level `"agent"` key on
each graph node, but `cm_topology::Node` only deserializes `{id, role,
level, attrs}` — serde silently dropped it. `TurnRequest::agent` came back
`None` and every mission turn fell back to `ZEROCLAW_DEFAULT_AGENT`
(`scout`), running with scout's workspace and tools instead of the
mission's claws. The runtime trace confirms it: every turn logged
`"agent_alias":"scout"`.
That is why mission agents reported an "empty greenfield" workspace and
emitted artifacts inline instead of writing them: scout is jailed to
`/zeroclaw-data/.zeroclaw/agents/scout/workspace` and cannot see
`/mission/repo`. The per-mission provisioning and `workspace.path` pinning
shipped earlier were correct — they were just applied to agents that
nothing ever drove.
- bind into `node.attrs["agent"]` (top-level key kept for display/debug)
- extract the DB-free `apply_node_agents` and add a regression test that
round-trips through the real `TopologyGraph` deserializer, which is the
guard that was missing
- log loudly in `topology_exec::run_turn` when a node falls back to the
default agent, instead of silently swapping in a different agent
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
rsync --delete excludes frontend/public/dl/, so the directory does not exist
on the build host and the `cp` of clawmates-node into it aborted the deploy
(set -e) before anything was pushed.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Mission turns execute against the per-mission runtime container, but claws
were provisioned via RuntimeProvisioner::from_env() — i.e. the GLOBAL gateway.
That daemon loads config once at boot and never re-reads the file, so the
per-mission daemon had no claw_* agents at all: querying it for a mission
claw's risk_profile returned 404 while the global daemon returned 200. With
the alias unresolvable, the daemon silently fell back to the default `scout`
agent, which is jailed to the global workspace — agents reported "the scout
agent workspace" and "/mission/repo isn't accessible", produced no files, and
burned tokens. This is the deeper cause behind the empty-output runs; the
tool-allowlist and workspace-pin fixes were necessary but not sufficient.
- RuntimeProvisioner::for_gateway(url) — aim the provisioner at a specific
gateway (mirrors ZeroClawDriveExecutor::from_env_for_gateway); from_env now
delegates to it.
- mission_orchestrator captures the per-mission endpoint from ensure_container
and provisions every claw there, falling back to the global gateway only
when there is no per-mission runtime (dev/no-docker).
- workspace.path is file-only (the config prop API cannot set a PathBuf), and
the daemon never re-reads the file, so pin_agent_workspaces is now followed
by restart_container(): restart + wait for /health to answer. Agents created
through the daemon's own config API are already persisted to that file, so
they survive; the pairing code is re-minted on every launch.
The readiness probe inspects the /health BODY — exec_capture only fails on
docker errors, so a curl that cannot connect still "succeeds".
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The gateway compose pulls server/frontend from the web-01 registry
(100.94.185.103:5000, tags main-<sha> + :latest), so `docker save | docker
load` + a local retag does NOT stick — the next `docker-compose up` pulls
`:latest` and silently reverts to the last-pushed image (a green edge on the
old image hid this). Rewrite the server/frontend path to: build on the build
host → tag :latest + :main-<sha> → push to the registry → `compose pull +
up --force-recreate` in /opt/clawmates (the real project dir, not the stale
/root/clawmates) → verify the RUNNING image id equals the pushed one (fail
loudly on mismatch instead of trusting HTTP 200). Agent :dev images stay on
the save|load path (not in any registry).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
DELETE /api/missions/{id} was a bare `DELETE FROM missions` relying on FK
cascades that only cover mission-owned tables. Everything the mission
provisioned leaked: per-mission runtime container, host workspace dir,
teams (created lifecycle=permanent, so no cascade + skipped by the
ephemeral-teardown path), and every claw's ZeroClaw config, .brain files,
and DB rows. Observed live with 0 missions in the DB: 174 orphaned gateway
claw configs, 7 orphaned teams, 31 agents, 39 .brain files, 6 workspace
dirs, a 4-day-old orphaned container, and 123 detached topology_runs.
delete() now calls reap_mission_resources() before the row delete:
- resolve the mission's teams (mission_teams) → claws (team_members)
- per claw: deprovision_claw (gateway) + rm .brain files + hard_purge (DB),
reusing the manual agent-reap pattern in routes/claws.rs
- delete the permanent-lifecycle teams (team_members cascades)
- delete the mission's topology_runs (else they linger with mission_id
nulled by the cascade and accumulate)
- teardown_container(), now extended to also rm the /mission/repo workspace
dir and tolerate an already-gone container (idempotent for the sweeper +
delete paths)
Runtime-side steps are best-effort (Postgres authoritative; fleet sweeper
reconciles daemon config); DB purges are logged on failure but never block.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The server COPYs templates/ and skills/ then drops to USER 65532. When the
build context arrives with mode-700 dirs (e.g. rsync -a preserving a dev's
local perms), COPY bakes 700 into the image and the nonroot runtime user
can't read them — the skills/team-template builtin seed silently skips
("Permission denied (os error 13)"). chmod -R a+rX after the COPYs makes
the seed dirs readable regardless of source perms.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Mission agents were burning ~275K tokens producing nothing: the coder had
only file_read and its workspace was the empty ephemeral sandbox, so it
dumped a full spec inline instead of writing files. Two root causes:
1. Risk-profile allowlists used pre-0.8 tool names. `coding_readwrite`
allow-listed `file_write` (renamed to `file_edit` in ZeroClaw 0.8, and
`file_write` now refuses on ephemeral workspaces) and omitted file_edit
/ content_search / glob_search / git_operations — the exact tools the
phase prompt tells agents to use. Since allowed_tools is a strict
allowlist, agents were effectively read-only. Documents the correct
profiles in agent.config.example.toml (they only lived in host config;
the live runtime profiles were corrected via its config API).
2. workspace.path never got set. `agents.<alias>.workspace.path` is an
Option<PathBuf> the ZeroClaw Configurable macro skips from prop
enumeration, so provision_claw's set_prop always 404'd and the whole
call errored into a swallowed eprintln. Removes the dead set_prop and
pins the workspace out-of-band: MissionRuntimeProvisioner::
pin_agent_workspaces patches the shared config file on the per-mission
container (format-preserving via toml_edit, atomic temp+mv); the daemon
applies it on the same reload that surfaces the freshly-provisioned
claws. Covered by unit tests for the TOML stamp.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Two stacked issues after risk_profile was fixed:
1. Claws had file_edit + 46 other tools available, but the templates
trained the agents to expect file_read/file_write (older ZeroClaw
tool names). Result: agent output kept saying "I only have file_read"
and dumped implementations into the context window as text.
2. Even with file_edit, the sandbox pointed at
/zeroclaw-data/.zeroclaw/agents/<alias>/workspace/ — NOT
/mission/repo where the checked-out mission repo actually lives.
unrestricted_filesystem=false blocked agents from reaching it.
Fixes:
- provision_claw now takes workspace_path. mission_orchestrator passes
/mission/repo — pins the per-claw workspace via
agents.<alias>.workspace.path to the bind-mount path so file_edit /
content_search / glob_search operate on the mission's git checkout.
- phase_task_text prepends an explicit tool inventory (file_edit,
content_search, glob_search, git_operations, git_forge, ...) plus a
WORKSPACE line pinned at /mission/repo. Each phase directive is
rewritten to reference file_edit / git_operations explicitly and to
call out "do NOT paste code in your reply expecting the platform to
save it."
The provisioner was hardcoding risk_profile=toolfree for every claw,
which the ZeroClaw config explicitly configures to EXCLUDE every
usable tool (shell, file_read, file_write, http_request, browser).
Result: coder/tester/committer claws had zero tools and produced text
in the context window with no ability to actually write files or run
tests — exactly what the last mission summary showed.
Fixes:
- provision_claw now takes risk_profile: &str, passed through from
the team template (development teams already had coding_readwrite,
which now actually gets applied).
- Research team templates updated from toolfree → research_readonly
(file_read) and papers_research → research_web_readonly
(file_read + web_search + web_fetch). Applied to both the on-disk
TOML files and the live DB rows.
- Added RuntimeProvisioner::default_risk_profile_for_role for
auto-provision code paths that lack a template context — picks
coding_readwrite for coder-like roles, research_readonly otherwise.
- Split rebind_model out of provision_claw so the model-change UI
path doesnt inadvertently clobber the existing risk_profile.
Templates DB fixup for missions launched pre-deploy is already
applied via manual UPDATE.
- RunOutputPanel: each turn now renders as <details> with a 1-line
peek in the summary. First turn open by default (so operators see
something without a click), subsequent turns collapsed. Same shape
applies to research + coding runs (shared component).
- PhaseSummaryCard: click the header to collapse the whole card;
narrative peek shows in the collapsed state. State persisted per
phase_id in localStorage so it stays remembered across visits.
- PhaseSummaryCard Section: cap max height at 280px with internal
scroll so long tooling / sources / next-action lists dont blow
out the card height.
- Mission header title/description now collapsible via chevron next
to the title. Persisted in localStorage so it stays hidden across
mission switches once the operator has read it — clears more room
for phases/tasks/team panels below.
- Tabs row: overflow-x auto + per-tab flex:none + whiteSpace:nowrap
so 8+ tabs scroll horizontally instead of wrapping and cutting off.
- Phase card action row (retry/security/benchmark buttons): flexWrap
wrap so long button rows stack cleanly instead of overflowing.
- Phase card status row wraps too, and the card itself gets
overflow:hidden + minWidth:0 so long content stays inside the
border and the parent tab-panel scroll handles vertical growth.
New phase_summarizer background worker fires on any mission_phase
transition to a terminal state (completed/failed). Aggregates every
topology_runs.checkpoint.outputs[] + mission_tasks + mission_artifacts
bound to that phase and asks Claude Opus 4.8 to produce a structured
JSON card:
{ narrative, metrics, sources, tooling, next_actions }
Rendered inline on the mission page under each completed phase via
new PhaseSummaryCard component. Metrics grid is kind-specific:
research surfaces insights/sources/int_cards/artifacts, coding
surfaces cards_picked_up/commits/tests/issues, benchmark surfaces
regressions/improvements, security surfaces findings-by-severity.
New table: mission_phase_summaries (migration 0060), unique per
phase_id — regenerates on retry.
New endpoint: GET /api/missions/{id}/phases/{phase_id}/summary.
Model overridable via CLAWMATES_SUMMARIZER_MODEL. Reuses the
ANTHROPIC_API_KEY prod already carries for mission_refiner.
Deprecated ZEROCLAW_WORKSPACE env var (schema.rs:17467) is used by
the daemon as a legacy config-dir pointer that overrides everything
else. Setting it to /mission/repo made the mission daemon compute
its config dir as /mission/repo/.zeroclaw (empty) and fall back to
defaults — zero agents loaded.
This is the actual root cause of Unknown agent errors on WS. The
seed-mount + admin/paircode/new + per-node-agent-injection fixes
we shipped earlier were correct but couldnt take effect because
the daemon wasnt reading our bind-mounted config at all.
Per-agent workspace pinning belongs in config.toml as
agents.<alias>.workspace, not env.
The topology graph shipped from team.graph only carries node.role,
not node.agent. The executor then defaults to alias_for(role) which
falls to ZEROCLAW_DEFAULT_AGENT (scout) — no such agent → 400.
Look up team_members(node_id → claw_id) at enqueue time and stamp
node.agent = claw_<hex> onto every node. Executor now dials the
specific claw provisioned for THIS teams role.
Was masked pre-C3 because the shared runtime hit the same 400 —
never noticed because no one clicked through to a real run there.
Pairing codes are single-use / expiring — a mission that reuses an
existing runtime container on a retry needs a fresh code, not the
stale one from the initial launch. Drop the runtime_endpoint gate
so ensure_container always fires, and its fast path re-mints via
/admin/paircode/new for existing containers.
Fresh runtimes had zero agents in their config so WS handshake with
?agent=scout returned 400. Bind-mount the shared runtimes data dir
so per-mission gateways inherit the seeded claw_* agents.
Per-mission pairing (minted via /admin/paircode/new) still works
against the shared devices.db — each mission gets its own accepted
token. Concurrency caveat on sqlite sessions.db documented in the
const doc comment.
Fresh gateways sometimes boot claim-ing already paired (no
pairing_code in the log banner), which broke the log-scrape approach.
Instead, docker exec into the container and hit the localhost
/admin/paircode/new endpoint that always mints a fresh one-time
code and returns JSON we can parse.
The seed-mount approach didnt work: even with the shared runtimes
data dir bind-mounted, a fresh gateway instance mints a new pairing
key and requires re-pairing. The topology_worker connect returned
401 forever.
New approach — per-mission gateways self-pair:
- Provisioner tails container logs after start, extracts the
X-Pairing-Code from the boot banner
- Persists it on missions.runtime_pairing_code (migration 0059)
- topology_worker constructs ZeroClawDriveExecutor with THAT code
via from_env_for_gateway_with_code, which triggers the lazy
/pair handshake on first turn and caches the returned bearer
Drops the shared-runtime data-dir mount — each per-mission gateway
now owns its own state, restoring the C3 isolation guarantee.
Removes refreshClicks counter + console.log now that the fetch-hang
was root-caused (fetch: cache no-store) and fixed. Keeps the
updated-at timestamp indicator as ongoing visual feedback.
Fresh mission runtime containers had no ZEROCLAW pairing token so
the topology_worker got 401 Unauthorized on WS connect. Mount the
shared runtimes /root/clawmates-runtime/data as /zeroclaw-data so
the gateway boots pre-paired and accepts the servers ZEROCLAW_TOKEN.
Seed dir overridable via CLAWMATES_RUNTIME_SEED_DIR.
Known caveat: sqlite sessions dir is shared across concurrent
mission runtimes. Fine while topology_worker runs sequentially per
mission; next iteration should copy-on-write per-mission.
fetch(url, { cache: no-store }) was hanging forever through the
edge proxy on the mission API endpoints — requests never reached
postgres and the client-side loading state was stuck true, making
Refresh appear broken. Regressed in d42398d.
Switch to a per-request _t=Date.now() query param on GETs — same
cache-defeat effect, doesn't change fetch semantics.
The refresh button was suspected of being inert when loading is
somehow stuck true. Removes disabled and renders a bulletproof
click counter + loading state next to the icon:
clicks:0 · updated 14:05:12 · idle
- clicks bumps SYNCHRONOUSLY in onClick before any await, so a
non-zero counter proves the click event reaches the handler
- console.log fires alongside for devtools verification
- disabled={loading} removed; if load happens to hang, at least
the user can click again to retry
Temporary scaffolding — will collapse once the root cause is clear.
- api client: cache: no-store so manual Refresh guarantees a fresh
server response (was potentially hitting stale HTTP cache).
- MissionCanvas: renders "updated HH:MM:SS" next to the refresh
button; the timestamp bumps on every successful load so the click
is visibly acknowledged even when nothing else on the page changed.
UUIDv7 encodes time in the leading bytes so 12-hex prefixes are
NOT unique across missions minted in the same second. Docker
accepts up to 253 chars; use the full uuid.
