daf8d121578756f8f255e180bc7ecb333e7ed0c5
25
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
2f1a870949 |
feat(skills): a skill that must be read cannot be left to be noticed
The `index` arm hands an agent a list of uris and trusts it to fetch what applies. Measured on the first A/B pair, that is mostly what happens — each agent fetched the skill bound to its own role and no other, which is the result that made Trigger observable at all. `workspace-repo-commit-protocol` is the case it fails on. It scored Trigger=FAIL beside a PASSING boundary check: the rule was live and unread. A procedure that applies to everyone who writes reads as nobody's in particular, so no agent recognises it as theirs and no agent fetches it. Upstream ZeroClaw arrived at the same place from the other direction and gave its compact injection mode an `always: true` frontmatter escape hatch (#9520). This is that hatch as a column: `skills.always_inject`, default FALSE, so nothing changes for an existing skill and the inline arm is untouched either way. Two halves, because delivering it and scoring it are different mistakes: - Delivery: under `Index`, an `always_inject` skill renders its BODY. - Scoring: the arm belongs to the PROMPT and `always_inject` belongs to the SKILL, so the scorer now asks per skill which one it got. A skill whose body is in the prompt was handed over, and a Trigger miss cannot be charged against an agent that was never asked to fetch anything. `skill_was_indexed` reads that off the rendered prompt via `READ_IT`, a constant now shared with `index_entry` — two spellings of one marker is how a detector quietly stops detecting. Suite: 108 binaries, 840 tests, green. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
f52cff3e04 |
feat(skill-use): progressive disclosure, as an arm and not a switch
Trigger — did the agent reach for the skill when it applied? — cannot be measured while every body is inlined into the prompt. Nothing was reached for. `skill_use` has been reporting `NotObservable` for that reason, and it was right to. The skills door made retrieval possible; this makes it a delivery arm. `index` sends each pinned skill's name, description, `when_to_use` and the uri that returns its body, and the agent fetches what it judges relevant. `inline` is unchanged and stays the default. An A/B rather than a switch, because `index` can only cost Compliance: under `inline` the procedure sits in front of the model whether or not it noticed it applied. Trading a measured axis for an unmeasured regression in another is not an improvement, so both arms stay runnable and the arm is recorded on the mission row. Three things the mechanism refuses to do: - `index` without a door falls back to `inline`. An index names bodies and says how to fetch them; with no `clawmates_skills` server reachable that is a list of dead ends, and it fails as an agent ignoring its skills rather than as a missing config. `install_skills_door` now returns whether it installed, because the caller needs the answer and not just the log line. - The scorer reads the arm off the recorded PROMPT, not off the mission row. The row says what the mission is configured to do now; the score is being computed against a turn that ran then. - Under `index`, a skill that was offered and never read is a Fail, not the inline arm's `NotObservable` — but only where the skill had a checkable consequence in that phase. Reusing the inline text would have said "this skill was inlined into the prompt" about a skill whose body was never sent, and scoring a real miss as a structural blind spot is the failure this measurement already made once. The arm is per mission (`config.skill_delivery`), not only per deployment. Both arms run against one server process; restarting between them would put a confound in the comparison that the numbers would not show. 829 tests, 108 binaries, green. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
ea0b989b3f |
docs(research): missions DO call tools — the claim was wrong, and the truth is worse
Deep research into "missions can't call tools at all", which I wrote and which is false. docs/TOOL-CALL-ARCHITECTURE.md has the full findings. WHAT IS ACTUALLY TRUE Three of the four mission paths end in `claude -p` with Claude Code's own toolset and permissions PRE-ACCEPTED: solo microVM Read Edit Write Bash Agent --permission-mode acceptEdits composed microVM same, per node same direct session Read Edit Write Bash acceptEdits So the position is not "no tools". It is: mission agents run Bash and Write with permissions pre-accepted, and nothing in this platform can gate them. That is a stronger finding than the one it replaces — "can't call tools" sounds like a missing feature; "calls tools freely, ungated, and mostly unobserved" is a security posture, and it is ours. Observe and gate are different and both are partial. vm_tool_tap is a PostToolUse hook: it fires AFTER the tool ran and exit-0s unconditionally, so it is telemetry and structurally cannot gate. The direct-session tier has no tap at all. GatePolicy has exactly one enforcement site — the chat loop — and its approvals key on (session_id, message_id), which no mission phase can produce. WHY THE CONTAINER TIER LOOKED TOOL-FREE `claude_cli` runs `claude -p --output-format json`, which returns a single final result object, and the provider hardcodes `tool_calls: Vec::new()`. The calls happen; the transport discards them. The comment reading that emptiness as "§15 by construction: agents are provisioned tool-free" was inferring a design property from a serialization choice. Verified against the deployed Claude Code 2.1.228 rather than assumed: `--output-format stream-json --verbose` emits `tool_use` blocks with the tool name and `tool_result` blocks. The calls are fully observable; we ask for the wrong format. THE DOOR WE ALREADY BUILT AND NEVER PLUGGED IN claude_cli.rs is OURS — upstream zeroclaw-labs/zeroclaw has no such file — and so is 88eef99d4 "claude_cli --mcp-config + allow/disallow tools (act via door)". The provider already accepts mcp_config (claude's own MCP client reaches our door), tools, and disallowed_tools (lock out the natives so the gated door is the ONLY actuator). agent.config.example.toml documents the whole shape. In the live runtime: clawmates-mcp.json does not exist, there is no [providers.*] block, and every mission claw binds to claude_cli.default which sets none of it. My earlier "claude_cli cannot reach MCP, therefore the skills server is unreachable" was wrong in its reasoning — the capability is built, documented by us, and never deployed. Related: we set `agents.<alias>.mcp_bundles`, which configures ZeroClaw's OWN MCP client for its native loop. A claude_cli agent's actuator is the claude subprocess, which reads `mcp_config` on the PROVIDER. We were turning a knob wired to a loop that does not run. UPSTREAM 218 commits behind. No upstream work on claude_cli (the file is ours). ACP already exists in the fork; the three new commits are workspace-default and localization fixes, not new capability. The one item worth pulling is "feat(plugins): add shared egress policy foundation (#9137)" — a network guard with DNS pinning and metadata-address blocking, defence for the egress problem we have not solved. Stale claims corrected in place, in topology_exec.rs and the runtime config, so the codebase stops asserting the thing that is false. Recommended order, cheapest first: stream-json for observability; the PreToolUse hook for a real gate (it FIRES under claude -p per vm_stop_gate, and has zero call sites); then deploy the door. The executor swap is NOT recommended — the blockers are structural, not wiring, and the cheap fixes deliver what it was wanted for. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
91a6b4e304 |
feat(skills): the first Skill-Use measurement, and the three defects it found
Scored on the paper's three axes against two real missions on the local
stack. docs/SKILL-USE-BASELINE.md has the numbers, the method, and the
limits.
Trigger is reported as NOT OBSERVABLE, never zero
The paper measures progressive disclosure: the agent sees a name and
description and must retrieve the body, and that retrieval is the Trigger
event. We inline full bodies, because mission claws run on claude_cli which
cannot surface a tool call — there is nothing to retrieve with. So the
agent never reaches for a skill, it simply holds one.
Scoring that zero would report a delivery-model property as an agent
failure, which is the same confusion that kept 55 empty bindings invisible
for months. The verdict type carries NotObservable(reason) as a distinct
case from Fail for exactly this.
Compliance is checked by running the REAL task_card_parser rather than a
copy of its rules — a second implementation would drift, and then the score
would pass while the mission loop still stalled. Skills without a
machine-checkable consequence score not_applicable rather than a guess.
WHAT THE MEASUREMENT FOUND
1. The prompt format made its own record unparseable. Skills were
introduced with `## <name>` and skill bodies are markdown full of `##`
headings, so run 1 scored "Sizing heuristic" and "The output shape" —
subheadings inside decompose-int-items — as skills with no catalogue
row. Now an unambiguous `--- SKILL: <name> ---` marker, with both
writers sharing one renderer so the reader cannot drift from the writer.
2. A prompt was recorded that was never sent. My own Phase 1 work recorded
the phase prompt at the dispatch fork, before the tier was chosen — and
the container tier does not send that text, it sends the bare task and
appends skills per turn. Every container mission logged a `solo` prompt
that reached no agent. A provenance record of something that did not
happen is worse than no record: it is the wrong answer, delivered
confidently. Recording now happens inside each tier, with a test that
every launcher records the prompt it actually sends.
3. int-xx-marker-protocol documents a marker the platform never
implemented. PLAN_COMPLETE is in the skill's ladder and task_card_parser
has no such kind and never has, so an agent following the skill exactly
emits a marker that is silently ignored. Observed live: run 2's planner
emitted `PLAN_COMPLETE: INT-01..02`, which is also the range form — on
the kinds that ARE parsed that yields the id `INT-01..02`, a task card
for an item that does not exist while the two real items stay open.
This is a skill/implementation mismatch, not an agent failure, and it is
exactly what the measurement exists to find: the agent did what it was
told and what it was told was wrong. Both shapes now score as failures.
The reconciliation — implement PLAN_COMPLETE or drop it from the skill —
is left as a decision rather than guessed at.
The boot log now shows what the plan asked for: 53 skills, 11 templates,
every one `N role skills bound` with NO unresolved clause. Live missions
confirm per-role delivery — the planner receives decompose-int-items, the
coder receives write-rust-current-edition.
GET /api/missions/{id}/skill-use exposes the scores, and says in its
payload whether an empty result means "nothing delivered" or "the evidence
was reaped" — those have very different causes and must not look the same.
n = 2. No spread is reported because two runs cannot establish one, and the
document says so rather than letting the number be quoted as a baseline it
is not.
Full workspace suite green: 106 binaries.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
769e002bb3 |
feat(skills): deliver on every tier, record what agents receive, let them self-author
Three phases of the approved plan, plus a correction to what the last one
claimed.
CORRECTION: skills reached ONE tier, not all of them
The previous commit said "skills can now reach a mission agent". That was
true only for the container/ZeroClaw tier — the fall-through that queues a
topology_runs row for topology_worker, which drives the executor that was
patched. compose_turn_prompt/pinned_skills_text had exactly one production
caller, and phase_runner's three other paths (composed microVM, solo
microVM, direct session) never called it. CAPABILITY-REVIEW.md said the
broad thing too; both are corrected.
Those three tiers share one task string and have no per-turn alias, so
their skills resolve per PHASE from the mission's crew and are appended
there. The container tier deliberately still injects per turn, with the
running node's own role — appending in both places would put every crew
member's skills in every turn twice.
The behavioural tests prove phase_skills_text and compose_turn_prompt work.
They cannot prove the three launch_* calls pass the composed string, and
that substitution is a one-word edit that would silently return all three
tiers to delivering nothing with every test still green. So there is also a
source-level assertion on the call sites, following the precedent in
mission_events::the_cap_is_enforced_in_one_statement. Its negative control
names the exact tier.
PROVENANCE: what an agent received, and what it said it did
Both were unanswerable. The prompt was never stored anywhere on any tier —
re-deriving it later re-runs the skill lookup against a catalogue that has
since changed, and once agents author their own skills it certainly will
have. The reasoning rows were durably write-only: pushed live once, then
never read from the database again by anything except the GC that deletes
them.
- prompt.composed records the exact bytes, on all four tiers
- the session tier writes its checkpoint record and a reasoning row,
instead of eprintln! and nothing — the same defect the solo microVM
path was fixed for, in the last tier that still had it
- narrative_for_mission reads both back
Found while doing it: the 400-event per-phase cap counted EVERY kind, so a
busy phase could push out its own phase.completed and its own provenance.
The cap now counts only the two unbounded kinds it was written for.
Negative control confirms the old behaviour dropped the prompt.
Retention is now a per-mission hold (0080) rather than a raised global —
with a test asserting unheld missions are still reaped, because an
exemption that applies to everything is not an exemption.
SELF-AUTHORING: agents apply their own skill drafts, no human click
By operator decision. level_up has generated complete drafts from a model
since it shipped; only a checkbox stood between propose and apply.
What replaces the gate is not another gate but four properties, each held
by a test:
- workspace-scoped, so a hand-authored skill can never be modified
- a draft cannot take a hand-authored skill's name. Ids are scoped and
bindings resolve by skill_id, so it could not overwrite or shadow one
anyway — but two procedures under one name means nobody reading a
transcript can tell which the agent followed, and that ambiguity is
fatal in a system where the skill is the standard being graded against
- every revision appends a skill_versions row, so it can be reverted and
a past run can be read against the text it was actually judged under
- approved_by = NULL. An agent's decision is never attributed to a person
who did not make it
Only skill_candidate applies autonomously. identity_refinement and
brain_consolidation still wait for a human: they change what an agent IS
rather than adding a procedure it can consult. State is announced at boot,
because a safety gate that changes silently is one nobody notices changed.
CLAWMATES_SKILL_SELF_AUTHORING=0 restores it.
Also: the test Postgres ran out of /dev/shm mid-suite (Docker's 64MB
default) and surfaced it during MIGRATIONS, which reads like a schema fault
and is not one. --shm-size=1g, and a pointer to the `clean` subcommand that
already existed for the 779 leaked test databases.
Full workspace suite green: 106 binaries, no failures.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
e4942ce985 |
fix(missions): skills can now reach a mission agent at all
Repairing the 55 broken skill bindings made the catalogue correct. This
makes it reachable, which it was not — for any skill, on any mission, since
the catalogue was built.
The skills had exactly ONE delivery channel: the `clawmates_skills` MCP
server. A mission claw could not reach it for three independent reasons:
1. `provision_claw` wrote the constant `["clawmates_door"]` and ignored
the template's mcp_bundles — which mission_orchestrator had already
resolved and stored on the team row.
2. The runtime config defines no `clawmates_skills` bundle. The live
local config defines no bundles at all, not even the door.
3. Mission claws run on `claude_cli`, which the runtime's own config
comments document as text-only: it cannot surface a tool call, so no
MCP server is reachable from a mission turn regardless of bundles.
And a mission turn's whole system context is two sentences synthesised from
the role slot in topology_exec::build_prompt. The template's role prose is
not used either — mission_orchestrator documents this, and it means the
role prompts describing which procedures to follow were never read.
Two doc comments in cm-runtime describe the mission path as already having
the summary-and-fetch contract. It never did. The belief was written down
twice and checked zero times, which is why nobody looked — and it is why
the Skill-Use measurement this review planned could only ever have returned
a trigger rate of zero. That would have read as a finding about the agents.
- provision_claw takes the bundles, with clawmates_door always added: a
template that forgets to list it must not get an ungated agent
- all 11 templates now request clawmates_skills; web_fetch removed, since
a list that is honoured must not name a bundle that does not exist
- the re-provision sweep re-asserts the team's own stored bundles rather
than a constant, which would have silently stripped a capability
mid-mission
- pinned skill BODIES are injected into the mission prompt, bounded and
with truncation stated. Bodies, not an index: there is no `skills.read`
tool on this path, so an index would advertise a capability that does
not exist — the exact failure this whole change is about
Three tests: the body reaches the prompt, an agent with no skills adds no
heading (an empty "Your skills" section announces skills the agent does not
have), and the composition is exercised separately from the lookup, because
`pinned_skills_text` working and `run_turn` calling it are different claims
and the second is the one that was false.
Also adds the three review documents: CAPABILITY-REVIEW (inventory, what
was repaired, what is deferred and why), PROVENANCE-ASSESSMENT (assess
only, per decision — what each store answers and the two candidate paths),
and RESEARCH-SWEEP (the fortnight's papers and what we did about each,
including the ones we deliberately did nothing about).
Full workspace suite green.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
6e8785f159 |
fix(missions): a server restart no longer kills a running mission
Mission 01a00538 ("ClawHDF5 REsearch and Refactor") failed 19 minutes and 93,762
tokens into its research phase with `pair failed: 403 Forbidden`, and its coding
phase was then correctly skipped as unreachable. The cause was not the coding
phase and not the model — it was pairing.
A per-mission runtime is authenticated with a SINGLE-USE pairing code, and the
bearer token it returns was cached in memory only. Any restart of the server
discarded that token; the next turn re-paired with a code the gateway had
already spent and got 403 — permanently, for that mission. A deploy, a crash or
an OOM would each do it. The durable-run machinery exists precisely so work
survives a restart; pairing was the one thread that did not, and it failed
closed.
`missions.runtime_token` persists the token at the moment pairing succeeds, and
the worker seeds the executor's cache from it, so a new process reuses the
credential instead of re-pairing. Persisting is best-effort: failing to save
must not fail a turn that just paired successfully.
Verified by reproducing the original failure: launched a mission, confirmed the
token was written, restarted the server MID-PHASE, and watched the mission run
to completion with no pairing failure.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
43436d7181 |
feat(telemetry): push bus for live agent frames
/api/world/live is a 2s database poll, which is right for queryable state and wrong for a token stream: reasoning only became visible after a step finished and its row was written. This adds a process-wide broadcast bus that topology_exec publishes to as the runtime's WebSocket delivers frames, and the SSE handler forwards without waiting for the next tick. Measured: the pushed frame arrived ~2.2s before the polled copy of the same text. Design notes worth keeping: - A global (OnceLock), not an AppState field. The publisher is reached through phase_runner -> topology_worker -> MissionTap, none of which hold AppState; threading a handle through all of them would put a UI concern into four layers that have no other reason to know about one. - Lossy by design. A slow subscriber lags and skips rather than applying backpressure to the agent producing. mission_events remains the durable record; this bus is the fast path, never the source of truth. - Only `claw_<uuid>` aliases are attributed. The governor, door and evaluator drive real turns under other names, and attributing their output to an agent would put words in someone's mouth. Asserted in a test. - The poll no longer emits `reasoning`: with both paths live, every turn arrived TWICE — once pushed, once polled ~2s later. The row is still written; this feed just is not its second mouth. CEILING, measured rather than assumed: turns are not token-level because the runtime is not streaming. zeroclaw's claude_cli provider runs `claude -p --output-format json`, which returns ONE result object when the turn completes — there are no incremental tokens to forward. Making this genuinely token-by-token needs `--output-format stream-json` and incremental parsing in the zeroclaw fork, not here. The bus is in place and will carry them the day it does. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
d810fc0a86 |
fix(viz): read the gateway's real tool_call keys, and correct the record
The frame is `{"type":"tool_call","id","name","args"}` — zeroclaw-gateway
/src/ws.rs. The tap read `tool` then `name`, and `arguments` then `input`.
`name` happened to be in the fallback chain; `args` was not in it at all,
so a container-tier tool call would have been recorded with its name and
NO path — a tool that reads as having touched nothing. `tool` and
`arguments` belong to `approval_request`, which is where they came from.
Also corrects what the histogram was read as saying. A mission turn on
gw-04 carried only chunk/done/session_start, and the first reading was
"there is no tool_call frame". Wrong: `grep -c tool_call` on the deployed
0.8.3 binary returns 46. Container-tier agents are provisioned tool-free
behind the MCP door (§15), so they call nothing — there is nothing to
observe on that tier, and nothing is broken.
That distinction is exactly what the histogram was shipped to make
possible: a tap matching no frame is otherwise indistinguishable from a
mission that used no tools.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
9e61e3ba35 |
feat(viz): what the agents actually did, as structured events
The World could draw a mission's shape but nothing about the work. The
detail existed only as prose in checkpoint.log and model output, where a
tool name is indistinguishable from an agent *talking about* a tool — so
it was never parsed, deliberately. `mission_events` is the structured
channel that replaces it.
Three taps, one table:
- Container tier: the `_ => {}` at the end of topology_exec's typed frame
stream now matches `tool_call` and reads the tool's JSON ARGUMENTS for a
path. Never the prose summary — a path scraped from a sentence would put
files on the map that no agent opened, and the test proves a Grep whose
summary says "src/main.rs" produces no file touch. The frame name itself
is unverified, so the same commit ships an unmatched-frame-type
histogram: a tap that matches nothing looks exactly like a mission that
used no tools, and this is how one gw-04 run names the real frame.
- microVM tier: a `PostToolUse` hook, the seam vm_stop_gate already proved
fires under `claude -p`. It copies stdin to /root/tap and exits 0
unconditionally — a non-zero PostToolUse hook talks back to the model,
which would turn the observer into a participant. Drained before collect,
since the VM is destroyed moments later.
- Phase transitions: five identical copies of the pending→running UPDATE
became one `mark_phase_running`, and `close_finished_phases` grew
RETURNING. Its CASE decides each phase's status inside SQL from rows the
statement does not change, so it cannot be re-derived afterwards without
writing that CASE twice — without RETURNING it emits zero phase.completed
and reports success.
The settings.json hazard the plan called out: the stop gate wrote the
WHOLE document, so a second hook writer would have silently erased it and
a coding phase would then complete having written nothing — the exact
failure the gate exists to catch. There is now one composer,
`vm_tool_tap::guest_settings`, one writer, and a source-walk test that
fails if anything else writes a settings document.
`mission_events.run_id` carries no FK on purpose: phase_runner DELETEs
topology_runs on retry, and a cascade would erase a phase's whole history
the moment it retried — silently, since a cascade is not an error.
world.rs streams it with a cursor that separates backfill from motion.
Everything already in the table when a subscriber arrives is drawn as
settled history; only what lands afterwards animates. Otherwise opening a
finished mission replays an hour of tool calls as a burst storm.
Bounded twice: 400 events per phase (enforced inside the INSERT, since
two concurrent taps would each read a count below the cap) and a 7-day
retention sweep in mission_gc.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
4c418f7d9b |
fix(runtime): a turn gets the phase's budget, not 100s more than one call
A turn is an agent LOOP, not one model call. Each call inside it is bounded separately by the daemon — `claude_cli`'s `timeout_secs`, 600s on gw-04, verified in the live config — so TURN_TIMEOUT has to cover however many calls the loop makes, not one of them. It was 700s. MEASURED: a healthy research turn is ~157s. A throttled one blew the budget with one slow call plus a second, and the executor killed it at 11m43s with no error from the daemon, because nothing had failed yet. The operator got "turn executor failed: turn timed out" and the container holding the reason was torn down minutes later. An hour matches the phase's own budget. A genuinely stuck CALL is still caught at 600s by the daemon and surfaces as a real error; this only stops us killing turns that are working, slowly. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
b2e2735583 |
fix(diag): "turn timed out" now says what the agent was doing
A research_and_code mission failed with: turn executor failed: turn timed out turn executor failed: turn timed out and that is the entire record. Investigating it found: the run produced zero steps and zero output, it died at exactly 700s (TURN_TIMEOUT), the node→claw aliases were bound correctly, and the same zeroclaw team path passes in the `multirole` scenario. So the platform path is fine and the agent simply never finished a turn — but the one place the reason lived, the per-mission runtime container, is torn down after the phase and takes its log with it. By the time anyone looks, all that survives is the string. The timeout now reads the last 40 lines out of that container while it still exists, and reports which agent alias and which gateway it was driving. Best-effort by construction: it runs on a path that is ALREADY failing, so a docker error there degrades to a note rather than replacing the real failure with a second one. `container_name` derives the container from the gateway URL and returns None rather than guessing, because this feeds a diagnostic — a wrong name would put a different container's log under a failure and send the reader somewhere else entirely. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
f848248fac |
feat(missions): goal conditions and phase iteration, judged on the subscription model
A phase used to complete when its topology_runs reached a terminal state --
purely structural. It marked itself done whether the agents produced the
artifact or wrote nothing at all, and it ran exactly once: execute_resumable's
skip(start) is resume, not repeat, and the only re-run path was a human
hitting the retry endpoint.
A phase can now carry `done_when`, a completion condition judged after each
pass against the evidence the agents actually surfaced. Not met and passes
remain -> the phase goes back to pending with iteration bumped, and the
verdict's reason is appended to the next pass's task text. That feedback is
what makes iteration converge rather than repeat -- the same mechanism /goal
uses, and that swarm.rs already uses for rejected work.
The evaluator runs on the SUBSCRIPTION model. CLAWMATES_EVALUATOR_MODEL
defaults to judge_model(), and a `runtime:<alias>` spec routes through
ZeroClawDriveExecutor -- a container agent on claude_cli, i.e. Claude Code on
the OAuth subscription, needing no platform API key. Same routing the door
governor uses.
Two deliberate departures from the governor's contract, both required:
- FAIL-CLOSED. Runtime::judge is fail-open and reads a verdict by
!contains("DENY"), so a model explaining why it *would* deny reads as
approval and an empty reply reads as approval. For completion that is
backwards: unsure must mean not done. The contract is swarm.rs's strict
JSON {"met","reason"} with .unwrap_or(false). Six tests cover the closed
paths -- prose, empty, missing field, non-boolean, transport error.
- judge_raw returns the raw reply; judge collapses to a bool too early to
carry a structured verdict.
Iteration scoping is the subtle part and has its own test: on pass 2 the
phase's own iteration is 1 but pass 1's completed run is still in the table,
so "are this phase's runs all finished?" must ask about the CURRENT pass or
that stale row closes out pass 2 the instant it is enqueued.
Evidence comes from phase_summarizer::collect_evidence, extracted from the
existing collect_material so the evaluator and the summary card cannot
disagree about what a phase produced.
done_when/max_iterations are promoted from phase config into columns (the
sweep filters on them every tick) and max_iterations is clamped to 20 at
insert -- the UI limits it too, but a runaway loop must not be one crafted
request away.
A phase with no condition completes exactly as before; that regression guard
is the first test in the file.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
9bc5f6a142 |
fix(missions): bind graph nodes to claws via attrs, not a dropped top-level key
`inject_node_agents` wrote the claw alias as a top-level `"agent"` key on
each graph node, but `cm_topology::Node` only deserializes `{id, role,
level, attrs}` — serde silently dropped it. `TurnRequest::agent` came back
`None` and every mission turn fell back to `ZEROCLAW_DEFAULT_AGENT`
(`scout`), running with scout's workspace and tools instead of the
mission's claws. The runtime trace confirms it: every turn logged
`"agent_alias":"scout"`.
That is why mission agents reported an "empty greenfield" workspace and
emitted artifacts inline instead of writing them: scout is jailed to
`/zeroclaw-data/.zeroclaw/agents/scout/workspace` and cannot see
`/mission/repo`. The per-mission provisioning and `workspace.path` pinning
shipped earlier were correct — they were just applied to agents that
nothing ever drove.
- bind into `node.attrs["agent"]` (top-level key kept for display/debug)
- extract the DB-free `apply_node_agents` and add a regression test that
round-trips through the real `TopologyGraph` deserializer, which is the
guard that was missing
- log loudly in `topology_exec::run_turn` when a node falls back to the
default agent, instead of silently swapping in a different agent
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar Sobh
|
b569688e04 |
fix(mission_runtime): per-mission auto-pair via container log scrape (C3 auth)
The seed-mount approach didnt work: even with the shared runtimes data dir bind-mounted, a fresh gateway instance mints a new pairing key and requires re-pairing. The topology_worker connect returned 401 forever. New approach — per-mission gateways self-pair: - Provisioner tails container logs after start, extracts the X-Pairing-Code from the boot banner - Persists it on missions.runtime_pairing_code (migration 0059) - topology_worker constructs ZeroClawDriveExecutor with THAT code via from_env_for_gateway_with_code, which triggers the lazy /pair handshake on first turn and caches the returned bearer Drops the shared-runtime data-dir mount — each per-mission gateway now owns its own state, restoring the C3 isolation guarantee. |
||
|
Omar Sobh
|
212e68f6b1 |
research/probe: end-to-end smoke test endpoint (skip topics/loops/spawn)
New: POST /api/research/probe — one-shot pipeline probe against the
workspace's shared ZeroClaw gateway. Drives a trivial 'reply OK'
turn via ZeroClawDriveExecutor::drive and reports per-step timings.
Purpose: stop guessing what's broken in the wizard flow by exercising
JUST the executor→daemon→claude→response path. If probe succeeds
within a few seconds, we know:
- ZEROCLAW_TOKEN + gateway URL config is correct
- Daemon can reach and authenticate against claude
- The full ws round-trip works
…and every other failure we've been chasing (turn timed out, LLM
request failed, ws connect DNS error, etc.) is spawn-config or
prompt-size specific.
Request body (both optional):
{ "prompt": "Reply OK", "agent": "coordinator" }
Response:
{
"verdict": "ok" | "fail",
"total_duration_ms": …,
"prompt_len": …,
"response_preview": "OK",
"steps": [
{ name: "build_executor", duration_ms: …, status: "ok" },
{ name: "drive_turn", duration_ms: …, status: "ok", detail: "tokens=N, output_len=M" }
]
}
Wire-up:
- new routes/probe.rs
- pub mod probe in routes/mod.rs
- POST /api/research/probe registered in lib.rs router
- ZeroClawDriveExecutor::drive promoted from private to pub so the
probe handler can call it (behavior unchanged, other callers were
all inside the same struct).
Usage from anywhere (curl, browser dev tools, etc.):
curl -X POST https://clawmates.work/api/research/probe \
-H "Content-Type: application/json" \
-H "Cookie: <session cookie>" \
-d '{}'
Follow-up: a small frontend button (e.g. bottom of ResearchList) that
POSTs this and renders the response inline, so users don't need to
curl. Skipping in this commit to ship the useful part first.
|
||
|
Omar Sobh
|
3373f57da0 |
topology_exec: bump TURN_TIMEOUT to 700s to outlast daemon claude_cli
Verbose daemon logs revealed the real turn failure: the daemon's own claude_cli provider was timing out at 180s while running a 10-15k token coordinator prompt. The daemon reads its per-provider timeout from ZEROCLAW_providers__models__claude_cli__default__timeout_ms — we set that to 600000 (10 min) on gw-04 via a compose patch. Now the executor's TURN_TIMEOUT (was 300s) must exceed the daemon's own limit, or we kill the ws before the daemon can reply. New sequence: daemon has up to 600s to invoke claude and return a response; executor waits up to 700s (100s headroom) for the ws event stream to drain. If claude takes 500s, both survive. If the daemon really does hang past 600s, its own timeout fires first and we get a proper "provider timed out" error instead of a phantom executor timeout. Compose env applied on gw-04 in the same session: - ZEROCLAW_providers__models__claude_cli__default__timeout_ms=600000 - ZEROCLAW_providers__models__claude_cli__door__timeout_ms=600000 (backup: /opt/clawmates/docker-compose.yml.bak-timeout) Server container recreated to pick them up; env verified. Follow-up: the coordinator prompt is legitimately huge (autonomy contract + roster + description + repo tree + integration-plan template + operator notes = 10-15k tokens). We should consider either shrinking it or breaking the work into multiple smaller turns so the daemon isn't gambling on a single call taking 3-8 minutes. |
||
|
Omar Sobh
|
63305689be |
topology_exec: bump TURN_TIMEOUT to 300s
Turn timeout was 90s. Research coordinator turns run 60-120s routinely (long prompt + cold tool selection) — 90s was tripping legitimate turns while the daemon was still working, throwing away completed inference. 300s gives real turns room while still bounding worst-case at a walk-away limit. Latest wizard-created topic hit this precisely: bridge attached, handshake worked, claude auth verified in the container, but the run's status went to failed with 'turn executor failed: turn timed out' after exactly ~90s. The topology worker's stale-run sweep at 180s covers the case where the worker itself dies mid-turn — a 300s turn still checkpoints every step so a stuck worker will get requeued. |
||
|
Omar Sobh
|
88c78bd16e |
research: topology_worker points executor at per-topic gateway (commit 2/3)
Commit 2 of the path-B plan. The container that commit 1 spawns
now actually receives the run's turns — up until now it was
started but unused. This is the payoff commit: research runs are
truly isolated per topic.
Backend
- topology_exec.rs: from_env() refactored to a thin wrapper over a
new from_env_for_gateway(url) helper. Same shape (env-derived
aliases + default + token) but the caller supplies the URL. The
auth token/pairing code still comes from ZEROCLAW_TOKEN /
ZEROCLAW_PAIRING_CODE on the server; research_container's
inherited_env propagates those into the team container so the
same credentials work at both endpoints.
- research_container.rs: new wait_ready(url, deadline) that polls
<url>/health with a 1.5s per-request timeout every 500ms until
it 200s or the deadline passes. reqwest-based so it doesn't need
bollard. Called by the worker after claim, before pair, to bridge
the "container is starting, gateway not yet listening" gap.
- topology_worker.rs run_job:
1. Look up research_topic_id for the claimed run.
2. If Some, load the topic and read zeroclaw_gateway_url.
3. If a URL is present:
- best-effort wait_ready(url, 30s); a timeout logs but
doesn't abort — the pair call below will just fail
faster than pinging forever
- build the leaf via from_env_for_gateway(url)
Else fall back to from_env() (workspace-wide gateway).
4. The rest of run_job is unchanged — the leaf drops into
either SubTopologyExecutor (org/company) or direct drive
(team tier) as before.
What now works end-to-end
Starting a research topic with a bound repo:
1. clone-shallow into per-topic workspace
2. docker create + start the clawmates-runtime container, name
= research-<topic>-team, joined to clawmates_core so the
server reaches it by name
3. persist container name + gateway URL on the topic row
4. enqueue the topology run tagged with research_topic_id
5. worker claims → looks up the topic → waits for the team
gateway's /health → constructs a from_env_for_gateway
executor pointed at http://research-<topic>-team:42617
6. every turn's `/ws/chat?agent=…` hits the isolated container;
agents inside see the repo at /workspace/repo (rw); each
topic's memory/state lives under its own /zeroclaw-data mount
Deploy prereqs (unchanged from commit 1)
- clawmates_server compose service needs a bind-mount of
CLAWMATES_RESEARCH_WORKSPACE_ROOT so the paths spawn() writes to
are visible on the host and the spawned team container mounts
the same underlying data.
- socket-proxy ACL needs POST + DELETE on /containers (prod ✓).
|
||
|
Omar SobhandClaude Opus 4.8
|
cbfa0ff24f |
feat: agent-to-agent platform on ZeroClaw 0.8.2 — rooms, delegation, A2A ingress
Builds on the v0.8.2 runtime. Four workstreams, all behind the §15 MCP door:
- Group rooms (Phase 1): migration 0026; N-way threads repo with a DM/room
count-guard; chat.send {room} + room.create/invite/leave tools; RoomMessage
-> room.message SSE; /api/claw-chat/rooms* APIs; Observer room badge.
- Per-claw door identity: door caller_agent resolves the X-ZeroClaw-Agent
header (set by the fork) to the specific claw, falling back to roster[0].
- Gated delegation bridge (Phase 3): clawmates__delegate door tool drives a
sibling via the existing /ws/chat ZeroClawDriveExecutor (not A2A); self-deny,
per-workspace hourly budget, audit trail, untrusted-banner result. Native
in-daemon delegation stays off (it would bypass the door).
- A2A tenant ingress (Phase 2): migration 0027 (workspace_a2a + a2a_tokens);
runtime_provision enable_a2a_server/publish_claw; routes/a2a.rs tenant-aware
proxy (per-workspace tokens, injected internal bearer, daemon stays internal,
cards URL-rewritten to the cm-api edge); a2a.invoked taxonomy.
Tests: cm-db room repos, cm-runtime chat tools, door units. sqlx cache updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
3eca4ed70c |
Recursive deploy ladder: Company + Org tiers, mesh mark, two-tier rail
Completes the scale ladder (single → team → company → org). Every tier is a
topology whose nodes are the tier below; running a parent recursively runs each
child's sub-topology down to the leaf claws.
Backend:
- migration 0011: companies/company_teams, orgs/org_companies, topology_runs.tier
- cm-db repos for companies + orgs (mirror teams)
- TurnRequest.attrs (forwarded from node.attrs) for child-id binding
- SubTopologyExecutor (recursive_exec.rs): a parent "turn" runs the child's
sub-topology; durability via parent updated_at keepalive + cancel propagation
+ depth cap; boxed future breaks the org→company recursion
- topology_worker selects executor by job.tier
- routes: /api/companies, /api/orgs (create/list/get/run) + unified
/api/structure/{level}/{id} for the zoom canvas
Frontend:
- MeshMark: node-mesh brand glyph (replaces the claw PNG), tier variants
- TopologyGraphView: optional onNodeClick/nodeMeta + dark-token theming
- StructureCanvas + Breadcrumb: one recursive zoom view for every tier
(drill down on node click, breadcrumb up); TeamRunPanel extracted + shared
- two-tier Discord-style rail: StructureRail (mesh mark + org/company/team
glyphs + tools popover + deploy + user) | RosterColumn (selected group's
children, or your claws); SecondaryNav for cross-cutting tools
- ComposeWizard (company/org) wired into DeployWizard; /companies + /orgs pages
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
3402a3b56d |
Door governor: runtime-agent judge (Kimi-as-judge on the subscription)
The §15 governor can now be judged by a ZeroClaw runtime agent instead of a server-side registry/API model. ZeroClawDriveExecutor::judge drives an agent with the governor prompt and parses ALLOW/DENY (fail-open). mcp_door routes to it when CLAWMATES_JUDGE_MODEL=runtime:<alias>. This unblocks Kimi-as-judge with NO Kimi Platform key: Kimi runs on the membership via kimi_cli, so CLAWMATES_JUDGE_MODEL=runtime:judge_kimi makes the coding agent the governor. (Same path works for any subscription-only model.) cm-runtime re-exports judge_model. clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
99e2fd3ec3 |
orchestrator: per-node model selection via node.attrs["agent"]
TurnRequest gains an optional `agent` sourced from the graph node's attrs["agent"]. The ZeroClaw executor binds a node to that alias directly when present, falling back to the role→alias map otherwise. This lets a single POST /api/topologies/run specify a different model per role (heterogeneous topologies) entirely in the graph JSON — no server ZEROCLAW_AGENT_MAP change or recreate per configuration, which makes quota-frugal model×role sweeps practical. cm-orchestrator 13 + cm-api topology_exec 4 tests pass, clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
d1409a0776 |
Topology executor: accept a durable ZEROCLAW_TOKEN (skip one-time pairing)
ZeroClaw pairing codes are single-use, so a static ZEROCLAW_PAIRING_CODE only works for the first run. Add ZEROCLAW_TOKEN: pair once out-of-band, set the durable bearer, and runs are repeatable. PAIRING_CODE stays as a fallback; one of the two is required. Validated live on gw-04: a single-node pipeline driven through the deployed authed POST /api/topologies/run drove a real ZeroClaw role-agent over /ws/chat and returned a RunRecord with real output + token metering, persisted to topology_runs. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
6d77a0acc1 |
Topology P-zc 1A: ZeroClawDriveExecutor — real role-agents over /ws/chat
cm-orchestrator owns the topology graph; each turn now drives a real ZeroClaw
role-agent in a container via the proven gateway drive recipe, instead of a
tool-free cm-llm call.
- crates/cm-api/src/topology_exec.rs: ZeroClawDriveExecutor impl TurnExecutor —
pair (POST /pair + X-Pairing-Code, token cached) -> ws /ws/chat?agent=<alias>
-> send {type:message,content} -> drain chunk/done/approval_request/error.
approval_request is recorded as a BLOCKED GatedAction, never auto-approved (§15).
Role->alias via ZEROCLAW_AGENT_MAP, fallback ZEROCLAW_DEFAULT_AGENT (scout).
- POST /api/topologies/run {task,graph} -> execute() -> RunRecord, persisted
best-effort to the existing topology_runs table (no migration). compare stays
tool-free. from_env() is read in-handler so cm-api still boots unset.
- deploy/clawmates-runtime: example config now declares a tool-free multi-agent
role-cast; README documents the ZEROCLAW_* knobs + run endpoint.
tokio-tungstenite 0.26 (already in lock) + dev axum `ws` for the hermetic test.
3 lib tests green, clippy clean, SQLX_OFFLINE build clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|