d23f30e92950c7b67fa24bad1e06882be61c044d
453
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
128b423205 |
fix(ci): the tool gate needs node, and an inert gate must say so
The first push of the PreToolUse gate failed CI, and the reason is a property of the gate worth fixing rather than a CI quirk. The hook parses its JSON payload with `node` — no jq in the runtime image, and node is guaranteed there because Claude Code is a node program. CI runs `cargo test --workspace` inside `rust:1.96-slim`, which has no node. The extraction returned nothing, the gate allowed everything, and the two "blocks" tests failed. That is correct behaviour with a dangerous appearance. A gate that cannot read its input must not block the phase — failing closed on a parse error denies every tool call, which is what an earlier `case`-syntax bug did. But allowing silently makes an INERT gate indistinguishable from one that simply matched nothing, which is this codebase's recurring defect exactly. So the gate now records `inert` when node is absent, still allowing, and a test pins both halves: exit 0, and the marker written. The host can check for that file rather than infer a working gate from an absence of denials. CI installs nodejs so the shell tests exercise the gate instead of its inert path. Verified in a rust:1.96-slim container: without node the force-push payload returns 0, with node it returns 2. Also confirmed the generated script behaves under dash — Linux /bin/sh — not only under macOS sh. An earlier apparent dash failure was invalid JSON in the probe command, not the gate. Full workspace suite green: 106 binaries. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
547b5d9987 |
feat(missions): a pre-execution gate on mission tool calls
The second half of the tool-call research. Until now a mission agent's
Bash call was gated by nothing, anywhere.
WHY THERE WAS NO GATE
vm_tool_tap is a PostToolUse hook: it fires after the tool has already run
and exit-0s unconditionally, because a non-zero PostToolUse talks back to
the model. It is telemetry and says so. GatePolicy — the §15 door — has one
enforcement site, the chat loop, and its approvals key on
(session_id, message_id), which no mission phase can produce. Meanwhile the
solo tiers run `claude -p --permission-mode acceptEdits` with Read, Edit,
Write and Bash pre-approved.
PreToolUse fires under `claude -p` in this image — measured by vm_stop_gate,
which also proved the exit-2-plus-stderr contract — and had zero callers.
This is that hook.
WHAT IT IS, AND IS NOT
A deterministic policy gate: a short deny list of actions with no legitimate
form inside a mission, blocked before they run, with the reason handed back
so the model can choose differently.
It is NOT the §15 human approval gate, and the module says so. A hook blocks
the agent's process while it runs and a human decision takes minutes to
hours; waiting inside the hook would wedge the turn. This closes the gap
between nothing and something.
The deny list is short on purpose. A gate that blocks legitimate work is
worse than none: the agent cannot ask a human, so it either works around the
block — doing something stranger than what was denied — or burns the turn.
FOUR BUGS THE TESTS FOUND, IN ORDER
1. Substring matching denied `grep -rn 'rm -rf /' docs/`. Searching for a
string is not running it. Now rules anchor to the start of a shell
segment, with a separate flag-style match that exempts text tools.
2. The `case` patterns were unquoted, so a needle containing a space made
the whole script a SYNTAX ERROR — which as a PreToolUse hook exits
non-zero and denies EVERY call. Every text assertion passed while the
script was in that state; only running it under a real `sh` found it.
3. The hook receives JSON, not a command, so "starts with" could never
match — `case` saw `{"tool_name":"bash",...` every time. Now extracts
tool_name and tool_input.command with `node` (no jq in the image; node is
guaranteed because Claude Code is a node program).
4. `IFS='\n'` in POSIX sh sets IFS to backslash and the letter n, not a
newline. Nothing split, so only commands with no separator were ever
tested and `cd /tmp && rm -rf /` sailed through. Now a literal newline.
Every failure path allows. A gate that fails closed on a parse error blocks
the whole phase, which is exactly what bug 2 did.
Wired through vm_tool_tap::guest_settings, still the single writer of the
guest settings document — a third hook makes the clobber it prevents more
likely, not less, and a test asserts all three survive one document and that
PreToolUse points at the gate's own script rather than the tap's.
Honest limit, stated in the module: a determined agent defeats any
string-matching gate. This is aimed at accidents and obvious cases; the real
isolation is the container and microVM boundary.
Full workspace suite green: 106 binaries, zero build errors.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
ea0b989b3f |
docs(research): missions DO call tools — the claim was wrong, and the truth is worse
Deep research into "missions can't call tools at all", which I wrote and which is false. docs/TOOL-CALL-ARCHITECTURE.md has the full findings. WHAT IS ACTUALLY TRUE Three of the four mission paths end in `claude -p` with Claude Code's own toolset and permissions PRE-ACCEPTED: solo microVM Read Edit Write Bash Agent --permission-mode acceptEdits composed microVM same, per node same direct session Read Edit Write Bash acceptEdits So the position is not "no tools". It is: mission agents run Bash and Write with permissions pre-accepted, and nothing in this platform can gate them. That is a stronger finding than the one it replaces — "can't call tools" sounds like a missing feature; "calls tools freely, ungated, and mostly unobserved" is a security posture, and it is ours. Observe and gate are different and both are partial. vm_tool_tap is a PostToolUse hook: it fires AFTER the tool ran and exit-0s unconditionally, so it is telemetry and structurally cannot gate. The direct-session tier has no tap at all. GatePolicy has exactly one enforcement site — the chat loop — and its approvals key on (session_id, message_id), which no mission phase can produce. WHY THE CONTAINER TIER LOOKED TOOL-FREE `claude_cli` runs `claude -p --output-format json`, which returns a single final result object, and the provider hardcodes `tool_calls: Vec::new()`. The calls happen; the transport discards them. The comment reading that emptiness as "§15 by construction: agents are provisioned tool-free" was inferring a design property from a serialization choice. Verified against the deployed Claude Code 2.1.228 rather than assumed: `--output-format stream-json --verbose` emits `tool_use` blocks with the tool name and `tool_result` blocks. The calls are fully observable; we ask for the wrong format. THE DOOR WE ALREADY BUILT AND NEVER PLUGGED IN claude_cli.rs is OURS — upstream zeroclaw-labs/zeroclaw has no such file — and so is 88eef99d4 "claude_cli --mcp-config + allow/disallow tools (act via door)". The provider already accepts mcp_config (claude's own MCP client reaches our door), tools, and disallowed_tools (lock out the natives so the gated door is the ONLY actuator). agent.config.example.toml documents the whole shape. In the live runtime: clawmates-mcp.json does not exist, there is no [providers.*] block, and every mission claw binds to claude_cli.default which sets none of it. My earlier "claude_cli cannot reach MCP, therefore the skills server is unreachable" was wrong in its reasoning — the capability is built, documented by us, and never deployed. Related: we set `agents.<alias>.mcp_bundles`, which configures ZeroClaw's OWN MCP client for its native loop. A claude_cli agent's actuator is the claude subprocess, which reads `mcp_config` on the PROVIDER. We were turning a knob wired to a loop that does not run. UPSTREAM 218 commits behind. No upstream work on claude_cli (the file is ours). ACP already exists in the fork; the three new commits are workspace-default and localization fixes, not new capability. The one item worth pulling is "feat(plugins): add shared egress policy foundation (#9137)" — a network guard with DNS pinning and metadata-address blocking, defence for the egress problem we have not solved. Stale claims corrected in place, in topology_exec.rs and the runtime config, so the codebase stops asserting the thing that is false. Recommended order, cheapest first: stream-json for observability; the PreToolUse hook for a real gate (it FIRES under claude -p per vm_stop_gate, and has zero call sites); then deploy the door. The executor swap is NOT recommended — the blockers are structural, not wiring, and the cheap fixes deliver what it was wanted for. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
771092b165 |
fix(skills): a pinned skill contradicted the platform inside the same prompt
Extending the Skill-Use mechanical checks, per the baseline's own next step, found something bigger than a missing check. THE DEFECT `workspace-repo-commit-protocol` told agents that `/workspace/repo` was "the ONLY path where source-modifying edits belong". The platform mounts and advertises `/mission/repo` — 26 references in the code; `/workspace/repo` appears in none of them. The skill is bound on 29 role bindings and was delivered TWICE in the run already measured, so an agent received the real path in its tool preamble and a skill contradicting it a few hundred tokens later, in one prompt. An agent that obeyed the skill wrote source into a directory nothing collects — the phase then delivers nothing, and looks like an agent that did no work. The same skill instructed `file_read` / `file_write` / `shell`: ZeroClaw's names, the exact ones `phase_task_text` was fixed to stop advertising after five agents on a single mission spent 7.4k tokens describing the mismatch instead of working. The prompt was corrected and the skill kept saying it. Rewritten against what the code actually does, including the repo-less case (`/mission/repo` exists, is collected as artifacts, has nothing to push). THE CLASS, AND THE GUARD The skills were never checked against the platform they describe. Nothing compared them, so a skill could contradict the prompt it ships inside and stay that way indefinitely — the same shape as PLAN_COMPLETE being documented and never implemented. Two tests in `skills_loader::contradiction_tests` now hold it: no skill may name a repo path the platform does not mount, and none may instruct a tool the agent's subprocess does not expose. The second matches backticked instructions and skips corrective lines, so a skill may still WARN against the wrong names — as this one now does. Both negative-controlled by restoring the old wording. AND THE CHECK THAT STARTED IT `workspace-repo-commit-protocol` now has a Boundary check: writing outside `/mission/repo` fails, and the message names the consequence — a phase that delivers nothing — rather than just the wrong path. docs/SKILL-USE-BASELINE.md records this as the fourth defect the measurement found, and corrects the "next unit of work" note now that this one is done. Full workspace suite green: 106 binaries, zero build errors. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
113de610ec |
fix(security): a signed Slack request could be replayed forever
Phase 5. The headline is not the coverage work — it is what looking for coverage found. A CAPTURED SLACK REQUEST AUTHENTICATED INDEFINITELY `slack_signature_valid` verified the HMAC correctly, and nothing anywhere checked how old the timestamp was. The timestamp is an input to the basestring, so an old request's signature verifies exactly as well as a fresh one — meaning anyone holding a single captured signed request (a proxy log, a mirrored packet, a leaked webhook body) could replay it forever, and every replay would authenticate. Slack's documented 5-minute window is now enforced IN THE BROKER, not the caller: the broker does not trust its caller (§15), and a check the caller can forget to make is one that will eventually be forgotten. Symmetric, so a far-future timestamp cannot mint a request valid for as long as the attacker chooses. Seven unit tests over the pure function with the clock injected, and the HTTP-level test now asserts an hour-old but validly signed request is refused. Negative control: removing the window fails the stale and future cases specifically. The existing slack_inbound test used the literal timestamp "12345" — a 1970 date — which passed only because nothing checked freshness. That is the shape of the whole finding: the fixture could not have failed, so it never told us anything. COVERAGE, RE-EXAMINED The review ranked crates by raw test count. That metric was misleading and found the wrong crates: cm-safety's seven tests already cover the decide CAS, grant double-consume, expiry and the approved/rejected split, and the audit_log immutability trigger is tested over in cm-db. Reading the API surface against the tests found the real gaps — verify_slack_signature above, and `credits_for_tokens`, pure pricing arithmetic that every existing billing test went through the database to reach without ever checking directly. Now pinned: the round-up contract, the deliberate one-credit floor, and that an absurd token count cannot wrap into a negative charge (a refund granted by an overflow). Still genuinely thin: cm-brain, where 6 of 9 tests need live clawbrainhub.com. Stubbing it means reproducing an external registry protocol we have no spec for — its own piece of work, not a coverage chore. Recorded rather than faked. GATEWAY PREFLIGHT ZEROCLAW_GATEWAY_URL and ZEROCLAW_TOKEN have no defaults and are read at FIRST USE, so a deployment missing them boots clean, serves every page, and fails the first time someone presses run. Third sibling of runtime_preflight and validator_preflight, same stance: a report, not a gate. The message names the consequence — "container-tier missions cannot run" — rather than only the unset variable. One process note: `cargo test -p cm-secrets` passed while the LIBRARY build was broken, because `time` is a dev-dependency there and my reference to it only resolved under cfg(test). Switched to std. Checking `cargo build --workspace` as well as the test profile is the guard. Full workspace suite green: 106 binaries, zero build errors. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
5c2c63f8e8 |
feat(missions): a human can finally reach the plan/roster review gate
Phase 4 of the plan, plus the PLAN_COMPLETE decision and the gitea_forge
cleanup from Phase 5.
THE REVIEW UI
mission_plan and mission_roster have been complete and reachable by curl
since they shipped, with zero frontend. That matters more than a missing
screen usually would: the decide step is not a convenience, it IS the
safety mechanism. Approving a plan replaces the mission's phases; approving
a roster flips it to the composed engine. A gate nobody can reach is a gate
that is always open or always shut.
MissionProposalDrawer, modelled on LevelUpDrawer which already does
load → review → decide. Reached from a mission's SETUP tab. Verified end to
end against the live backend, not just compiled: a model proposed a roster,
approval flipped the mission to `composed`, and approval on a non-draft
mission was refused.
The plan view shows each phase's done_when, and says plainly when one is
absent — a phase without a completion condition is never judged and reports
completed whatever it did, so its absence is the thing worth seeing.
AND THE DEFECT BUILDING IT FOUND
Every refusal path computed a precise reason — "the mission is running, not
a draft", "no node can boot that backend any more" — logged it to stderr,
and returned a bare {"error":"bad request"}. The person who needed the
sentence was the one clicking Approve; they got two words, and the reason
went to a server log they cannot read.
ApiError::Refused(String) carries it now. Same argument ApiError::Unavailable
was added for ("a 500 with 'internal error' sent them looking for a bug that
was not there"), one status code down. Live: the 400 now reads "this mission
is completed — a roster can only be approved while it is a draft, because
approving one rewrites how the mission will run".
PLAN_COMPLETE, decided
The Skill-Use measurement found that int-xx-marker-protocol documents
PLAN_COMPLETE and task_card_parser never implemented it, so an agent
following the skill exactly was silently ignored. Implemented rather than
removed from the skill: the planner needs a way to say it is done
specifying, and agents already emit it.
Marker ids are now strictly INT-<digits>. `starts_with("INT-")` accepted the
range form `INT-01..02` — observed live — which parsed into an id matching
no real item, so a task card appeared for something that did not exist while
the two items it covered stayed open. Rejecting is right: an ignored marker
is visible, a plausible row is not.
GITEA_FORGE, REMOVED
Named in nine places, defined in none. Harmless while provision_claw ignored
the bundle list; once the list was honoured, an undefined name became a
capability an agent is told it has and does not. Removed from seven team
templates, a workflow recipe, the auto-provision path, and a dropdown a user
could pick it from.
A new test asserts every bundle a template names is defined in the runtime
config — and it immediately found `web_fetch` in two templates I had missed
removing by hand. Same shape as the skill-binding test, one layer up.
Agents reach the forge through git over HTTPS with the ambient GITEA_TOKEN,
which is why nothing ever broke.
Full workspace suite green (106 binaries); frontend builds clean.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
91a6b4e304 |
feat(skills): the first Skill-Use measurement, and the three defects it found
Scored on the paper's three axes against two real missions on the local
stack. docs/SKILL-USE-BASELINE.md has the numbers, the method, and the
limits.
Trigger is reported as NOT OBSERVABLE, never zero
The paper measures progressive disclosure: the agent sees a name and
description and must retrieve the body, and that retrieval is the Trigger
event. We inline full bodies, because mission claws run on claude_cli which
cannot surface a tool call — there is nothing to retrieve with. So the
agent never reaches for a skill, it simply holds one.
Scoring that zero would report a delivery-model property as an agent
failure, which is the same confusion that kept 55 empty bindings invisible
for months. The verdict type carries NotObservable(reason) as a distinct
case from Fail for exactly this.
Compliance is checked by running the REAL task_card_parser rather than a
copy of its rules — a second implementation would drift, and then the score
would pass while the mission loop still stalled. Skills without a
machine-checkable consequence score not_applicable rather than a guess.
WHAT THE MEASUREMENT FOUND
1. The prompt format made its own record unparseable. Skills were
introduced with `## <name>` and skill bodies are markdown full of `##`
headings, so run 1 scored "Sizing heuristic" and "The output shape" —
subheadings inside decompose-int-items — as skills with no catalogue
row. Now an unambiguous `--- SKILL: <name> ---` marker, with both
writers sharing one renderer so the reader cannot drift from the writer.
2. A prompt was recorded that was never sent. My own Phase 1 work recorded
the phase prompt at the dispatch fork, before the tier was chosen — and
the container tier does not send that text, it sends the bare task and
appends skills per turn. Every container mission logged a `solo` prompt
that reached no agent. A provenance record of something that did not
happen is worse than no record: it is the wrong answer, delivered
confidently. Recording now happens inside each tier, with a test that
every launcher records the prompt it actually sends.
3. int-xx-marker-protocol documents a marker the platform never
implemented. PLAN_COMPLETE is in the skill's ladder and task_card_parser
has no such kind and never has, so an agent following the skill exactly
emits a marker that is silently ignored. Observed live: run 2's planner
emitted `PLAN_COMPLETE: INT-01..02`, which is also the range form — on
the kinds that ARE parsed that yields the id `INT-01..02`, a task card
for an item that does not exist while the two real items stay open.
This is a skill/implementation mismatch, not an agent failure, and it is
exactly what the measurement exists to find: the agent did what it was
told and what it was told was wrong. Both shapes now score as failures.
The reconciliation — implement PLAN_COMPLETE or drop it from the skill —
is left as a decision rather than guessed at.
The boot log now shows what the plan asked for: 53 skills, 11 templates,
every one `N role skills bound` with NO unresolved clause. Live missions
confirm per-role delivery — the planner receives decompose-int-items, the
coder receives write-rust-current-edition.
GET /api/missions/{id}/skill-use exposes the scores, and says in its
payload whether an empty result means "nothing delivered" or "the evidence
was reaped" — those have very different causes and must not look the same.
n = 2. No spread is reported because two runs cannot establish one, and the
document says so rather than letting the number be quoted as a baseline it
is not.
Full workspace suite green: 106 binaries.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
769e002bb3 |
feat(skills): deliver on every tier, record what agents receive, let them self-author
Three phases of the approved plan, plus a correction to what the last one
claimed.
CORRECTION: skills reached ONE tier, not all of them
The previous commit said "skills can now reach a mission agent". That was
true only for the container/ZeroClaw tier — the fall-through that queues a
topology_runs row for topology_worker, which drives the executor that was
patched. compose_turn_prompt/pinned_skills_text had exactly one production
caller, and phase_runner's three other paths (composed microVM, solo
microVM, direct session) never called it. CAPABILITY-REVIEW.md said the
broad thing too; both are corrected.
Those three tiers share one task string and have no per-turn alias, so
their skills resolve per PHASE from the mission's crew and are appended
there. The container tier deliberately still injects per turn, with the
running node's own role — appending in both places would put every crew
member's skills in every turn twice.
The behavioural tests prove phase_skills_text and compose_turn_prompt work.
They cannot prove the three launch_* calls pass the composed string, and
that substitution is a one-word edit that would silently return all three
tiers to delivering nothing with every test still green. So there is also a
source-level assertion on the call sites, following the precedent in
mission_events::the_cap_is_enforced_in_one_statement. Its negative control
names the exact tier.
PROVENANCE: what an agent received, and what it said it did
Both were unanswerable. The prompt was never stored anywhere on any tier —
re-deriving it later re-runs the skill lookup against a catalogue that has
since changed, and once agents author their own skills it certainly will
have. The reasoning rows were durably write-only: pushed live once, then
never read from the database again by anything except the GC that deletes
them.
- prompt.composed records the exact bytes, on all four tiers
- the session tier writes its checkpoint record and a reasoning row,
instead of eprintln! and nothing — the same defect the solo microVM
path was fixed for, in the last tier that still had it
- narrative_for_mission reads both back
Found while doing it: the 400-event per-phase cap counted EVERY kind, so a
busy phase could push out its own phase.completed and its own provenance.
The cap now counts only the two unbounded kinds it was written for.
Negative control confirms the old behaviour dropped the prompt.
Retention is now a per-mission hold (0080) rather than a raised global —
with a test asserting unheld missions are still reaped, because an
exemption that applies to everything is not an exemption.
SELF-AUTHORING: agents apply their own skill drafts, no human click
By operator decision. level_up has generated complete drafts from a model
since it shipped; only a checkbox stood between propose and apply.
What replaces the gate is not another gate but four properties, each held
by a test:
- workspace-scoped, so a hand-authored skill can never be modified
- a draft cannot take a hand-authored skill's name. Ids are scoped and
bindings resolve by skill_id, so it could not overwrite or shadow one
anyway — but two procedures under one name means nobody reading a
transcript can tell which the agent followed, and that ambiguity is
fatal in a system where the skill is the standard being graded against
- every revision appends a skill_versions row, so it can be reverted and
a past run can be read against the text it was actually judged under
- approved_by = NULL. An agent's decision is never attributed to a person
who did not make it
Only skill_candidate applies autonomously. identity_refinement and
brain_consolidation still wait for a human: they change what an agent IS
rather than adding a procedure it can consult. State is announced at boot,
because a safety gate that changes silently is one nobody notices changed.
CLAWMATES_SKILL_SELF_AUTHORING=0 restores it.
Also: the test Postgres ran out of /dev/shm mid-suite (Docker's 64MB
default) and surfaced it during MIGRATIONS, which reads like a schema fault
and is not one. --shm-size=1g, and a pointer to the `clean` subcommand that
already existed for the 779 leaked test databases.
Full workspace suite green: 106 binaries, no failures.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
e4942ce985 |
fix(missions): skills can now reach a mission agent at all
Repairing the 55 broken skill bindings made the catalogue correct. This
makes it reachable, which it was not — for any skill, on any mission, since
the catalogue was built.
The skills had exactly ONE delivery channel: the `clawmates_skills` MCP
server. A mission claw could not reach it for three independent reasons:
1. `provision_claw` wrote the constant `["clawmates_door"]` and ignored
the template's mcp_bundles — which mission_orchestrator had already
resolved and stored on the team row.
2. The runtime config defines no `clawmates_skills` bundle. The live
local config defines no bundles at all, not even the door.
3. Mission claws run on `claude_cli`, which the runtime's own config
comments document as text-only: it cannot surface a tool call, so no
MCP server is reachable from a mission turn regardless of bundles.
And a mission turn's whole system context is two sentences synthesised from
the role slot in topology_exec::build_prompt. The template's role prose is
not used either — mission_orchestrator documents this, and it means the
role prompts describing which procedures to follow were never read.
Two doc comments in cm-runtime describe the mission path as already having
the summary-and-fetch contract. It never did. The belief was written down
twice and checked zero times, which is why nobody looked — and it is why
the Skill-Use measurement this review planned could only ever have returned
a trigger rate of zero. That would have read as a finding about the agents.
- provision_claw takes the bundles, with clawmates_door always added: a
template that forgets to list it must not get an ungated agent
- all 11 templates now request clawmates_skills; web_fetch removed, since
a list that is honoured must not name a bundle that does not exist
- the re-provision sweep re-asserts the team's own stored bundles rather
than a constant, which would have silently stripped a capability
mid-mission
- pinned skill BODIES are injected into the mission prompt, bounded and
with truncation stated. Bodies, not an index: there is no `skills.read`
tool on this path, so an index would advertise a capability that does
not exist — the exact failure this whole change is about
Three tests: the body reaches the prompt, an agent with no skills adds no
heading (an empty "Your skills" section announces skills the agent does not
have), and the composition is exercised separately from the lookup, because
`pinned_skills_text` working and `run_turn` calling it are different claims
and the second is the one that was false.
Also adds the three review documents: CAPABILITY-REVIEW (inventory, what
was repaired, what is deferred and why), PROVENANCE-ASSESSMENT (assess
only, per decision — what each store answers and the two candidate paths),
and RESEARCH-SWEEP (the fortnight's papers and what we did about each,
including the ones we deliberately did nothing about).
Full workspace suite green.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
18dc0b964b |
fix(missions): the security scan phase now scans, and task upserts work
Four defects, found by checking the audit's claims instead of trusting them. Two of the audit's own findings turned out to be wrong, and the registry that exists to record which config keys are read was itself inaccurate — so the corrections are part of the change. upsert_task raised 42P10 on every call, for every caller `mission_tasks_external_uniq` is a PARTIAL unique index (WHERE external_id IS NOT NULL). Postgres will not match a partial index to an ON CONFLICT target unless the statement repeats the predicate, so the upsert failed on its first row. Both callers — the task-card parser that turns INT markers into tasks, and the security scanner — map the error to a string their caller logs. Two features were broken and nothing was red. Regression test in cm-db with a negative control: reverting the WHERE reproduces 42P10 exactly. the security scan never ran `security_scan::run` was reachable only from an operator button, so security_hardening.toml — a workflow whose entire first phase is a scan — ran an agent that was never told to scan and never fired the scanner either. phase_runner now sweeps finished security_scan phases, mirroring the benchmark baseline sweep that was added for the identical defect. Guarded on a new completion marker rather than on findings: a clean scan writes no findings, so a findings-guard would rescan forever. The marker also answers the question an operator actually asks, which is not "how many findings" but "was this looked at, by what, and when". two recipes could not fail security_hardening.toml and benchmark.toml carried no `task` and no `done_when` on any phase. A phase without done_when never enters evaluating, is never judged, and reports completed whatever it did — so a security mission could scan nothing and go green, and a benchmark mission could record no baseline that the next refactor would then compare against. Both now state the work and the condition, with inert keys annotated inline rather than deleted, so the gap between what a recipe asks for and what a phase receives stays visible. the config registry was wrong in both directions `harness` was listed NOT IMPLEMENTED while benchmark_runner reads it and phase_runner runs a baseline through it. `tools` was listed NOT IMPLEMENTED while security_scan::run reads it. A registry that exists so an operator can trust what a recipe does is worse than useless when it is inaccurate. Both corrected, `bench_name` and `cmd` added, and `test_command` deleted — it had neither a reader nor a writer, so it described a situation that could not arise. Also: CLAWMATES_JUDGE_MODEL had two different defaults (opus-4-8 in routes/topology.rs vs opus-5 in cm_runtime::judge_model) and a doc comment naming a third; topology now calls the one function. GITEA_TOKEN's absence in mission_plan is stated rather than degrading to the same "could not be read" string a private repo produces. BRAINHUB_API_KEY needed no change — hub::push already rejects an unset key with a named error. That half of the finding was overstated. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
4358964c05 |
fix(skills): every team-template skill binding now resolves
55 of 85 role skill bindings pointed at skills that were never authored,
so 10 of 11 team templates bound a smaller context bundle than their role
prompts assumed. Three roles bound nothing at all (gpu.bench_engineer,
threejs.shader_author, threejs.perf_engineer) while their prompts described
procedures they had no way to read.
The loader comment at team_template_loader.rs:167 already diagnosed this —
snake_case slugs in TOML against kebab-case skill files — and it was
half-fixed: the kebab names were corrected, the snake_case ones left.
It was invisible because both existing tests assert authored ⊆ referenced
(30/30, green) and the second explicitly declines to check the other
direction. So the failing half was the half nobody asserted.
Resolved every name by one of three explicit choices:
- 23 skills authored where the role genuinely needed the procedure
(gpu, threejs, research, analysis, frontend, mobile, backend, platform)
- renames onto authored skills where one existed in substance, including
the four-near-duplicate cases that collapse onto one real skill
- 22 aspirational references deleted — a binding an agent cannot read is
a promise, not a capability
Two tests now hold it. The unit test checks referenced ⊆ authored against
the files. The new integration test runs both loaders in boot order and
asserts the bindings survive the trip through the database, which is a
different question: resolution goes through skills_catalog rows, so a skill
file that exists but fails to ingest still leaves the role empty.
Negative controls: the unit test failed naming all 55; the integration test
fails naming the exact role when one name is reverted.
threejs.shader_author and .perf_engineer gained a second and third skill
after the collapse — pin_in_context pins idx < 2, so a role left with one
skill silently pins less than the policy intends.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
ba98c29481 |
fix(podcast): the left rail showed the workforce, not the episodes
The PODCAST tier had no branch in the left column, so it fell through to the default — the org/company/team roster. Opening the podcast page showed a list of agents, which is the one thing on that screen that has nothing to do with it. `PodcastList` now fills the rail with one card per episode (date, title, duration, size), the same shape `MissionsList` and `RepoList` give their tiers: objects in the rail, the selected one in the canvas. It selects the newest on first load so the canvas is never blank, and refreshes on the render sweep's own two-minute cadence so a new episode appears without a reload. `PodcastPanel` loses the duplicated list and becomes what a canvas should be: how to subscribe, and the selected episode with a player. The empty state points at the Continuous Research mission that produces one rather than just saying there is nothing. Verified on the served page: PODCAST renders between AGENT and REPOS. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
fe45f72f09 |
feat(podcast): a PODCAST tier, a topics field, and a feed a phone can actually reach
Three gaps between "the pipeline works" and "you can use it".
**1. Topics could not be set.** The wizard never sent `config.topics`, so every
mission created through the UI silently fell back to
`library::default_topics()` — a hardcoded list that is somebody else's research
interests. The card now takes one arXiv search per line, and the description
field says plainly that for this template it IS the brief the agents judge
relevance against.
**2. There was nowhere to see or subscribe.** New PODCAST tier in the left rail,
between AGENT and REPOS: the feed URL with a copy button, the episode list, and
an inline player for checking one at a desk. `GET /api/podcast/episodes` and
`/subscription` back it. The panel also reports how many missions produced no
audio, so a missing day reads as a known gap rather than silence.
**3. The feed 404'd for the only client that will ever request it.** Three
layers each assumed a browser:
- `resolveBearer` is server-only (`next/headers`), so a client component that
imported it broke the build outright. The panel now goes through the
same-origin proxy like every other panel, and the backend mints the feed URL
because the session lives in an httpOnly cookie JavaScript cannot read.
- The `/api` proxy demanded a session COOKIE. A podcast app has none and
carries `?token=` instead — the same shape as the existing `hooks/` prefix,
which is already exempt for exactly this reason.
- The local autologin middleware 307'd it to `/auth/autologin`. A podcast app
follows redirects blindly and would have stored an HTML page as the episode.
Neither exemption weakens auth: the backend still validates the token and
answers 401 to a bad one, verified. `episode_audio` accepts the token from
either the query string or an Authorization header, because the app fetches it
one way and the browser player the other, and refusing either breaks one of the
two ways this is listened to.
`CLAWMATES_PUBLIC_URL` matters and was wrong first: the tailnet root proxies to
a different service on :18789, and this frontend is on :8443. A feed advertising
an unreachable origin syncs silently forever, so `/subscription` returns a
`reachable` flag and the panel warns when it is still localhost.
Verified from a phone's point of view: feed 200 application/rss+xml over the
tailnet, enclosure 200 with 6,739,582 bytes of audio at 421s, bad token 401.
367 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
f4adc8d0f9 |
fix(podcast): stop reading identifiers aloud, and pitch the episode at a teenager
Two things the operator found by listening to a real episode.
**1. Identifiers were spoken as digit soup.** The script genuinely said
"arxiv 2608.12888", which the voice reads as "two six zero eight point one two
eight eight eight". Same for three-decimal values: "0.506" and "0.004" became
long strings of spoken digits. A listener on a treadmill cannot write an
identifier down and does not need a third decimal place.
`speakable()` strips arXiv references and bare identifier-shaped numbers, and
rounds decimals to two places — with a carve-out that matters: 0.004 rounds to
0.00, which would claim the value was ZERO when the whole point was that it
collapsed to nearly nothing, so it says "under 0.01" instead.
Deliberately narrow: it removes identifiers and shortens over-precise decimals,
and does not paraphrase, reorder or summarise. The agents' words are still the
episode. It also preserves the sentence's full stop — swallowing it turned
"…financial retrieval, arxiv 2608.00183. This one's a catch." into one run-on
sentence, and the pause is how a listener knows a thought ended.
Note that `podcast-dialogue-writing.md` ALREADY said "no arXiv ids" and the
writer included them anyway. That is this project's recurring lesson restated:
an instruction is a request, and a listener deserves a guarantee. The prose asks
and the code enforces.
**2. It was written for someone who already knew the field.** The skill and the
script phase's task now target a bright sixteen-year-old: define an acronym in
the sentence that first uses it, describe the mechanism rather than naming it
("a road map with motorways and side streets" instead of "a hierarchical
navigable small world graph"), one idea per sentence. The test offered is
whether the listener could explain the finding to a friend afterwards.
That is not dumbing down — it is the constraint that forces a writer to say what
a thing actually does rather than what it is called.
Tested against the exact lines from the episode that was listened to.
366 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
1f39f642a3 |
feat(podcast): render finished missions into episodes, and serve them as a feed
The renderer existed but nothing called it. This wires it to the missions and puts the result somewhere a phone can reach. **A sweep, not a phase step.** Rendering is not the agents' work and must not be able to fail a phase that succeeded; a transient API error simply retries next tick, and a mission already rendered is skipped because its episode row exists. `podcast_episodes` is that record — without it the sweep would re-render on every pass and re-bill for it, the same lesson `corpus_items` taught for papers. **It is racing a reaper.** script.md lives in the mission checkout, and `mission_runtime`'s sweeper deletes that tree 30 minutes after the mission reaches a terminal state. So the sweep runs every 2 minutes, leaving ~15 attempts inside the window. When it does lose — as it did for three missions that had completed hours before this shipped — it now SAYS so and records a marker rather than skipping in silence, which is how a feed ends up quietly missing a day. The feed filters those markers out: a zero-byte enclosure shows a broken episode in a podcast app, where showing nothing is honest. **Duration is read from the audio, not estimated from the script.** The feed advertises a length and that length should be the real one — and it is the check that catches a 6 MB file playing for six seconds. **The feed authenticates by query-string token**, because no podcast app can set headers. That is a real trade: the token lands in the app's database and any proxy log. It reuses `AuthService::authenticate`, so revoking the session revokes the feed with it rather than creating a second secret to forget to rotate. Titles are XML-escaped — one raw ampersand makes a client reject the WHOLE feed, not one episode. 363 tests pass. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
55b16f25c8 |
fix(podcast): the episode played for six seconds
Concatenating whole MP3 files does not make a longer MP3. Each TTS clip is a
standalone file: a small ID3v2 tag, then a first frame carrying an `Info`/`Xing`
VBR header that declares THAT CLIP's frame count. Joined raw, a player reads
clip one's header, believes the file is that long, and stops. The 6.1 MB
"episode" played for 6.9 seconds.
Caught by the operator listening to it. I had verified the byte count, the ID3
magic and a >100 KB size floor — every proxy for "this is audio" — and never
that it plays. The assertion I needed was duration, and none of the ones I wrote
could fail on this bug.
Measured on two real clips of 4.86s and 4.68s:
raw concat -> 4.86s only clip one plays
strip second clip's ID3 -> 4.86s
strip both clips' ID3 -> 4.86s the tag was never the problem
strip ID3 *and* the Info frame -> 9.53s correct
The ID3 tag is ~45 bytes and harmless. The header FRAME is what lies, and my
first attempt at a fix — scanning the joined file for `ID3` — was worse than
useless: it matched those bytes inside audio data and silently deleted half the
stream.
`strip_container` removes both from every clip, leaving pure frames a player
times from the stream itself. Real ElevenLabs output is committed as a fixture
so the test pins the actual wire format, not an approximation of it, and a
junk-input case proves a malformed clip fails the render rather than panicking.
Re-rendered the same script: 380.8s, up from 6.9s.
361 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
656662850d |
feat(podcast): render the episode from the agents' own script
GenFM is unreachable. `GET /v1/studio/projects` and `POST /v1/studio/podcasts` both return 403 — "Access to the Studio API requires your account to be explicitly whitelisted to use it. Please contact our sales team." Measured with two different keys on the account, so it is an account restriction, not a key scope. Plain text-to-speech on the same key returns a valid MP3. That suits the operator's choice better than GenFM would have. GenFM always runs its own LLM over the source, so the agents' script would have been REWRITTEN; rendering each line ourselves speaks it verbatim. The agents did the reading and the judging, and the episode says what they wrote. `AudioBackend` is a trait because every candidate has a different shape: NotebookLM documents no programmatic retrieval at all, GenFM needs a sales conversation, Gemini TTS is a third form. The renderer hands over a `Script` and gets bytes. `parse_script` is a parser rather than a `read_to_string` because structure must not be spoken: headings, rules and block quotes are skipped, a wrapped paragraph stays ONE turn (splitting per line would stutter at the seam), and a colon mid sentence does not start a new speaker — "The finding: recall dropped" would otherwise be truncated to everything after the colon. Voices are assigned by order of appearance, so a script using names instead of HOST/GUEST still alternates, and an unexpected third speaker falls back rather than failing. `from_env` returns None without a key, so a deployment with none produces no audio instead of failing a mission that otherwise succeeded. Proven end to end on the real script this morning's mission wrote: 25 turns, 887 words, 5,614 billable characters, 6.1 MB of MP3 in 33 seconds. The live test drives `parse_script` + `ElevenLabs::render` — the production path — and is `#[ignore]`d because it spends credits. 359 tests pass. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
850f11838b |
fix(research): the manifest belongs to the mission, and a brittle phrase must not mean silence
Two defects from the first on-topic run.
**1. The manifest could never be updated twice in a day.** It was written into
the VAULT at a per-DATE path, but it is per-RUN data. A second mission the same
day rewrites a file that already exists, and `auto_merge` correctly refused the
whole branch:
diff is not additive (1 non-add change(s), first:
M ContinuousResearch/2026-08-18/harvest.jsonl); left for a human
So `main` kept the FIRST run's manifest, the next mission cloned it, and the
agents analysed yesterday's papers while every log line reported a successful
harvest. The merge policy was right; the placement was wrong. The manifest now
goes into the mission's own checkout after `ensure_checkout`, which keeps the
vault additive and gives each mission exactly its own papers. The agents commit
it alongside their analysis through the normal delivery path.
**2. A quoted phrase that matches nothing looked like a quiet day.** Phrase
search is precise and brittle: "hybrid retrieval BM25 dense" is a reasonable
topic and appears verbatim in no paper on arXiv — measured, 0 hits — while the
same four terms unquoted return exactly the hybrid-retrieval evaluations the
topic asked for. Harvesting zero because of adjacency is indistinguishable from
a genuinely quiet field, which is the distinction `Harvest::healthy()` vs
`added_anything()` exists to preserve. `search` now retries unquoted when the
phrase finds nothing, and says so in the log.
Proven in one run, all three behaviours at once:
"approximate nearest neighbor search" -> 5 candidates, 5 already held, 0 shelved
"hybrid retrieval BM25 dense" -> no exact phrase match, retrying broad
-> 5 candidates, 0 already held, 5 shelved
"LLM as a judge evaluation" -> 5 candidates, 5 already held, 0 shelved
wrote 5 paper(s) to .../ContinuousResearch/2026-08-18/harvest.jsonl
The seen-set suppressing 10 of 15 is the whole point of a recurring mission, and
the 5 that landed are on topic for the first time: RAG architecture evaluation,
agent-controlled search over chat logs, compute-aware retrieval and reranking,
hybrid retrieval in hyperbolic space, sparse-dense fusion limits.
353 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
2cd0872e50 |
fix(papers): the arXiv topic was never actually searched for
The operator's topic went RAW into `search_query=`, unfielded. arXiv matched
essentially nothing, and `sortBy=submittedDate` then returned the newest
submissions across the entire archive — so the library shelved whatever had been
posted in the last few minutes and called it research.
A real run for "agentic topology", "retrieval augmented generation" and "vector
index pruning" shelved, among 13 papers: Galois extensions of geometric fixed
point spectra, a bulk path integral for a quantum black hole microstate, blazar
boosted dark matter in IceCube, and colloidal packing. Nothing was broken —
every layer reported success, the notes were written, the seen-set was updated,
the branch auto-merged. The papers were simply unrelated to anything asked for.
Measured against the live API:
speculative decoding -> pixel-space diffusion, simplicial actions
all:"speculative decoding" -> S2-MoE self-speculative decoding, DARTree
So a bare topic is quoted into `all:` and bound to `cat:cs.*`. The quotes make
it a phrase (unquoted, "vector index pruning" matches any paper containing all
three words anywhere, which is most of cs), and the category bound is needed
because the archive's physics and maths volume dominates any recency sort.
A topic that already starts with a field prefix passes through untouched, so an
operator who knows arXiv syntax keeps control.
The passthrough originally also accepted anything containing " AND "/" OR ", and
the injection test caught it on the first run: `agent" OR cat:hep-th` escaped the
phrase and rewrote the category bound. Only a LEADING field prefix counts now.
348 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
d524107b37 |
fix(missions): harvest before the checkout, and stop an unreachable judge failing done work
Two bugs from the first real Continuous Research run, both found by running it. **1. The harvest ran AFTER the checkout.** `on_launch` cloned the vault and then harvested, so the mission's working copy predated the manifest push. The reader agent found no `harvest.jsonl` and — being resourceful — queried arXiv itself and wrote its own. That is exactly what `skills/research/arxiv-daily.md` forbids: the papers it found are not checked off in `corpus_items`, so the next run re-offers them, while the 13 the real harvest DID shelve went unread. The harvest now runs first, so the clone contains the manifest. The analysis it produced was otherwise very good — it named `crates/clawhdf5-ann/src/hnsw.rs`, cited the ROADMAP's serial insert loop and proposed a concrete pre-build probe — which is the behaviour the whole design is for. It was reading the wrong papers. **2. An unreachable judge consumed a pass.** `Verdict.error` exists to distinguish "could not judge" from "judged incomplete" and nothing acted on it. glm-5.3 returned "transport error: error decoding response body", the phase counted it as a failed pass, and with two budgeted that single outage failed a phase whose work was done and committed. The evaluator was right to refuse a same-family fallback — that would trade independence for availability — so the fix belongs here: an unreachable judge no longer spends an iteration. Retrying forever would trade a wrong failure for an invisible hang, so the wait is bounded by `judge_blocked_since` (migration 0078), mirroring how `capacity_blocked_since` bounds a phase waiting on a VM slot. Thirty minutes is many sweep ticks, so a blip recovers inside it; past that the phase FAILS with the transport reason rather than requeueing, because re-running spends a container re-doing work that was never the problem. 346 tests pass. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
a02e0cba69 |
feat(missions): Continuous Research harvests at launch, and cards launch by clicking
The card shipped in
|
||
|
Omar SobhandClaude Opus 5
|
e20b321055 |
feat(missions): Continuous Research is a mission type, not just a team checkbox
`templates/teams/continuous_research.toml` has existed with three well-written roles since it was authored, but no workflow recipe pointed at it — every recipe in templates/workflows/ defaults `default_team_template = "rust_sdlc"`. So the only way to reach it was as a checkbox under Advanced. It is now a Step-1 card: the registry loads it at boot and `GET /api/workflows` serves it, with no frontend change (MissionWizard renders whatever the endpoint returns). Both phases are kind `research`, deliberately, rather than new `read`/`script` kinds. An unrecognised kind falls through `purposes_for`'s `_ => ["mission"]` and is absent from `PRODUCING_KINDS`, so it would get the generic directive AND be exempt from the empty-delivery rule — a phase that produces nothing and still passes. That is the shape this codebase keeps paying for; two `research` phases differentiated by `task` keep both guards. `commit_policy = "always"`, not `on_green_tests`: the vault is prose with no suite, so a test gate would find nothing to run and land every branch `-wip`. The harvest is NOT an agent phase. `continuous_research.rs` calls the existing `library::run_to_vault` — arXiv search, seen-set check, PDF shelf, vault note, attributed by `mission_id` — because that path is deterministic, takes seconds, and owns the `corpus_items` seen-set that is the whole reason a recurring mission knows what it already covered. An agent redoing it would be slower and would lose that. The manifest path is not invented either: the team template has told `signal_harvester` to write `ContinuousResearch/<date>/harvest.jsonl` all along. This makes the code produce what the prompt already promised, and a test pins the path and every documented key so the two cannot drift into an agent reading a file nothing writes. DEFAULT_CORPUS / DEFAULT_VAULT_URL exported rather than duplicated, so the route and the launch hook cannot disagree about which vault. 344 tests pass. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
f87853ecf9 |
fix(missions): scheduled missions never fired — nothing read missions.schedule
The wizard has collected a cron since `0047_missions.sql` ("schedule JSONB
carries the trigger config (cron | one_shot | on_event)"), the frontend posts
`{kind:"cron", cron}`, and the API persists it faithfully. Nothing has ever read
it back: the only due-work enumerator in the codebase was `routines::claim_due`.
So every scheduled mission ever created sat in `draft` forever while the UI
reported it was on a schedule.
Proven before fixing, on the shipped build: a mission with `* * * * *` sat in
`draft` for 4m34s and started ZERO topology runs. After this change the same
mission launched on its next occurrence and recorded one `fired` row.
Two pieces were missing, and they are the two `routines` already had:
- `missions.next_run_at` — schedule STATE. `schedule` is user intent and stays
untouched; without somewhere to record which occurrence is owed there is
nothing to put a `<= now()` predicate on, which is why no enumerator could
be written against the JSONB alone.
- `mission_fires` — one row per (mission, occurrence). 0063_routine_fires.sql
called this exact case: "For a scheduled *mission* it costs a container, a
repo checkout, and real money — which is why this lands before mission
scheduling does."
`mission_schedule.rs` deliberately mirrors `cm-scheduler`'s shape rather than
inventing a second one: atomic `FOR UPDATE SKIP LOCKED` claim, reschedule
BEFORE dispatch so a failing launch cannot stall the clock, claim the slot
before launching so a crash mid-launch is retried rather than dropped, and a
fan-out cap. The cap is 5, not the scheduler's 25, because a mission firing is
a container and a checkout where a routine firing may be one turn.
The claim skips `status = 'running'`: a daily cron on a mission that takes
longer than a day must skip the occurrence, not stack a second crew on the same
workspace. Launch goes through `mission_orchestrator::on_launch` +
`missions::set_status`, the same path as the draft→running transition, so one
code path mints a crew. An unattended launch acts as the workspace owner
(`users::owner_of_workspace`) since missions carry no creator column; a
workspace without one settles the occurrence `failed` with the reason rather
than dropping it silently.
Backfill blast radius was MEASURED, not assumed: prod has zero missions with a
cron, this workstation had exactly one — the control created to prove the bug.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
3cc65c22c4 |
feat(judge): room to analyse — and a panic in the evidence path
Three changes, one of them a live bug. **The bug.** `phase_summarizer` truncated agent output with `&s[..remaining]`, a BYTE slice of arbitrary UTF-8. Agent turn output routinely carries arrows, box-drawing and emoji, so a cut landing mid-character panics — taking down the evaluation sweep for that phase, triggered by nothing more than an agent writing a long enough line with a non-ASCII character at the wrong offset. Replaced with `clamp_to_char_boundary`, tested across every cut offset of a pure-4-byte string. It is precisely the bug the clawhdf5 agents found and fixed in `clawhdf5-migrate/src/validate.rs` this week — in our own code, in the path that feeds the judge. **Evidence budget** 60 KB -> 120 KB. Output headroom is worthless if the judge cannot see the work: the verdict is only as good as what reaches it. **Judge max_tokens** 2048 -> 16384. glm-5.3 is a reasoning model that spends most of its budget on a `thinking` block before writing the verdict, and running out mid-thought truncates it. A truncated verdict parses as empty and FAILS CLOSED, burning one of the phase's passes on a judge that never answered — how mission 01a00bbb lost one. Measured ceiling: z.ai accepts max_tokens up to 131072 on both glm-5.1 and glm-5.3 (131073 -> 400, "限制数值范围[1,131072]"), so 16384 is chosen for cost and latency rather than capability, and only emitted tokens are billed. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
9c4b0722e8 |
feat(judge): an independent GLM judge, on the newest model z.ai publishes
Every phase verdict this session was Anthropic grading Anthropic, and the boot
log said so on each start:
validator_preflight: no CLAWMATES_VALIDATOR_MODEL — phase verdicts are
judged by the house model, which is NOT an independent check
`evaluator.rs` already preferred a cross-provider judge and refused to call a
same-family one `independent`; the local stack simply had no non-Anthropic
credential. It now carries the same `glm` provider gw-04 has had all along —
`format = "anthropic"` is load-bearing, since z.ai's OpenAI-compatible endpoint
is ToS-throttled for raw SDK access while its Anthropic-format one is not.
Model: glm-5.3, the newest z.ai lists (4.5, 4.5-air, 4.6, 4.7, 5, 5-turbo,
5.1, 5.2, 5.3 as of 2026-08-17). gw-04 still runs glm-4.7.
glm-5.3 is a REASONING model: it emits a `thinking` block before its JSON. Our
SSE parser ignores `thinking_delta` and keeps the text, so the wire shape is
compatible — but on a realistic phase-evidence prompt it spent 819 of the
evaluator's 1024 output tokens. A longer phase would truncate the verdict, and
a truncated verdict parses as empty and FAILS CLOSED, burning one of the
phase's passes on a judge that never answered — precisely how mission 01a00bbb
lost a pass. max_tokens raised to 2048.
Measured before wiring: asked to judge 25 commits claiming INT-01..INT-25 with
tests passing, glm-5.3 returned met=false because the evidence never
established what the brief actually required. That skepticism is the point of
an independent judge.
Boot now reports: `validator_preflight: independent validator glm:glm-5.3
answered`.
The key lives in .env (gitignored), never in this file.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
b5032a732a |
fix(phase_runner): collect the agent's work BEFORE judging it
`Sandbox::for_mission` builds the judge's verification copy from the HOST
checkout. In copy mode the agents write inside the container, and their work
only reached the host when `sync_out` ran — in the capture sweep, AFTER the
phase closed. So every phase was judged against a tree that did not yet contain
the pass being judged, and the judge truthfully reported nothing there.
Mission 01a00cfa is the proof. Research pass 2 wrote a 434-line
IMPLEMENTATION_BRIEF.md, `cargo test` passed, and it was pushed to a clean
branch (clawmates/mission-01a00cfa-c69f39fd-i2 at 563cdd21). Its verdict:
failed after 2 pass(es) — met=false — research/IMPLEMENTATION_BRIEF.md
does not exist anywhere
logged one line BEFORE `captured (+434/-0 across 1 file(s))`. A phase that
succeeded was failed because the evidence had not been collected yet.
This hid because it only bites a phase judged on its OWN pass. The v2 coding
verdict cited real commits (339a5bd, 167671f) — research had already synced
that work to the host in an earlier phase.
`evaluate_finished_phases` now runs `sync_out` first, and on failure leaves the
phase `evaluating` for the next sweep rather than recording a verdict nobody
could stand behind — the same policy the capture sweep already applies, for the
same reason. microVM keeps its carve-out: `microvm_executor` collects out of
the guest over this same path before the VM is destroyed.
Research goes to 3 passes. On 01a00cfa it got no real attempts out of two: one
spent on a fabricated commit claim the judge correctly rejected, one on this
bug.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
d341640255 |
fix(mission_fs): drop build output when collecting work back from a container
`pack_dir` (host -> container) skips `transport_excludes`; `copy_out`
(container -> host) is the raw Docker archive API and carries the whole tree,
`target/` included. The asymmetry was invisible for as long as the runtime
image had no cmake — nothing could compile, so no `target/` existed.
The moment missions could actually build, every collection died on a build
artifact:
failed to unpack `…/repo/target/debug/build/ahash-…/build_script_build-…`
`phase_runner` then correctly refused to capture, rather than record a stale
tree as an empty diff — so mission 01a00c57's coding phase, which had done the
work, delivered nothing and retried forever. A fix that let missions compile
created a delivery failure one layer down.
`unpack_into` now skips excluded entries by NAME at any depth (a workspace has
a `target/` per crate) and logs how many it dropped.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
99dd29cc8a |
fix(worker): the stuck-run reaper was killing healthy sonnet-5 turns
REAP_STUCK_AFTER_SECS was 15 minutes; the runtime grants a single turn `timeout_secs = 3000` (50 minutes). A run journals its first step record when its first step COMPLETES, so a turn still legitimately in flight is indistinguishable from a wedged container — and with a window shorter than the turn timeout the reaper does not detect stuck runs, it kills slow healthy ones. The old value was calibrated on haiku, where "healthy first-step latency is typically 5-60s" held. Moving mission agents to sonnet-5 made first turns longer than the window: mission 01a00c41's research phase was reaped at 900s having already written +402/-39 across 13 files. We only know it was healthy because the delivery path captured and pushed that work anyway, to branch clawmates/mission-01a00c41-421200ee at b08df7b6. Raised to 60 minutes, above the turn timeout, with the invariant written down so the next person changing either number sees the relationship. Generalises: a liveness timeout calibrated against one model becomes a correctness bug when the model changes. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
69c294addc |
fix(models): coding runs on sonnet-5, judging on opus-5, haiku only as last resort
Operator model policy: haiku ONLY for genuine yes/no questions; anything
requiring thinking is opus-5; coding is sonnet-5.
The mission AGENTS were running haiku, and nothing in the product said so.
`provider_alias_for` maps every `claude-*` binding onto the single alias
`claude_cli.default`, so a crew whose `model_binding` reads `claude-sonnet-5`
— as this deployment's does — still ran whatever that alias pointed at, which
was `model = "haiku"` in the runtime config. The binding is cosmetic; the
alias is the truth.
Measured consequence on mission 01a00bbb: the coding agents claimed six INT
items complete and had committed three, and the done_when judge caught it by
auditing git history against the claims.
Model assignments, by what the component actually does:
evaluator (done_when judge) haiku -> opus-5 reads evidence, audits it
against the repo, writes
guidance. The verdict is a
boolean; the work is not —
and this is the one component
whose failure mode is passing
work that was never done.
judge_model 4-8 -> opus-5
mission_refiner 4-8 -> opus-5 composition
phase_summarizer 4-8 -> opus-5 composition
swarm planner 4-8 -> opus-5 planning
subscription preflight head 4-8 -> opus-5
fallback chain head 4-6 -> sonnet-5 haiku stays BELOW it as a
last-resort link, never a peer
Every value stays env-overridable; only the shipped defaults move.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
53da4d7e6d |
fix(runtime): a mission could not build the repo it was given
`clawmates-runtime` shipped with `gcc` and `make` but no `cmake`, no `g++` and no `python3-dev`. Measured on clawhdf5, three probes: no cmake → "is `cmake` not installed?" exit 101 after 13s no python3-dev → "cannot find -lpython3.11" exit 101 at link with both → cargo test PASSES exit 0 after 69s This is not only the delivery gate. The AGENTS run in this image, so a coding phase was writing Rust it had no way to compile or test — which reframes the last run's 11 agent commits as unverifiable by construction. `images/agent-toolchain/Dockerfile` (the microVM path) has had `cmake build-essential` all along, and its own header warns about precisely this: "if `cargo` is present in one image and absent in another, the same mission passes or fails depending on which backend it landed on, and nothing says why." Both images now install the same set — it was missing `python3-dev` too. `images/runtime-toolchain.Dockerfile` is a thin local overlay so the laptop can run today without recompiling zeroclaw from the fork; it is meant to be deleted once a runtime image built from the corrected deploy/ Dockerfile is published. Also: a build failure is no longer reported as a red suite. Both are cargo exit 101, and `verify_tests` mapped every non-zero to `Failed(code)` — so a missing toolchain was recorded as the USER's tests failing. It now returns `CouldNotRun` with the reason when the output shows a compile or link failure. Deliberately narrow: a failing `assert!` still reads as red, because letting broken code past `on_green_tests` is the expensive direction to be wrong in. Both directions are pinned by tests built from today's two real samples. And the coding phase finally has a loop: `research_and_code.toml` declared `loop = "until_no_more_int_items"`, which `phase_config.rs` lists as DECLARED_BUT_UNREAD. Iteration is driven by `max_iterations` + `done_when`, and with `max_iterations = 1` and no `done_when` the phase ran ONCE and was never judged — reporting `completed` whatever it produced. Now 3 passes against a stated goal, wording per the measured rule (say what the tree must CONTAIN). Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
10341cf7fe |
fix(missions): a retry's work is no longer silently destroyed
Two independent bugs, either of which loses everything a retried phase
produced, and neither of which reports a failure.
1. Capture is suppressed forever on a retry. Both
`capture_finished_coding_phases` and the sweeper's last-chance
`capture_outstanding_phases` skip any phase that already has a
`code_diff` artifact. That guard is right for a phase that ran once and
catastrophic for a retried one: the artifact from the FAILED attempt
suppresses capture of the new attempt, the container is reaped on its
normal grace, and everything the agents committed inside it is gone.
The UI keeps showing the old diff, so the mission reads as delivered.
`retry_phase` now clears the reopened phases' captures in the same
transaction that reopens them, which is what makes its own doc comment
("the phase card starts fresh on the retry") true of the artifacts too.
2. `git add` exits non-zero over a gitignored path while staging correctly.
Measured: with a populated `target/`, `git add -- . :(exclude)target`
exits 1 and stages the right files; `-c advice.addIgnoredFile=false`,
`--ignore-errors`, `-A` and `:/` all behave identically. Propagating
that with `?` aborted the commit AFTER a successful staging — no branch,
no commit, no push — for every Rust repo an agent has built in.
`capture_phase_diff_at` already treats the same command as advisory;
the commit path now does too, and the staged index decides.
Mission 01a00538 hit both: it completed research and coding on the retry,
11 agent commits and all, delivered a patch dated the previous day, and
lost the commits when the container was reaped. The remote was never
touched — its HEAD still equalled the mission's own base_sha.
Covered by a test that drives real git and asserts the files are staged
regardless of the exit code.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
fd5e71ccfe |
fix(missions): re-assert a mission's crew when its container is recreated
Claws are provisioned exactly once, at on_launch. `ensure_container`
RECREATES a container that is not running, and recreation reseeds
.zeroclaw from the seed directory — which does not hold this mission's
claws. The agents still exist in Postgres and the crew query looks
perfect, so nothing reads as broken; the alias is simply gone from the
daemon and /ws/chat answers 400 Bad Request. A retried mission could
therefore never connect again.
Re-assert the crew after ensure_container. provision_claw is idempotent,
so this costs one call per claw on the happy path and is the difference
between a resumable mission and a dead one.
Two things this has to get right, both of which fail silently:
- Aim at the per-mission daemon via for_gateway(ec.endpoint), never
from_env() — that targets the shared global gateway and leaves this
container with no claws at all, exactly as for_gateway's own doc
comment warns.
- Provisioning creates the agent but cannot set workspace.path (the
config prop-schema has no way to express it), so follow with
pin_agent_workspaces or every claw runs in its own sandbox and
delivers nothing.
Verified on mission 01a00538: research and coding phases both completed
after four straight failures, with all five graph aliases present and
ten claws pinned to /mission/repo.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
85a6038c08 |
fix(missions): create /mission before copying the checkout in
`copy_in` cannot create its own destination, so a mission whose container had no /mission directory failed its checkout sync outright. In copy mode that is how the agent gets the code at all, so the phase launched against an empty tree. Exec `mkdir -p /mission` as root first. Idempotent, and it costs one exec on a path that already shells out. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
6e8785f159 |
fix(missions): a server restart no longer kills a running mission
Mission 01a00538 ("ClawHDF5 REsearch and Refactor") failed 19 minutes and 93,762
tokens into its research phase with `pair failed: 403 Forbidden`, and its coding
phase was then correctly skipped as unreachable. The cause was not the coding
phase and not the model — it was pairing.
A per-mission runtime is authenticated with a SINGLE-USE pairing code, and the
bearer token it returns was cached in memory only. Any restart of the server
discarded that token; the next turn re-paired with a code the gateway had
already spent and got 403 — permanently, for that mission. A deploy, a crash or
an OOM would each do it. The durable-run machinery exists precisely so work
survives a restart; pairing was the one thread that did not, and it failed
closed.
`missions.runtime_token` persists the token at the moment pairing succeeds, and
the worker seeds the executor's cache from it, so a new process reuses the
credential instead of re-pairing. Persisting is best-effort: failing to save
must not fail a turn that just paired successfully.
Verified by reproducing the original failure: launched a mission, confirmed the
token was written, restarted the server MID-PHASE, and watched the mission run
to completion with no pairing failure.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
43436d7181 |
feat(telemetry): push bus for live agent frames
/api/world/live is a 2s database poll, which is right for queryable state and wrong for a token stream: reasoning only became visible after a step finished and its row was written. This adds a process-wide broadcast bus that topology_exec publishes to as the runtime's WebSocket delivers frames, and the SSE handler forwards without waiting for the next tick. Measured: the pushed frame arrived ~2.2s before the polled copy of the same text. Design notes worth keeping: - A global (OnceLock), not an AppState field. The publisher is reached through phase_runner -> topology_worker -> MissionTap, none of which hold AppState; threading a handle through all of them would put a UI concern into four layers that have no other reason to know about one. - Lossy by design. A slow subscriber lags and skips rather than applying backpressure to the agent producing. mission_events remains the durable record; this bus is the fast path, never the source of truth. - Only `claw_<uuid>` aliases are attributed. The governor, door and evaluator drive real turns under other names, and attributing their output to an agent would put words in someone's mouth. Asserted in a test. - The poll no longer emits `reasoning`: with both paths live, every turn arrived TWICE — once pushed, once polled ~2s later. The row is still written; this feed just is not its second mouth. CEILING, measured rather than assumed: turns are not token-level because the runtime is not streaming. zeroclaw's claude_cli provider runs `claude -p --output-format json`, which returns ONE result object when the turn completes — there are no incremental tokens to forward. Making this genuinely token-by-token needs `--output-format stream-json` and incremental parsing in the zeroclaw fork, not here. The bus is in place and will carry them the day it does. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
ba9d7aa185 |
feat(telemetry): WORKING ON NOW shows the mission an agent is on
The last of the three declared-but-never-emitted event types. `agent.task.update` had no producer anywhere in the backend, so the card read "idle — no active task" for an agent that was mid-turn. Derived rather than newly instrumented: an agent is working on its crew's RUNNING mission, and that mission's phases are the steps (completed/skipped → done, running/evaluating → active, else pending). Nothing is emitted for an agent with no running mission, so "idle" stays truthful rather than freezing on a stale last-known task. Verified on a live mission: 116 agent.task.update events observed on /api/world/live, carrying the mission title and phase steps, with the state advancing pending → active as the phase started. That closes the set. Of the seven cards in the command centre, five were dark: three had no emitter at all and two read a table the mission path never wrote. DOORS and LOOPS were correctly wired the whole time and were reporting an honest zero. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
bf40d10064 |
feat(telemetry): the reasoning stream actually streams
`agent.reasoning.delta` and `agent.tool.call` have been declared in the taxonomy and listened for by the command centre since it shipped — and NOTHING ever emitted them. The world feed emitted five types; neither was among them, so REASONING STREAM could not populate no matter what an agent did. The feed is a database poll, not a push bus, so a live card can only show what was persisted. The worker already holds each step's output text and the claw that produced it, so it records a `reasoning` mission_event (truncated — the card renders a tail, not a transcript, and mission_events is capped per phase), and the feed emits it forward from a cursor that starts at the current max so a page load streams rather than replaying history. `tool.call` is emitted from the same place. On the container tier it will stay empty, and that is correct rather than broken: those agents are tool-free behind the §15 door. Tool lines appear where agents actually hold tools. Verified on a live mission: agent.reasoning.delta observed on /api/world/live carrying the agent's own text, keyed by agentId. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
8be7b3c9b2 |
feat(telemetry): record per-agent usage for mission turns
The command centre's SPEND, ACTIVITY and THROUGHPUT cards read `usage_events`, and nothing on the mission path ever wrote a row: `cm_billing::charge` was called only from the agent-run path. Measured mid-mission with 14 agents live, `usage_events` was 0 while a crew had just burned 15k tokens — so an agent that had done real work reported zero cost and zero activity. The worker already knew everything needed: it logs node, role and token count per step, and the node's `attrs.agent` carries the `claw_<uuid>` binding the runtime dispatches on. This routes that to the ledger. `charge`'s run_id is now Option. `usage_events.run_id` references `agent_runs`, and a topology turn has no row there — passing its `topology_runs` id was a foreign-key violation, which is exactly what the first attempt hit. NULL is the honest value; the agent-run caller still passes its real id. The executor reports one total rather than an in/out split, so the cost is right (credits price the sum) and the columns record it as output rather than inventing a split. Verified end to end on a real mission: 4 agents, 1046-8670 tokens each, credits attributed per agent, and the SPEND/ACTIVITY queries now return real numbers. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
8ef7067467 |
fix(repos): a scoped connection can name a user, not just an org
Scoping a Gitea connection to `osobh` — the personal namespace clawmates itself
lives in — failed with "org 'osobh' not found or PAT lacks access". The sync
only ever called /orgs/{owner}/repos, and Gitea serves user namespaces from
/users/{owner}/repos. The error pointed at permissions for what was really a
wrong endpoint, which is the kind of message that sends you to rotate a token
that was fine.
Retry as a user on 404 before giving up, and say what was actually checked.
Verified: owner=osobh now syncs 7 repos, owner=redclaw 22 — 29 instead of the
182 an unscoped connection pulls.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
b290025fc4 |
feat(agents): make delete permanent, and expose the census
A soft delete marked the row and left it. The agent stayed in the table forever,
kept appearing on any surface that forgot `deleted_at IS NULL`, and deleting it
again did nothing — the decision was recorded and never honoured. Two agents on
this deployment had been in that state since June.
`deleted` is now a fifth lifecycle state, collected with NO grace window: a
human already decided, months ago. It takes usage_events with it, which is the
explicit trade — the alternative is rows that outlive the decision to delete
them.
Two endpoints, because this was previously only answerable by reading the
database by hand:
GET /api/claws/lifecycle the census: who is active, completed,
orphaned, deleted — and what is reapable
POST /api/claws/lifecycle/sweep run the reap now, rather than waiting out
the hourly timer for a decision already made
Verified end to end: census reported both as `deleted`/`reapable`, the sweep
returned {"reaped":2,"failed":0}, and agents and usage_events both went to 0.
The safety property is unchanged and re-asserted by a new test: adding `deleted`
did not make `owned` or `active` reapable.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
ccbc387f4b |
fix(agents): stop listing soft-deleted agents
Deleting an agent looked like a no-op: it disappeared from the workforce but
stayed on the Team board, and deleting it again did nothing because the row was
already marked. Two queries selected from `agents` without `deleted_at IS NULL`:
routes/team.rs the leaderboard — the surface still showing them
routes/world.rs the "working" set — a deleted agent holding a stale
agent_containers row rendered as live
Observed on this deployment: /api/workforce correctly returned nothing while
/api/team/leaderboard returned two agents soft-deleted back in June.
NOT changed: those rows still exist. Making delete permanent means hard_purge,
which also deletes usage_events — billing history, 6 credits on one of these
two. Discarding that as a side effect of tidying a roster is an explicit
decision, not something a display fix should smuggle in.
.sqlx regenerated: team.rs uses the compile-time-checked query! macro, so the
cached entry no longer matched.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
a494634f81 |
feat(agents): classify agents by lifecycle and reap the finished and orphaned
A mission mints a crew, and the only thing that reaped one was DELETING the mission. A mission that merely completed left its agents in the roster forever, and a crew whose reap was skipped or failed left agents bound to nothing — indistinguishable in the UI from the operator's own staff. Four states, from one query: owned no agent_template_link row → hand-created. NEVER reaped. active on a running/draft mission → working right now. Kept. completed every mission terminal → reaped after a 24h grace. orphaned minted, bound to nothing → reaped. The discriminator is `agent_template_link`, which mission_orchestrator writes per minted claw. This matters more than it looks: verified on live data, a hand-created agent and an orphaned crew member both have ZERO team links and are structurally identical by binding alone. Judging orphanhood by "no team" would delete the user's workforce. Provenance is the only honest signal. The grace window exists because the results view, the World's 24h replay and "who did this work?" all read the crew AFTER the run ends; reaping on the terminal transition deletes the answer exactly when the question gets asked. A completed crew with no usable timestamp is KEPT — a missing date must never read as "old enough to delete". Also fixes the delete summary, which reported how many claws were FOUND rather than purged: "reaped 4 claw(s)" was printed by a delete that purged none, which is precisely the log you would read while wondering why the agents are still there. It now reports purged / kept / FAILED, and failed > 0 is the orphan case. Verified against live data — all four states observed, including the two that look alike. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
548f977212 |
style: rustfmt the four files the repo-less mission fix touched
Found while removing .github/workflows/ci.yml: three of the four files in that change were unformatted, and four of the diffs were newly introduced (the new prompt tests and the tool_preamble format! call). Formatting only the files that change already touched — a repo-wide `cargo fmt` would be 63 files of unrelated churn and belongs in its own commit. Mechanical; `cargo test -p cm-api --lib` stays at 322 passed. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
4dec77ae6d |
fix(missions): give repo-less container missions the workspace they are promised
Every agent on a research_only mission refused to work, each reporting it was "in Claude Code", had no /mission/repo, and only had Read/Edit/Bash. All three statements were true. The run still recorded completed — 5 turns, 7.4k tokens, 0 artifacts, no error. The machinery is correct when a repo IS bound (verified on a live prod per-mission container: /mission/repo present, all 5 agents pinned). Only the repo-less path was broken, in three layers that disagreed by construction: - sync_in no-oped without a host checkout and copy mode does not bind /mission, so NOTHING created /mission/repo. The microVM tier already creates it, for the stated reason that "the guest needs the workspace to exist before the agent writes into it". Creating it host-side also un-breaks sync_out, equally a no-op before, so work survives across phases instead of being wiped. - pin_agent_workspaces returned Ok after pinning ZERO agents, so the deliberately-fatal guard in mission_orchestrator could never fire. Its error text already described the exact outcome we got. - The prompt advertised ZeroClaw tool names and explicitly denied `bash`, while every executor ends in `claude -p`: microVM passes Read/Edit/Write/Bash/Agent, session passes Read/Edit/Write/Bash, and claude_cli agents get Claude Code's native toolset — ZeroClaw's gating never reaches the subprocess. It was telling agents to use missing tools and avoid present ones. And it went green because mission_outputs logged the failed collect and continued — with the fail-empty rule and the NO-OUTPUT marker both BELOW that continue, so the phase was retried forever and never failed. The retry is now bounded by a grace window off completed_at. Verified end to end: mission completed, agent wrote /mission/repo/research/firecracker_vs_docker.md, collected and registered as a document artifact (6.6 kB of real content). Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
dc8f65fc64 |
fix(metrics): GPU, network and disk IO were arriving and being dropped
`gpu_pct` read `info.g` as a scalar. Beszel 0.18 puts GPU in a different
collection entirely, as a MAP keyed by GPU index —
`{"0":{"n":"GeForce RTX 5060 Ti","u":0,"p":4.38}}` in `system_stats.stats`
— and `systems.info` carries no `g` at all. So every NVIDIA node reported
null while the data sat one request away. Null and "no GPU" are
indistinguishable downstream, so the fleet card showed nothing and a
`gpu_pct` drain rule could never fire, both without an error.
`net_sent_ps`, `net_recv_ps`, `disk_read_ps` and `disk_write_ps` were
columns nothing ever wrote. They come from the same sample.
The two array orders were MEASURED, not read off a schema, because
inverting one does not fail — it reports upload as download forever:
b = [sent, recv]. `stats.ni` gives per-interface [sent_ps, recv_ps,
total_sent, total_recv]; indices 2 and 3 matched /proc/net/dev
tx_bytes and rx_bytes on all four of tank's interfaces, and `b` is
the sum of the per-second pair across them.
dio = [read, write]. An 800 MB dd on tank moved index 1 from 7441 to
23688 while index 0 stayed near zero.
`info.ct` is deliberately NOT mapped to container_count. It reads 1 on
tank, which runs 1 container, and also 1 on architect, which runs 4 —
right exactly often enough to pass a spot check.
One extra request per poll, not one per node: the newest 1m sample for
every system arrives in a single sorted page. A hub that cannot answer it
falls back to the info snapshot rather than losing the CPU and memory
readings that still work.
GPU is the busiest card, not the mean — placement asks whether there is a
free GPU, and averaging a saturated card with an idle one answers a
question nobody asked.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
9f76f0915b |
fix(runtime): announce the runtime image once, not on every sweep tick
`MissionRuntimeProvisioner::from_env` is called per use — on every mission
launch and from the terminal-mission reaper sweep — so the line added in
|
||
|
Omar SobhandClaude Opus 5
|
cdc45bd082 |
chore(runtime): promote v0.8.4 from canary to the default image
Every mission on gw-04 was already running v0.8.4 — pinned by CLAWMATES_RUNTIME_IMAGE in .env. The canary is retired: the default tag `clawmates-runtime:sync` now IS that image, the override is commented out, and the built-in default is the single source of truth again. Promoting it exposed why the pin was load-bearing in the first place. The default tag resolved to zeroclaw 0.8.3 — two releases behind what was actually running — and the REGISTRY copy of the same tag was a different image again, 849MB against 2.31GB, without the Rust toolchain. A host that pulled `sync` rather than retagging it would have lost the on-green test gate with every probe still reporting success. A moving tag pointing somewhere old resolves perfectly, starts perfectly, and runs old code. Nothing anywhere said which image a mission got, so two things now do: - mission_runtime logs the image it resolved and whether that came from the env override or the built-in default, once at startup. - runtime_preflight probes `zeroclaw --version` alongside the other tools and prints every tool's VERSION, not just that it is present. A presence check passes happily on an image two releases behind, which is exactly what happened here and was found by running the binary by hand. Rollback is a retag: `clawmates-runtime:pre-v084-default` on gw-04 holds the previous default, and .env.pre-v084-default holds the previous pin. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
d810fc0a86 |
fix(viz): read the gateway's real tool_call keys, and correct the record
The frame is `{"type":"tool_call","id","name","args"}` — zeroclaw-gateway
/src/ws.rs. The tap read `tool` then `name`, and `arguments` then `input`.
`name` happened to be in the fallback chain; `args` was not in it at all,
so a container-tier tool call would have been recorded with its name and
NO path — a tool that reads as having touched nothing. `tool` and
`arguments` belong to `approval_request`, which is where they came from.
Also corrects what the histogram was read as saying. A mission turn on
gw-04 carried only chunk/done/session_start, and the first reading was
"there is no tool_call frame". Wrong: `grep -c tool_call` on the deployed
0.8.3 binary returns 46. Container-tier agents are provisioned tool-free
behind the MCP door (§15), so they call nothing — there is nothing to
observe on that tier, and nothing is broken.
That distinction is exactly what the histogram was shipped to make
possible: a tap matching no frame is otherwise indistinguishable from a
mission that used no tools.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
31158467f4 |
feat(viz): microVM tool motion is live, and needs no fleet-node change
The plan deferred this as "the only fleet-node binary change". It is not one. `fcagent` is thread-per-connection — its own comment says so, and the live log tail has relied on exactly that for the whole length of a turn, on a second connection. So the host can drain the tap WHILE the turn's exec is in flight, from the server alone. The turn and a 20s drain loop now run concurrently. A coding phase shows its files being touched as it works rather than an hour later, all at once, and the drain is bounded by a cursor so a repeated poll returns only what is new. The cursor counts LINES, not parsed events, and that distinction is the bug this commit would otherwise have shipped. The hook appends the event and then a newline of its own, so a two-event tap is four lines; advancing by event count leaves the cursor two lines short, `tail -n +N` hands back events already recorded, and the live drain re-records everything it has already written — worse the longer the turn runs, and silent throughout. Caught while writing the test, not by it. `tap_sink` and `VmOutcome::tools` are mutually exclusive by contract: with a sink, the sink owns recording including the final batch and `tools` comes back empty. Handing the same calls back on both would double every file orb's weight with no way for the caller to tell which it was looking at. The sink is an unbounded channel to a recorder task, so the VM executor stays free of the database: it observes, phase_runner records. The task ends when the sender drops with the phase. Verified before this change: the microVM tap is real. The `microvm` scenario passed 6/6 and left ten `tool.call` rows and a `file.touch` on MICROVM.md, repo-relative, from Claude Code's own PostToolUse hook. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
f8438c32ea |
feat(viz): kind-specific choreography and a finished mission you can read
Security: the pawns already orbited their destination, so homing them at
the security station gave circling for free. This adds the radial
press-and-retreat — an agent closing on the target and backing off reads
as probing it, where a fixed radius reads as waiting — and holds the
stochastic target release while probing, or the circling breaks up into
stray trips that look like distraction rather than a scan.
Findings are `mission_tasks` rows, one orb each, popped once. There is
deliberately no severity anywhere in the path: the scanner keeps
severity, file and line as substrings inside `title`, so a severity
parsed out of prose and rendered as an orb's RADIUS would be the picture
asserting a measurement the data never contained. Count only.
Benchmarks annotate the station, as text. `delta` has no schema —
compute_delta emits `{kind:"opaque"}` whenever the before/after metrics
were not structurally comparable, which is most drivers. The server
formats the shape it can parse and COUNTS the rest; an unparseable driver
reports "3 sample(s)" rather than an invented improvement, and an opaque
delta says nothing at all.
The finished map: the live label rule gates service/event nodes on
`heat > 0.12`, which is exactly backwards once everything has cooled — a
static map would be unlabelled dots. Frozen, the 25 most-touched nodes
label regardless of heat, phase stations carry a second line counting
what they produced, and the camera is released ONCE so it frames the
result even if the user panned during the run.
Every count in a caption is read off the drawn scene rather than a
parallel tally, so the words and the picture cannot disagree.
Co-Authored-By: Claude Opus 5 <[email protected]>
|