Commit Graph
9 Commits
Author SHA1 Message Date
Omar SobhandClaude Opus 5.5 db06c7d936 docs: frontend team delivers correct work; the offline judge cannot run npm
deploy / test (push) Successful in 5m7s
deploy / build (push) Successful in 1m1s
Mission 01a0cbc7 on the new clawmates-frontend-scratch repo: accessible Tabs,
10/10 tests and tsc clean when re-run locally from the delivered branch, but
failed both iterations because the judge's copy excludes node_modules and
clawmates_core has no route to the registry. Blocks every npm team. Also fixes
the stale continuous_research row and adds self_audit to the recipe table.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-22 20:49:54 -05:00
Omar SobhandClaude Opus 5.5 509b7ceb89 docs: continuous_improvement evidenced via self_audit — 7 of 12
deploy / test (push) Successful in 5m15s
deploy / build (push) Successful in 1m3s
With briefs in the record, the audit named the planted brief/condition
mismatch as a setup defect and proposed the brief change, and independently
found its own earlier quoting failures and proposed the fix already made.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-22 18:01:15 -05:00
Omar SobhandClaude Opus 5.5 9ca71e5fa4 fix(memory): verdict lines carry the brief and a distinguishing mission id
deploy / test (push) Successful in 4m50s
deploy / build (push) Successful in 5m45s
The first self_audit run on a planted brief/condition mismatch found the
pattern and quoted it, then diagnosed "the agent skipped the section": the
record held the condition but not the brief, and working agents never see the
condition. It also read two missions as one — a UUIDv7's first 8 chars are a
timestamp, and missions 34 s apart both rendered as 01a0cb38.

- verdict_line records the phase brief (config.task) beside the condition
- the short mission id is the uuid tail
- self_audit copies the record into the checkout (the judge cannot read
  /mission/memory) and compares brief with condition

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-22 17:42:37 -05:00
Omar SobhandClaude Opus 5.5 e789f6f7ea docs: continuous_improvement reaches its subject; the brief, not the role prompt, aims it
v2 got the record delivered but never read it (0/50 calls; role prompts are
inert on missions and the reused v1 brief pointed at brains). v3, with a brief
aimed at PROJECT-MEMORY.md, read it, reported 4 MET / 0 UNMET and proposed
nothing, and was failed by the judge for summarising rather than quoting.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
2026-09-22 15:50:29 -05:00
Omar SobhandClaude Opus 5.5 a260499060 docs: continuous_improvement runs honestly and cannot reach its subject
deploy / build (push) Canceled after 0s
deploy / test (push) Canceled after 4m16s
Its first run separated running from working. All three roles handed off
with attributed commits; it filed zero level-up proposals, committed "no
proposals warranted, evidence base too thin", and opened its audit by
naming the problem exactly: no .brain is accessible from the mission
filesystem because brains are held in the platform.

That is structural. The template's subject is every agent's .brain,
which lives in the server's /data/brains volume, and nothing delivers
brains into a mission — the same defect class as research and skills
having had no delivery channel. It audited the only agent-shaped thing in
reach, a ROSTER.md in the scratch repo, and read a guest kernel version
as an agent version. Not counted as evidenced.

A channel alone would not fix it: per-mission crews carry ~2 KB seed
brains with no history, because missions write memory to the repo brain.
The template assumes long-lived agents; the platform makes disposable
ones. Recorded as a design decision, not patched.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
2026-09-22 15:22:56 -05:00
Omar SobhandClaude Opus 5.5 17e6d08cd1 docs: papers_research evidenced — 6 of 12, and its citations are real
deploy / build (push) Canceled after 40s
deploy / test (push) Successful in 5m9s
First run of the papers_research team: a bounded library on microVM and
sandbox isolation for agents, delivered as a README index plus four
notes with complete frontmatter, judged met.

The judge could only confirm the frontmatter was present. A paper team
whose container has no pdf-to-text tool is exactly where invented
citations appear, so every arXiv id was checked against the arXiv API:
all four exist with exact title matches. One, 2603.02277
(SandboxEscapeBench), was already cited in the operator's own research
pass — independent evidence of on-topic work, not plausible filler.

It also confirmed ToolSearch by a real run, one of the allowlisted tools
the task-permission shadow had not yet seen.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
2026-09-22 15:17:02 -05:00
Omar SobhandClaude Opus 5 a88d51b078 docs: codebase_research evidenced — 5 of 12, and its citations check out
deploy / test (push) Successful in 5m15s
deploy / build (push) Successful in 1m27s
Run against the real clawmates repo with a task that cannot be faked
from filenames: trace every hop of a mission agent's tool call from the
guest hook to the recorded event, naming file, function, and guest vs
host for each.

It produced a 189-line research/GATE-MAP.md describing code committed
the SAME DAY — the four-field NODE_EXTRACT including agent_type, the
shadow-mode would-deny.jsonl semantics, install_with — and all six of
its line citations verify exactly. Nothing hallucinated. That is a
stronger result than the judge's verdict, which could only check that
the structure was present.

Also recorded: agents reach for Bash. 74 Bash calls against 34 Read, 3
Glob, 1 Edit, 1 Agent — a mapping mission that could have used Grep used
Bash 33 times. So the allowlist's dedicated-tool entries may never be
exercised, and the surface that actually needs governing is Bash, which
the floor rules already cover.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
2026-09-22 15:09:22 -05:00
Omar SobhandClaude Opus 5 b1a23727da docs: backend team evidenced — 4 of 12
deploy / test (push) Successful in 5m35s
deploy / build (push) Canceled after 40s
First run of the backend template, 2026-09-22. All five roles
provisioned with real agents; the mission delivered cursor pagination
(Paged<T>, paginate<T: Clone>, module declared, cargo test 3/0) and was
judged met on the FIRST pass, 2 files pushed.

One run is not maturity, but it answers the question the review could
not: whether anything staffed from this template runs at all. It does.

It also gave the task-permission shadow its first confirmed Edit call —
Bash 8, Read 2, Write 2, Edit 1, zero would-deny.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
2026-09-22 15:03:05 -05:00
Omar SobhandClaude Opus 5 594cd99e54 docs: template maturity — 3 of 12 teams evidenced, 2 of 6 recipes proven
deploy / test (push) Successful in 5m0s
deploy / build (push) Successful in 1m1s
A review of what our agents can actually be asked to do, graded by
evidence rather than by what the TOML declares.

Recipes: research_and_code (13 harness scenarios) and research_only
(staffing measured and corrected — 1 of 9 applicable skills under the old
rust_sdlc default, 4 of 4 under topic_research) are proven.
security_hardening, benchmark and refactor are exercised once each.
continuous_research is the outlier: the most moving parts of any recipe,
ran twice today, and NOTHING in the harness would notice if it broke.

Dead keys the recipes lean on, from phase_config::DECLARED_BUT_UNREAD:
loop, produces, input_from_phase, and mcp_bundles at phase level. Not
hidden — security_hardening.toml annotates its own decoration inline, and
the other five should copy that. The risk is a reader taking
loop = "until_no_more_int_items" for a loop.

Teams: all 12 resolve every declared skill (0 unbound, the
skill-binding-repair work holding), but only rust_sdlc, topic_research
and continuous_research have a run behind them. The other nine are
well-formed scaffolding. Five are blocked on a target stack we do not
have a repo for; four could be exercised against repositories we already
have, and nobody currently knows whether they run at all.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
2026-09-22 10:26:19 -05:00