fix(memory): verdict lines carry the brief and a distinguishing mission id
deploy / test (push) Successful in 4m50s
deploy / build (push) Successful in 5m45s

The first self_audit run on a planted brief/condition mismatch found the
pattern and quoted it, then diagnosed "the agent skipped the section": the
record held the condition but not the brief, and working agents never see the
condition. It also read two missions as one — a UUIDv7's first 8 chars are a
timestamp, and missions 34 s apart both rendered as 01a0cb38.

- verdict_line records the phase brief (config.task) beside the condition
- the short mission id is the uuid tail
- self_audit copies the record into the checkout (the judge cannot read
  /mission/memory) and compares brief with condition

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
Omar Sobh
2026-09-22 17:42:37 -05:00
co-authored by Claude Opus 5.5
parent fca828b5a1
commit 9ca71e5fa4
4 changed files with 96 additions and 15 deletions
+35 -1
View File
@@ -65,7 +65,7 @@ holding (it was 55 of 85 dangling once).
| `codebase_research` | code_archeologist architecture_mapper flow_tracer vault_scribe | research_readonly | 11 | **yes** — first run 2026-09-22 |
| `papers_research` | domain_scout paper_reader library_curator | research_web_readonly | 8 | **yes** — first run 2026-09-22 |
| `insight_research` | implementation_tracker novelty_hunter publication_drafter | research_readonly | 8 | no |
| `continuous_improvement` | brain_inspector improvement_proposer improvement_evaluator | research_readonly | 7 | **reaches its subject; right answer, judge-failed on quoting** |
| `continuous_improvement` | brain_inspector improvement_proposer improvement_evaluator | research_readonly | 7 | **finds real patterns; diagnosis blocked by the record (fixed, unre-run)** |
**Six of twelve are evidenced.** `backend` was exercised on 2026-09-22,
the first time anything had been staffed from it: all five roles
@@ -150,6 +150,40 @@ part is the case this team exists for, a record *with* failures. That
needs a repository whose history contains UNMET verdicts, and a default
brief so no caller has to write one. Still not counted as evidenced.
**The `self_audit` recipe, and a record with real failures (2026-09-22).**
The brief now lives in `templates/workflows/self_audit.toml`, so a mission
naming only `template_kind = "self_audit"` and a repo is staffed and aimed
without the caller writing anything. To give it something to find, two
missions were planted on the scratch repo whose `done_when` required a
*Limitations* section their brief never mentioned. The judge marked both
UNMET for exactly that. The record then held 7 lines: 4 MET, 3 UNMET, the
third being v3's quoting failure, a natural single-incident control.
Mission 01a0cb3f, launched with no brief of its own, read the record (8 of
25 tool calls), stated 7 / 4 / 3, found the one pattern, quoted both lines
verbatim, and named the quoting failure as an incident with no proposal. The
judge passed it on iteration 1 of 2.
Its **diagnosis was wrong**, and the record is why. It concluded the agents
"produced the main content but omitted the final required section" and
proposed a write-then-check checklist. The recorded prompts show the agents
were never told: none of the three mentions *Limitations*, because working
agents are not shown their completion condition. The checklist would not have
helped. The record carried the condition and not the brief, so a
brief/condition mismatch was invisible to any reader of it. Three fixes
followed:
- verdict lines now carry the phase's **brief** beside its condition;
- the mission id in a line is the uuid **tail**: the two planted missions,
launched 34 s apart, both rendered as `01a0cb38` (a UUIDv7 leads with its
timestamp), and the audit read them as one mission;
- the recipe has the auditor **copy the record** into the checkout, since the
judge cannot read `/mission/memory`. Iteration 0 failed as "could not verify
the quotes" until the agent did this unprompted.
Counts, pattern-vs-incident discipline and verbatim evidence: shown. A
correct diagnosis of a cause: not yet shown, and until now not possible.
**Two of the remaining six are still unevidenced**, and the other four
are blocked on a target stack. The other nine are well-formed scaffolding:
roles, prompts, brain seeds and resolving skills, and no run behind any of