Omar SobhandClaude Opus 5.5
9ca71e5fa4
The first self_audit run on a planted brief/condition mismatch found the pattern and quoted it, then diagnosed "the agent skipped the section": the record held the condition but not the brief, and working agents never see the condition. It also read two missions as one — a UUIDv7's first 8 chars are a timestamp, and missions 34 s apart both rendered as 01a0cb38. - verdict_line records the phase brief (config.task) beside the condition - the short mission id is the uuid tail - self_audit copies the record into the checkout (the judge cannot read /mission/memory) and compares brief with condition Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
51 lines
4.4 KiB
TOML
51 lines
4.4 KiB
TOML
key = "self_audit"
|
|
title = "Self-audit"
|
|
blurb = "Read every judge verdict this repository's missions have earned, find what keeps failing, and propose evidence-backed changes to how the work is set up."
|
|
# The subject is the repo brain, exported into the mission as
|
|
# /mission/memory/PROJECT-MEMORY.md. It exists only for missions with a repo.
|
|
requires_repo = true
|
|
|
|
# `continuous_improvement` existed without a recipe, so every mission that used
|
|
# it had to write its own brief — and the brief is the only thing that aims a
|
|
# mission. Role `system_prompt`s are inert on mission turns (they reach chat,
|
|
# not missions). Measured 2026-09-22: with the role prompts retargeted at the
|
|
# record and a brief left over from v1, 0 of 50 tool calls read the record and
|
|
# the judge passed an audit of a ROSTER.md. With the brief below, the record
|
|
# was read and the audit got the right answer. See docs/TEMPLATE-MATURITY.md.
|
|
default_team_template = "continuous_improvement"
|
|
|
|
[[phases]]
|
|
kind = "research"
|
|
order_idx = 0
|
|
[phases.config]
|
|
default_topology = "pipeline"
|
|
task = """
|
|
Audit what this repository's missions have learned.
|
|
|
|
The record is /mission/memory/PROJECT-MEMORY.md: one line per judged phase of every mission on this repository — MET or UNMET, the phase kind, the brief the agent was given (newer lines), the completion condition it was judged against, and the judge's reason. Working agents are NOT shown their completion condition; they see only the brief. That record is the whole subject of this audit. Do not look for agent brain files, rosters or anything else to audit; there is nothing else.
|
|
|
|
If the file is absent, the repository has no judged history yet. Write that in research/IMPROVEMENT-AUDIT.md and stop — that is a complete audit.
|
|
|
|
Otherwise, first copy the record verbatim to research/PROJECT-RECORD.md. The judge of this audit cannot read /mission/memory; that copy is how your quotes get checked.
|
|
|
|
1. Count the lines, and how many are MET and how many UNMET. State the numbers.
|
|
2. Look for patterns, not incidents: the same kind of work failing more than once, the judge asking for the same missing thing more than once, a condition that passed only after several iterations. One UNMET line is an incident, not a pattern — name it as an incident and propose nothing for it.
|
|
3. For each pattern, quote the record lines it rests on, copied verbatim as they appear in the file. A paraphrase or a summary table is not evidence; the quoted lines are.
|
|
4. For each pattern, propose ONE change to how the work is set up — a reworded completion condition (old and new wording), a missing skill, a recipe setting, a task brief — and say why it would have turned those UNMET lines into MET. A proposal that fewer than two lines support is dropped, not softened.
|
|
When a line carries a brief, compare it with the condition. A requirement that appears in the condition and not in the brief was never asked of the agent — that is a setup defect, not the agent skipping work, and the fix belongs in the brief, not in the agent's discipline.
|
|
5. If the record shows no failure pattern, say so, and say that no change is warranted. That is a correct result, and on a thin or clean record it is the expected one. Never invent a pattern to have something to report.
|
|
|
|
You cannot apply anything and there is nothing to submit to. Write everything to research/IMPROVEMENT-AUDIT.md; the operator decides.
|
|
"""
|
|
# Wording follows the measured rule: say what the file CONTAINS. The v3 run's
|
|
# condition said "quotes the record lines each finding rests on" and the judge
|
|
# failed an audit with no failure patterns for paraphrasing its observations.
|
|
# The judge cannot read /mission/memory, so iteration 0 of the first recipe run
|
|
# failed as "could not verify the quotes"; the agent then copied the record into
|
|
# the checkout unprompted and passed. The copy is now part of the brief.
|
|
# The quoting requirement now attaches to what needs evidence — a reported
|
|
# pattern — and the clean-record outcome is named so it reads as a pass.
|
|
done_when = "research/IMPROVEMENT-AUDIT.md exists and either states that no project record was present, or research/PROJECT-RECORD.md holds the record's lines and the audit states how many verdict lines the record contains with the MET and UNMET counts and then, for each failure pattern it reports, quotes at least two record lines verbatim and proposes one change, or states that the record shows no failure pattern and no change is warranted"
|
|
max_iterations = 2
|
|
commit_policy = "always"
|