clawmates/crates/cm-api/src/vm_stop_gate.rs

//! The completion gate, moved into the agent's own loop.
//!
//! Every check this platform has on a phase runs **after** the agent has
//! finished: the evaluator judges `done_when`, capture notices that a coding
//! phase delivered nothing, and either verdict costs a whole new VM — a fresh
//! boot, a fresh inject, and an agent starting again with none of the context
//! that got it that far. Meanwhile the documented failure of a long-running
//! agent is that it *stops too early*.
//!
//! Claude Code's `Stop` hook is the seam. **Exit code 2 blocks the stop and
//! feeds stderr back to the model as the reason.** Measured, not read off docs —
//! an agent told "say hello and do nothing else", whose `Stop` hook exited 2
//! saying `evidence.txt` was missing, created `evidence.txt` and then stopped.
//!
//! # What it may and may not check
//!
//! Deliberately mechanical: whether the repository changed, and whether a
//! command the phase author wrote exits 0. NOT the `done_when` verdict — that is
//! an LLM judgement made host-side by a *different provider* on purpose
//! ([[evaluator-verification]]), and re-implementing it inside the VM would put
//! the agent's own environment in charge of grading the agent, which is the
//! correlated failure the independent judge exists to break.
//!
//! # The cap is load-bearing
//!
//! A gate with no ceiling turns a stuck agent into a wedged one: it would be
//! blocked, retry, be blocked again, and burn the hour-long turn budget instead
//! of failing in a way the operator can see. After [`MAX_BLOCKS`] the gate lets
//! the agent stop, records that it did, and leaves the verdict to the existing
//! post-hoc path — which still runs, unchanged.
//!
//! # Which hooks exist here
//!
//! `TaskCompleted` / `TeammateIdle` were the plan's chosen seam. Measured under
//! `claude -p`: they never fire, because no team forms in print mode at all.
//! `Stop`, `SubagentStop`, `PreToolUse`, `PostToolUse`, `UserPromptSubmit` and
//! `SessionStart` do.


/// How many times the gate may refuse a stop before it gives up and lets the
/// agent finish. Three is enough for "you wrote nothing" → "you wrote something"
/// → "your check passes" without ever approaching the turn budget.
pub const MAX_BLOCKS: u32 = 3;

/// The file the gate writes when it gives up and lets the agent stop with its
/// condition still failing.
///
/// A separate file rather than a line in the log, because the log is not
/// parseable for this: a block reason embeds the check's own output, and an
/// output line beginning `cap:` would read as a cap release that never happened.
///
/// It exists because the block COUNT cannot answer the question. Three blocks
/// followed by a stop that finally passed, and three blocks followed by a
/// release at the cap, both report `blocks: 3` — and they are opposite outcomes.
/// Without this, the second one completed the phase green.
pub const CAPPED_FILE: &str = "capped";

/// Where the gate lives in the guest.
///
/// Under `/root`, never under the repository. Anything written into
/// `/mission/repo` is collected and diffed, so a gate script placed there would
/// arrive in the user's delivered patch as if an agent had authored it.
pub const GATE_DIR: &str = "/root/gate";

/// What must hold before this phase's agent is allowed to stop.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct StopGate {
    /// The phase must leave the repository changed. Set for coding phases that
    /// have not declared `allow_empty` — the same rule
    /// `empty_delivery_is_a_failure` applies post-hoc, applied while the agent
    /// can still do something about it.
    pub require_changes: bool,
    /// `config.done_when_check`: a shell command, run in the repo, that must
    /// exit 0. The deterministic half of a completion condition — a command,
    /// not a judgement.
    pub check: Option<String>,
}

impl StopGate {
    /// The gate for a phase, or `None` when there is nothing to enforce.
    ///
    /// `None` matters: installing a hook that can never block would still cost a
    /// process per stop and would put a `--settings` flag on the command line
    /// for no reason.
    pub fn for_phase(kind: &str, config: &serde_json::Value) -> Option<StopGate> {
        let allow_empty = config.get("allow_empty").and_then(|v| v.as_bool()) == Some(true);
        let check = config
            .get("done_when_check")
            .and_then(|v| v.as_str())
            .map(str::trim)
            .filter(|s| !s.is_empty())
            .map(str::to_string);
        let require_changes = kind == "coding" && !allow_empty;
        if !require_changes && check.is_none() {
            return None;
        }
        Some(StopGate {
            require_changes,
            check,
        })
    }

    /// The same gate, for ONE NODE of a composed run.
    ///
    /// `require_changes` is a property of the phase, not of every node in it: a
    /// graph whose second node reviews or verifies is *supposed* to leave the
    /// tree alone, and a per-node gate would refuse its stop three times for
    /// doing exactly its job. Dropping it loses nothing, because
    /// `empty_delivery_is_a_failure` applies the same rule post-hoc to what the
    /// phase as a whole delivered.
    ///
    /// That "post-hoc" claim used to be written as covering the `check` too. It
    /// did not: nothing outside this hook has ever re-run `done_when_check`, so
    /// a release at [`MAX_BLOCKS`] completed the phase green with the check
    /// still failing. [`CAPPED_FILE`] is what closes that.
    ///
    /// A declared `check` DOES apply per node: it is a command the phase author
    /// wrote, and every stage of the work should satisfy it.
    pub fn per_node(self) -> Option<StopGate> {
        self.check.map(|check| StopGate {
            require_changes: false,
            check: Some(check),
        })
    }

    /// The hook script, as POSIX `sh`.
    ///
    /// `repo` and `dir` are parameters rather than the constants above so a test
    /// can run this script — the real one, not a paraphrase — against a real git
    /// repository in a temp directory.
    pub fn script(&self, repo: &str, dir: &str) -> String {
        let mut s = String::from("#!/bin/sh\n# ClawMates stop gate. Exit 2 refuses the stop.\n");
        s.push_str(&format!("REPO={}\nGATE={}\nMAX={MAX_BLOCKS}\n", q(repo), q(dir)));
        s.push_str("N=$(cat \"$GATE/blocks\" 2>/dev/null || echo 0)\nreason=''\n");

        if self.require_changes {
            // Two questions, because either alone is answerable "no" by a
            // perfectly good phase: an agent that committed its work leaves a
            // clean tree, and an agent that did not commit leaves HEAD where it
            // was. Only both together mean nothing happened.
            s.push_str(
                "BASE=$(cat \"$REPO/.git/clawmates-base\" 2>/dev/null || echo '')\n\
                 DIRTY=$(git -C \"$REPO\" status --porcelain 2>/dev/null | head -c 400)\n\
                 HEAD=$(git -C \"$REPO\" rev-parse HEAD 2>/dev/null || echo '')\n\
                 if [ -z \"$DIRTY\" ] && [ -n \"$BASE\" ] && [ \"$HEAD\" = \"$BASE\" ]; then\n\
                 \x20 reason='This phase has changed nothing: the working tree is clean and \
                 HEAD is still the commit you started from. Do the work the task describes \
                 and leave it in the tree. If the task genuinely requires no code change, \
                 say so explicitly in your final message.'\n\
                 fi\n",
            );
        }

        if let Some(check) = &self.check {
            s.push_str(&format!(
                "if [ -z \"$reason\" ]; then\n\
                 \x20 out=$(cd \"$REPO\" && sh -c {} 2>&1); rc=$?\n\
                 \x20 if [ \"$rc\" -ne 0 ]; then\n\
                 \x20   reason=\"This phase's completion check exited $rc, so the work is not \
                 done yet. The check is: {}\n\nIts output:\n$(printf '%s' \"$out\" | tail -c 1500)\"\n\
                 \x20 fi\n\
                 fi\n",
                q(check),
                // Inside a double-quoted assignment, so the command text itself
                // must not carry a `\"` or a `$` that the shell would expand.
                check.replace('\\', "\\\\").replace('"', "'").replace('$', "\\$"),
            ));
        }

        s.push_str(
            "if [ -z \"$reason\" ]; then echo pass >> \"$GATE/log\"; exit 0; fi\n\
             if [ \"$N\" -ge \"$MAX\" ]; then\n\
             \x20 echo \"cap: $reason\" >> \"$GATE/log\"\n\
             \x20 echo 1 > \"$GATE/capped\"\n\
             \x20 exit 0\n\
             fi\n\
             N=$((N+1)); echo \"$N\" > \"$GATE/blocks\"\n\
             echo \"block $N: $reason\" >> \"$GATE/log\"\n\
             printf '%s\\n' \"$reason\" >&2\n\
             exit 2\n",
        );
        s
    }

    /// One shell command that writes the gate SCRIPT into the guest.
    ///
    /// It deliberately does NOT write `settings.json`. It used to, and it wrote
    /// the whole document — so the moment a second feature needed a hook, the
    /// later writer would silently erase this one. The composed document is
    /// built in exactly one place: [`crate::vm_tool_tap::guest_settings`].
    ///
    /// Written by `printf` through an exec rather than injected as part of the
    /// tar: the tar lands in `/mission/repo`, which is exactly where this must
    /// not be.
    pub fn install_command(&self, repo: &str, dir: &str) -> String {
        format!(
            "mkdir -p {d} && rm -f {d}/blocks {d}/log {d}/capped \
             && printf '%s' {script} > {d}/stop-gate.sh \
             && chmod +x {d}/stop-gate.sh",
            d = dir,
            script = q(&self.script(repo, dir)),
        )
    }
}

/// Single-quote for `sh`. Same rule as `microvm_executor::shell_quote`, kept
/// local so this module has no dependency on the executor it is used by.
fn q(s: &str) -> String {
    format!("'{}'", s.replace('\'', r"'\''"))
}

#[cfg(test)]
mod tests {
    use super::*;
    use serde_json::json;
    use std::path::Path;
    use std::process::Command;

    fn sh(script: &str, dir: &Path) -> std::process::Output {
        let path = dir.join("stop-gate.sh");
        std::fs::write(&path, script).unwrap();
        Command::new("sh").arg(&path).output().expect("run the gate")
    }

    /// A git repo with one commit and the clone-point marker the real checkout
    /// carries (`mission_workspace::record_base_commit` writes it).
    fn repo_with_base(root: &Path) -> std::path::PathBuf {
        let repo = root.join("repo");
        std::fs::create_dir_all(&repo).unwrap();
        let git = |args: &[&str]| {
            let o = Command::new("git")
                .arg("-C")
                .arg(&repo)
                .args(args)
                .output()
                .unwrap();
            assert!(o.status.success(), "git {args:?}: {:?}", o);
        };
        git(&["init", "--quiet"]);
        git(&["config", "user.email", "t@t"]);
        git(&["config", "user.name", "T"]);
        std::fs::write(repo.join("README.md"), "base\n").unwrap();
        git(&["add", "."]);
        git(&["commit", "--quiet", "-m", "base"]);
        let head = Command::new("git")
            .arg("-C")
            .arg(&repo)
            .args(["rev-parse", "HEAD"])
            .output()
            .unwrap();
        std::fs::write(
            repo.join(".git/clawmates-base"),
            String::from_utf8_lossy(&head.stdout).trim(),
        )
        .unwrap();
        repo
    }

    /// The failure this exists for: an agent that stops having written nothing.
    /// Post-hoc that costs a whole new VM; here it costs one sentence.
    #[test]
    fn an_agent_that_changed_nothing_is_not_allowed_to_stop() {
        let tmp = tempfile::tempdir().unwrap();
        let repo = repo_with_base(tmp.path());
        let gate = StopGate {
            require_changes: true,
            check: None,
        };
        let script = gate.script(&repo.display().to_string(), &tmp.path().display().to_string());

        let out = sh(&script, tmp.path());
        assert_eq!(out.status.code(), Some(2), "the stop must be refused");
        let why = String::from_utf8_lossy(&out.stderr);
        assert!(why.contains("changed nothing"), "{why}");

        // Uncommitted work counts — the usual case, since the agent is told to
        // leave its work in the tree rather than commit it.
        std::fs::write(repo.join("new.rs"), "fn done() {}\n").unwrap();
        let out = sh(&script, tmp.path());
        assert_eq!(out.status.code(), Some(0), "{:?}", out);
    }

    /// The gate gives up after [`MAX_BLOCKS`] and lets the agent stop — and it
    /// must LEAVE A MARK when it does. Nothing outside this hook ever runs a
    /// `done_when_check`, so a silent release completed the phase green with its
    /// condition still failing.
    ///
    /// The two files say different things and both are needed: `blocks` reaches
    /// 3 in this test AND in a run where the agent got it right on the fourth
    /// try, so the count alone cannot tell success from surrender.
    #[test]
    fn a_gate_that_gives_up_records_that_it_gave_up() {
        let tmp = tempfile::tempdir().unwrap();
        let dir = tmp.path().display().to_string();
        let repo = repo_with_base(tmp.path());
        let gate = StopGate {
            require_changes: false,
            check: Some("exit 1".into()),
        };
        let script = gate.script(&repo.display().to_string(), &dir);

        for n in 1..=MAX_BLOCKS {
            let out = sh(&script, tmp.path());
            assert_eq!(out.status.code(), Some(2), "block {n} must refuse the stop");
            assert!(
                !tmp.path().join(CAPPED_FILE).exists(),
                "the cap mark must not appear while the gate is still blocking"
            );
        }

        // One more stop: the gate is out of blocks and must let the agent go.
        let out = sh(&script, tmp.path());
        assert_eq!(out.status.code(), Some(0), "at the cap the stop is allowed");
        assert_eq!(
            std::fs::read_to_string(tmp.path().join(CAPPED_FILE))
                .unwrap()
                .trim(),
            "1",
            "the release must be recorded, or nothing downstream can see it"
        );
    }

    /// The negative control for the mark: a gate whose check PASSES releases the
    /// agent too, and that release must not be recorded as a surrender. Without
    /// this, "always write the file" would pass the test above and fail every
    /// healthy phase in production.
    #[test]
    fn a_gate_that_is_satisfied_leaves_no_cap_mark() {
        let tmp = tempfile::tempdir().unwrap();
        let repo = repo_with_base(tmp.path());
        let gate = StopGate {
            require_changes: false,
            check: Some("true".into()),
        };
        let script = gate.script(&repo.display().to_string(), &tmp.path().display().to_string());

        let out = sh(&script, tmp.path());
        assert_eq!(out.status.code(), Some(0));
        assert!(
            !tmp.path().join(CAPPED_FILE).exists(),
            "a satisfied gate must not look like one that gave up"
        );
    }

    /// And committed work counts too. An agent that committed leaves a CLEAN
    /// tree, so a gate that only looked at `git status` would refuse the stop of
    /// a phase that had done everything asked of it.
    #[test]
    fn work_the_agent_committed_satisfies_the_gate() {
        let tmp = tempfile::tempdir().unwrap();
        let repo = repo_with_base(tmp.path());
        std::fs::write(repo.join("new.rs"), "fn done() {}\n").unwrap();
        for args in [vec!["add", "."], vec!["commit", "--quiet", "-m", "work"]] {
            Command::new("git")
                .arg("-C")
                .arg(&repo)
                .args(&args)
                .output()
                .unwrap();
        }
        let gate = StopGate {
            require_changes: true,
            check: None,
        };
        let out = sh(
            &gate.script(&repo.display().to_string(), &tmp.path().display().to_string()),
            tmp.path(),
        );
        assert_eq!(out.status.code(), Some(0), "{:?}", out);
    }

    /// The cap. Without it a stuck agent is blocked, retries, is blocked again,
    /// and spends the whole hour-long turn budget instead of failing where an
    /// operator can see it.
    #[test]
    fn the_gate_gives_up_after_the_cap_and_says_so() {
        let tmp = tempfile::tempdir().unwrap();
        let repo = repo_with_base(tmp.path());
        let gate = StopGate {
            require_changes: true,
            check: None,
        };
        let script = gate.script(&repo.display().to_string(), &tmp.path().display().to_string());

        for i in 1..=MAX_BLOCKS {
            assert_eq!(
                sh(&script, tmp.path()).status.code(),
                Some(2),
                "block {i} of {MAX_BLOCKS}"
            );
        }
        assert_eq!(
            sh(&script, tmp.path()).status.code(),
            Some(0),
            "past the cap the agent must be allowed to stop"
        );
        let log = std::fs::read_to_string(tmp.path().join("log")).unwrap();
        assert!(log.contains("cap:"), "giving up is recorded: {log}");
        assert_eq!(
            std::fs::read_to_string(tmp.path().join("blocks"))
                .unwrap()
                .trim(),
            MAX_BLOCKS.to_string(),
            "and the count is exact, so the host can report it"
        );
    }

    /// A phase-declared check runs in the repo, and its OUTPUT comes back — a
    /// gate that said only "the check failed" would send the agent guessing.
    #[test]
    fn a_declared_check_must_pass_and_its_output_is_the_feedback() {
        let tmp = tempfile::tempdir().unwrap();
        let repo = repo_with_base(tmp.path());
        let gate = StopGate {
            require_changes: false,
            check: Some("test -f wanted.txt || { echo 'wanted.txt is missing'; exit 3; }".into()),
        };
        let script = gate.script(&repo.display().to_string(), &tmp.path().display().to_string());

        let out = sh(&script, tmp.path());
        assert_eq!(out.status.code(), Some(2));
        let why = String::from_utf8_lossy(&out.stderr);
        assert!(why.contains("exited 3"), "{why}");
        assert!(why.contains("wanted.txt is missing"), "{why}");

        std::fs::write(repo.join("wanted.txt"), "here\n").unwrap();
        assert_eq!(sh(&script, tmp.path()).status.code(), Some(0));
    }

    /// A check with quotes, `$` and apostrophes is ordinary. It travels through
    /// `sh -c` inside a script that itself travels through `sh -c` to reach the
    /// guest, and a quoting bug at either layer would run something else.
    #[test]
    fn a_check_with_shell_metacharacters_survives_both_layers() {
        let tmp = tempfile::tempdir().unwrap();
        let repo = repo_with_base(tmp.path());
        std::fs::write(repo.join("it's here.txt"), "x\n").unwrap();
        let gate = StopGate {
            require_changes: false,
            check: Some("test -f \"it's here.txt\" && echo $HOME > /dev/null".into()),
        };
        let out = sh(
            &gate.script(&repo.display().to_string(), &tmp.path().display().to_string()),
            tmp.path(),
        );
        assert_eq!(out.status.code(), Some(0), "{:?}", out);
        // And the install command it is embedded in is still one shell argument.
        let install = gate.install_command("/mission/repo", GATE_DIR);
        assert!(install.contains("stop-gate.sh"), "{install}");
    }

    /// Nothing the gate writes may land under the repository: `/mission/repo` is
    /// collected and diffed, so a file there arrives in the user's patch as if
    /// an agent had written it.
    #[test]
    fn the_gate_never_writes_into_the_delivered_tree() {
        let gate = StopGate {
            require_changes: true,
            check: Some("cargo test".into()),
        };
        assert!(GATE_DIR.starts_with("/root/"), "{GATE_DIR}");
        let install = gate.install_command("/mission/repo", GATE_DIR);
        for write in ["> /mission/repo", "/mission/repo/stop", "/mission/repo/.claude"] {
            assert!(!install.contains(write), "{install}");
        }
        assert_eq!(
            crate::vm_tool_tap::guest_settings(Some(GATE_DIR), None, None)["hooks"]["Stop"][0]["hooks"]
                [0]["command"],
            json!("/root/gate/stop-gate.sh")
        );
    }

    /// A composed run's nodes must not each be held to "this phase changed
    /// something". The graph's verifier node changes nothing BY DESIGN, and a
    /// per-node gate would refuse its stop until the cap — three wasted agent
    /// turns for doing its job correctly.
    #[test]
    fn a_composed_node_is_not_held_to_the_whole_phases_delivery() {
        let phase = StopGate::for_phase("coding", &json!({})).unwrap();
        assert!(phase.require_changes);
        assert!(
            phase.per_node().is_none(),
            "with nothing but the delivery rule, a node has no gate at all"
        );

        let with_check =
            StopGate::for_phase("coding", &json!({ "done_when_check": "cargo test" })).unwrap();
        let node = with_check.per_node().expect("the declared check still applies");
        assert!(!node.require_changes);
        assert_eq!(node.check.as_deref(), Some("cargo test"));
    }

    /// A gate with nothing to enforce must not be installed at all — a hook that
    /// can never block still costs a process per stop and a flag on the command
    /// line.
    #[test]
    fn a_phase_with_nothing_to_enforce_gets_no_gate() {
        let none = json!({});
        assert!(StopGate::for_phase("research", &none).is_none());
        assert!(StopGate::for_phase("coding", &json!({ "allow_empty": true })).is_none());

        let coding = StopGate::for_phase("coding", &none).expect("a coding phase must deliver");
        assert!(coding.require_changes);
        assert!(coding.check.is_none());

        // A declared check applies to any kind, including one that is allowed to
        // change nothing — a verification phase's whole job is that check.
        let verify = StopGate::for_phase(
            "research",
            &json!({ "allow_empty": true, "done_when_check": " ./verify.sh " }),
        )
        .expect("a declared check is a gate on its own");
        assert!(!verify.require_changes);
        assert_eq!(verify.check.as_deref(), Some("./verify.sh"));
    }
}