fix: close the three remaining gaps, and repair a test I silently disabled

FIRST, the self-inflicted one. My edit in a20702d inserted a test between an
existing `#[test]` and the function it belonged to. The result compiled and
looked fine: `every_anthropic_spelling_is_one_family` lost its attribute and
STOPPED BEING A TEST, its doc comment ended up describing my test instead, and
my test carried two `#[test]`s. It has not run since — in already-deployed code.
Nothing failed, which is the point: a test that does not run is indistinguishable
from one that passes. Found via a compiler warning I had not read.

The commit message on a20702d said "241 lib tests pass". 240 ran.

Then the three gaps.

1. A security scan could not read history. `ensure_checkout` clones with
   `--filter=blob:none` — full commits, blobs on demand — and the agent
   environment has NO network route to the forge. Measured: gitleaks on a
   4-commit repo reported "1 commits scanned" and "could not fetch <sha> from
   promisor remote". A credential committed and later deleted is exactly what a
   scanner looks for and exactly what a lazy blob withholds. Missions with a
   `security_scan` phase now clone fully; everything else keeps the cheap path.

   (I first blamed `--depth 1`, from a stale module doc comment. The code has
   said `--filter=blob:none` since it was written, and the comment at `clone`
   explains why NOT shallow — a shallow clone cannot push a branch back. Both
   the comment and my claim are fixed.)

2. `benchmark_runner` never ran as part of a benchmark phase. It was reachable
   only from an operator button, so the `author_and_baseline` recipe authored
   benchmarks and measured nothing — `benchmark_snapshots` stayed empty. Now
   baselined from the sweep, SPAWNED not awaited: BENCH_TIMEOUT is 30 minutes
   and that loop also starts, closes, evaluates and captures every phase on the
   platform. A `NOT EXISTS` guard on iteration 0 makes per-tick firing safe. A
   repo with no bench harness logs and does NOT fail the phase — but it logs,
   because "no baseline" must not read like "not attempted".

3. The World's rich layer was empty for every mission. `run_events::append` is
   called only from the a2a path, and `world.rs` tailed only that table —
   while mission per-step detail has always lived in
   `topology_runs.checkpoint.records`, which `topology::run_events_sse` streams.
   The data was never missing; the viz read the one source missions never write.
   Now both are tailed, mapped through the existing `step_started` vocabulary so
   no new event types are needed.

242 lib tests, 20 test binaries, zero warnings.
This commit is contained in:
Omar Sobh
2026-08-07 16:09:49 -07:00
parent 2a9a62c784
commit 0d8db7ff0b
4 changed files with 189 additions and 13 deletions
+73 -10
View File
@@ -11,7 +11,10 @@
//! - no repo_id → no-op (Ok(None))
//! - dir already a git repo → `fetch + reset --hard origin/<branch>`
//! to bring it in sync
//! - dir missing → `git clone --depth 1 <url> <path>`
//! - dir missing → `git clone --filter=blob:none --single-branch` (NOT
//! `--depth 1`: a shallow clone cannot push a new branch back, and delivery
//! needs exactly that — see `clone`). A mission with a `security_scan` phase
//! gets a FULLY HYDRATED clone instead; see `wants_full_history`.
//!
//! Auth: for `git.redclaw.dev` clones we inject the ambient
//! `GITEA_TOKEN` (already provisioned in the server container's env)
@@ -99,7 +102,7 @@ pub async fn ensure_checkout(
fetch_and_reset(&path, default_branch, &auth_url).await?;
}
} else {
clone(&path, &auth_url).await?;
clone(&path, &auth_url, wants_full_history(pool, mission_id).await).await?;
}
Ok(Some(path))
}
@@ -220,21 +223,59 @@ pub(crate) fn no_terminal_prompt(cmd: &mut Command) -> &mut Command {
cmd.env("GIT_TERMINAL_PROMPT", "0")
}
async fn clone(path: &std::path::Path, url: &str) -> Result<(), String> {
/// Does any phase of this mission need history it can READ, not just reference?
///
/// `--filter=blob:none` keeps every commit but fetches file contents on demand,
/// which is nearly free for a repo that gets read once — and silently useless to
/// a tool that walks history, because the agent environment has NO network route
/// to the forge. Measured: gitleaks on a 4-commit repo reported
/// "1 commits scanned" and "could not fetch <sha> from promisor remote". It was
/// not misconfigured; the blobs simply were not there and could not be got.
///
/// A security scan is the phase kind whose entire value is old content — a
/// credential committed and later deleted is exactly what it looks for, and that
/// is precisely what a lazy blob is. So those missions pay for a full clone and
/// everything else keeps the cheap one.
///
/// Best-effort: an unreadable phase list yields `false`, i.e. today's behaviour.
async fn wants_full_history(pool: &sqlx::PgPool, mission_id: Uuid) -> bool {
sqlx::query_scalar::<_, i64>(
"SELECT count(*) FROM mission_phases WHERE mission_id = $1 AND kind = 'security_scan'",
)
.bind(mission_id)
.fetch_one(pool)
.await
.map(|n| n > 0)
.unwrap_or(false)
}
/// The `git clone` flags, split out so the strategy is testable without a forge.
fn clone_args(full_history: bool) -> Vec<&'static str> {
let mut a = vec!["clone"];
if !full_history {
a.push("--filter=blob:none");
}
a.push("--single-branch");
a
}
async fn clone(path: &std::path::Path, url: &str, full_history: bool) -> Result<(), String> {
// `--filter=blob:none` rather than `--depth 1`. A shallow clone cannot
// usually push a new branch back ("shallow update not allowed"), and
// mission delivery needs exactly that. A partial clone keeps full history
// — so the base commit stays meaningful and a diff has something to be
// relative to — while fetching file contents only on demand, which is
// nearly as cheap as a shallow clone for a repo that gets read once.
if full_history {
eprintln!(
"mission_workspace: cloning {} with full history — a security_scan phase \
reads old file contents, which a partial clone cannot supply offline",
path.display()
);
}
let mut cmd = Command::new("git");
cmd.args([
"clone",
"--filter=blob:none",
"--single-branch",
url,
&path.display().to_string(),
]);
cmd.args(clone_args(full_history));
cmd.args([url, &path.display().to_string()]);
let out = no_terminal_prompt(&mut cmd)
.output()
.await
@@ -718,6 +759,28 @@ async fn fetch_and_reset(
#[cfg(test)]
mod tests {
/// The clone strategy is a fact worth pinning: `--depth 1` breaks delivery
/// (a shallow clone cannot push a new branch — "shallow update not
/// allowed"), and `--filter=blob:none` breaks history-reading tools offline.
/// Both failure modes are real and were both hit.
#[test]
fn the_clone_strategy_is_partial_by_default_and_never_shallow() {
let partial = clone_args(false);
assert!(partial.contains(&"--filter=blob:none"), "{partial:?}");
assert!(!partial.iter().any(|a| a.starts_with("--depth")), "{partial:?}");
// A security_scan mission must NOT get the lazy-blob filter: its scanner
// walks old file contents and cannot reach the forge to fetch them.
let full = clone_args(true);
assert!(!full.contains(&"--filter=blob:none"), "{full:?}");
assert!(!full.iter().any(|a| a.starts_with("--depth")), "{full:?}");
// Both keep --single-branch: the mission only ever works one branch.
for args in [partial, full] {
assert!(args.contains(&"--single-branch"), "{args:?}");
}
}
use super::*;
const TOK: Option<&str> = Some("secret123");