fix: close the three remaining gaps, and repair a test I silently disabled
FIRST, the self-inflicted one. My edit ina20702dinserted a test between an existing `#[test]` and the function it belonged to. The result compiled and looked fine: `every_anthropic_spelling_is_one_family` lost its attribute and STOPPED BEING A TEST, its doc comment ended up describing my test instead, and my test carried two `#[test]`s. It has not run since — in already-deployed code. Nothing failed, which is the point: a test that does not run is indistinguishable from one that passes. Found via a compiler warning I had not read. The commit message ona20702dsaid "241 lib tests pass". 240 ran. Then the three gaps. 1. A security scan could not read history. `ensure_checkout` clones with `--filter=blob:none` — full commits, blobs on demand — and the agent environment has NO network route to the forge. Measured: gitleaks on a 4-commit repo reported "1 commits scanned" and "could not fetch <sha> from promisor remote". A credential committed and later deleted is exactly what a scanner looks for and exactly what a lazy blob withholds. Missions with a `security_scan` phase now clone fully; everything else keeps the cheap path. (I first blamed `--depth 1`, from a stale module doc comment. The code has said `--filter=blob:none` since it was written, and the comment at `clone` explains why NOT shallow — a shallow clone cannot push a branch back. Both the comment and my claim are fixed.) 2. `benchmark_runner` never ran as part of a benchmark phase. It was reachable only from an operator button, so the `author_and_baseline` recipe authored benchmarks and measured nothing — `benchmark_snapshots` stayed empty. Now baselined from the sweep, SPAWNED not awaited: BENCH_TIMEOUT is 30 minutes and that loop also starts, closes, evaluates and captures every phase on the platform. A `NOT EXISTS` guard on iteration 0 makes per-tick firing safe. A repo with no bench harness logs and does NOT fail the phase — but it logs, because "no baseline" must not read like "not attempted". 3. The World's rich layer was empty for every mission. `run_events::append` is called only from the a2a path, and `world.rs` tailed only that table — while mission per-step detail has always lived in `topology_runs.checkpoint.records`, which `topology::run_events_sse` streams. The data was never missing; the viz read the one source missions never write. Now both are tailed, mapped through the existing `step_started` vocabulary so no new event types are needed. 242 lib tests, 20 test binaries, zero warnings.
This commit is contained in:
@@ -11,7 +11,10 @@
|
||||
//! - no repo_id → no-op (Ok(None))
|
||||
//! - dir already a git repo → `fetch + reset --hard origin/<branch>`
|
||||
//! to bring it in sync
|
||||
//! - dir missing → `git clone --depth 1 <url> <path>`
|
||||
//! - dir missing → `git clone --filter=blob:none --single-branch` (NOT
|
||||
//! `--depth 1`: a shallow clone cannot push a new branch back, and delivery
|
||||
//! needs exactly that — see `clone`). A mission with a `security_scan` phase
|
||||
//! gets a FULLY HYDRATED clone instead; see `wants_full_history`.
|
||||
//!
|
||||
//! Auth: for `git.redclaw.dev` clones we inject the ambient
|
||||
//! `GITEA_TOKEN` (already provisioned in the server container's env)
|
||||
@@ -99,7 +102,7 @@ pub async fn ensure_checkout(
|
||||
fetch_and_reset(&path, default_branch, &auth_url).await?;
|
||||
}
|
||||
} else {
|
||||
clone(&path, &auth_url).await?;
|
||||
clone(&path, &auth_url, wants_full_history(pool, mission_id).await).await?;
|
||||
}
|
||||
Ok(Some(path))
|
||||
}
|
||||
@@ -220,21 +223,59 @@ pub(crate) fn no_terminal_prompt(cmd: &mut Command) -> &mut Command {
|
||||
cmd.env("GIT_TERMINAL_PROMPT", "0")
|
||||
}
|
||||
|
||||
async fn clone(path: &std::path::Path, url: &str) -> Result<(), String> {
|
||||
/// Does any phase of this mission need history it can READ, not just reference?
|
||||
///
|
||||
/// `--filter=blob:none` keeps every commit but fetches file contents on demand,
|
||||
/// which is nearly free for a repo that gets read once — and silently useless to
|
||||
/// a tool that walks history, because the agent environment has NO network route
|
||||
/// to the forge. Measured: gitleaks on a 4-commit repo reported
|
||||
/// "1 commits scanned" and "could not fetch <sha> from promisor remote". It was
|
||||
/// not misconfigured; the blobs simply were not there and could not be got.
|
||||
///
|
||||
/// A security scan is the phase kind whose entire value is old content — a
|
||||
/// credential committed and later deleted is exactly what it looks for, and that
|
||||
/// is precisely what a lazy blob is. So those missions pay for a full clone and
|
||||
/// everything else keeps the cheap one.
|
||||
///
|
||||
/// Best-effort: an unreadable phase list yields `false`, i.e. today's behaviour.
|
||||
async fn wants_full_history(pool: &sqlx::PgPool, mission_id: Uuid) -> bool {
|
||||
sqlx::query_scalar::<_, i64>(
|
||||
"SELECT count(*) FROM mission_phases WHERE mission_id = $1 AND kind = 'security_scan'",
|
||||
)
|
||||
.bind(mission_id)
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.map(|n| n > 0)
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// The `git clone` flags, split out so the strategy is testable without a forge.
|
||||
fn clone_args(full_history: bool) -> Vec<&'static str> {
|
||||
let mut a = vec!["clone"];
|
||||
if !full_history {
|
||||
a.push("--filter=blob:none");
|
||||
}
|
||||
a.push("--single-branch");
|
||||
a
|
||||
}
|
||||
|
||||
async fn clone(path: &std::path::Path, url: &str, full_history: bool) -> Result<(), String> {
|
||||
// `--filter=blob:none` rather than `--depth 1`. A shallow clone cannot
|
||||
// usually push a new branch back ("shallow update not allowed"), and
|
||||
// mission delivery needs exactly that. A partial clone keeps full history
|
||||
// — so the base commit stays meaningful and a diff has something to be
|
||||
// relative to — while fetching file contents only on demand, which is
|
||||
// nearly as cheap as a shallow clone for a repo that gets read once.
|
||||
if full_history {
|
||||
eprintln!(
|
||||
"mission_workspace: cloning {} with full history — a security_scan phase \
|
||||
reads old file contents, which a partial clone cannot supply offline",
|
||||
path.display()
|
||||
);
|
||||
}
|
||||
let mut cmd = Command::new("git");
|
||||
cmd.args([
|
||||
"clone",
|
||||
"--filter=blob:none",
|
||||
"--single-branch",
|
||||
url,
|
||||
&path.display().to_string(),
|
||||
]);
|
||||
cmd.args(clone_args(full_history));
|
||||
cmd.args([url, &path.display().to_string()]);
|
||||
let out = no_terminal_prompt(&mut cmd)
|
||||
.output()
|
||||
.await
|
||||
@@ -718,6 +759,28 @@ async fn fetch_and_reset(
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
/// The clone strategy is a fact worth pinning: `--depth 1` breaks delivery
|
||||
/// (a shallow clone cannot push a new branch — "shallow update not
|
||||
/// allowed"), and `--filter=blob:none` breaks history-reading tools offline.
|
||||
/// Both failure modes are real and were both hit.
|
||||
#[test]
|
||||
fn the_clone_strategy_is_partial_by_default_and_never_shallow() {
|
||||
let partial = clone_args(false);
|
||||
assert!(partial.contains(&"--filter=blob:none"), "{partial:?}");
|
||||
assert!(!partial.iter().any(|a| a.starts_with("--depth")), "{partial:?}");
|
||||
|
||||
// A security_scan mission must NOT get the lazy-blob filter: its scanner
|
||||
// walks old file contents and cannot reach the forge to fetch them.
|
||||
let full = clone_args(true);
|
||||
assert!(!full.contains(&"--filter=blob:none"), "{full:?}");
|
||||
assert!(!full.iter().any(|a| a.starts_with("--depth")), "{full:?}");
|
||||
|
||||
// Both keep --single-branch: the mission only ever works one branch.
|
||||
for args in [partial, full] {
|
||||
assert!(args.contains(&"--single-branch"), "{args:?}");
|
||||
}
|
||||
}
|
||||
|
||||
use super::*;
|
||||
|
||||
const TOK: Option<&str> = Some("secret123");
|
||||
|
||||
Reference in New Issue
Block a user