slice 8: security scan runner + trigger endpoint
ci / gates (push) Successful in 3s
ci / frontend (push) Successful in 27s
ci / rust (push) Successful in 4m28s
ci / e2e (push) Skipped
ci / publish (push) Successful in 2m47s

Runs the security template's tool set (cargo-audit / gitleaks /
trivy fs / semgrep) inside the mission's team container and
materializes each finding as a mission_task keyed on the tool's
canonical id. The subsequent coding phase picks up the tasks and
applies remediations; the committer closes them by emitting
`COMPLETED: <external_id>` (Slice 5's task-card parser handles it).

Rust surface:
  - cm_api::security_scan::run(mission_id, phase_id)
  - Per-tool runners with JSON output parsing:
      cargo audit --json          → vulnerabilities[].advisory.id
      gitleaks detect --report-format=json → [{ Fingerprint, RuleID }]
      trivy fs --format=json      → Results[].Vulnerabilities[].VulnerabilityID
      semgrep --config=auto --json → results[] w/ rule+path+line fingerprint
  - Tool errors surface as a `warning` task instead of failing the
    scan — operator sees which need installing/fixing without a
    silent no-op.
  - Findings map to mission_tasks with external_id = "<tool>:<id>"
    (e.g. cargo_audit:RUSTSEC-2024-0001, gitleaks:<sha>,
    trivy_fs:CVE-2024-1234, semgrep:<rule>@<file>:<line>).

API:
  - POST /api/missions/{id}/security-scan { phase_id }
    → { findings, tasks[] } — full task list after upsert so the
    canvas can render immediately.

Frontend:
  - triggerSecurityScan helper in lib/api/missions.ts. Findings
    show up in the existing Tasks tab (Slice 5's UPSERT path).

Container requirements (opt-in):
  - Runs inside the team container via `docker exec`, so the tools
    must be present in that image. Missing = warning task, not fail.
  - `-w /workspace/repo` so scanners see the mounted repo. Reads
    teams.zeroclaw_container (populated on first phase run).

Follow-ups:
  - MCP bundle wrapping the same tools as agent-callable functions
    (currently agents scan by shelling out to `cargo audit` etc.
    directly; a typed MCP wrap lands with clean audit trail).
  - Auto-fire from the security_hardening workflow template on
    phase transition (currently manual via API trigger).
  - Bundle the four tools into the runtime image (or a dedicated
    security-tools image) so operators don't have to install them.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
This commit is contained in:
Omar Sobh
2026-07-19 16:19:29 -07:00
co-authored by Claude Opus 4.7
parent f843c9ddb1
commit 58963d5083
4 changed files with 391 additions and 0 deletions
+5
View File
@@ -17,6 +17,7 @@ mod recursive_exec;
pub mod research_container; pub mod research_container;
mod routes; mod routes;
mod runtime_provision; mod runtime_provision;
pub mod security_scan;
pub mod skills_loader; pub mod skills_loader;
pub mod swarm; pub mod swarm;
pub mod task_card_parser; pub mod task_card_parser;
@@ -441,6 +442,10 @@ pub fn router(state: AppState) -> Router {
"/api/missions/{id}/benchmark", "/api/missions/{id}/benchmark",
post(routes::missions::trigger_benchmark), post(routes::missions::trigger_benchmark),
) )
.route(
"/api/missions/{id}/security-scan",
post(routes::missions::trigger_security_scan),
)
.route( .route(
"/api/research", "/api/research",
get(routes::research::list_topics).post(routes::research::create_topic), get(routes::research::list_topics).post(routes::research::create_topic),
+34
View File
@@ -85,6 +85,17 @@ pub struct BenchmarkTriggerRequest {
pub iteration: Option<i32>, pub iteration: Option<i32>,
} }
#[derive(Debug, Deserialize)]
pub struct SecurityScanRequest {
pub phase_id: Uuid,
}
#[derive(Debug, Serialize)]
pub struct SecurityScanResponse {
pub findings: usize,
pub tasks: Vec<MissionTask>,
}
// ── Handlers ───────────────────────────────────────────────────── // ── Handlers ─────────────────────────────────────────────────────
pub async fn list( pub async fn list(
@@ -187,6 +198,29 @@ pub async fn trigger_benchmark(
Ok(Json(snaps)) Ok(Json(snaps))
} }
/// POST /api/missions/{id}/security-scan — run the security phase's
/// tool set (cargo-audit / gitleaks / trivy fs / semgrep) inside
/// the mission's team container and materialize each finding as a
/// mission_task keyed on the tool's canonical id.
pub async fn trigger_security_scan(
State(state): State<AppState>,
Authed(user): Authed,
Path(id): Path<Uuid>,
Json(body): Json<SecurityScanRequest>,
) -> Result<Json<SecurityScanResponse>, ApiError> {
let _ = cm_db::repo::missions::get(&state.pool, id, user.workspace_id.as_uuid())
.await?
.ok_or(ApiError::NotFound)?;
let findings = crate::security_scan::run(&state.pool, id, body.phase_id)
.await
.map_err(|e| {
eprintln!("security_scan for mission {id}: {e}");
ApiError::Internal
})?;
let tasks = cm_db::repo::missions::tasks_for(&state.pool, id).await?;
Ok(Json(SecurityScanResponse { findings, tasks }))
}
pub async fn set_status( pub async fn set_status(
State(state): State<AppState>, State(state): State<AppState>,
Authed(user): Authed, Authed(user): Authed,
+341
View File
@@ -0,0 +1,341 @@
//! Security scan phase executor — Slice 8.
//!
//! Runs the security template's tool set (cargo-audit, gitleaks,
//! trivy fs, semgrep) inside the mission's team container and
//! materializes each finding as a `mission_tasks` row keyed on the
//! finding's canonical id (RUSTSEC-YYYY-NNNN, gitleaks fingerprint,
//! CVE id, semgrep rule id). The subsequent coding phase picks up
//! the tasks and applies remediations.
//!
//! Tools not present in the container are skipped with a `warning`
//! task so the operator sees which need installing rather than a
//! silent no-op.
//!
//! Each tool ships a JSON output mode we can parse without regex:
//! cargo audit --json — { vulnerabilities: [{ advisory, package, versions }] }
//! gitleaks detect --report-format=json - [{ RuleID, Fingerprint, File, StartLine, ... }]
//! trivy fs --format=json — { Results: [{ Vulnerabilities: [{ VulnerabilityID, ... }] }] }
//! semgrep --json — { results: [{ check_id, path, start, extra: { severity } }] }
//!
//! Findings map to mission_tasks with:
//! external_id = <tool>:<canonical_id> e.g. cargo_audit:RUSTSEC-2024-0001
//! title = <one-line summary>
//! status = 'created'
//! The coding phase's committer emits COMPLETED: <external_id> to
//! close them via the task-card parser (Slice 5).
use serde_json::{json, Value};
use sqlx::PgPool;
use sqlx::Row;
use std::path::PathBuf;
use uuid::Uuid;
use cm_db::repo::missions::UpsertTask;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Finding {
pub external_id: String,
pub title: String,
pub tool: &'static str,
}
/// Run every configured tool + upsert findings. Returns the count
/// of findings inserted/updated. `phase.config.tools` gates which
/// scanners run; missing = run the default set (all four).
pub async fn run(pool: &PgPool, mission_id: Uuid, phase_id: Uuid) -> Result<usize, String> {
let cfg = load_phase_config(pool, phase_id).await?;
let tools: Vec<String> = cfg
.get("tools")
.and_then(|v| v.as_array())
.map(|a| {
a.iter()
.filter_map(|v| v.as_str().map(|s| s.to_string()))
.collect()
})
.unwrap_or_else(|| {
vec![
"cargo_audit".into(),
"gitleaks".into(),
"trivy_fs".into(),
"semgrep".into(),
]
});
let container = team_container_for_mission(pool, mission_id).await?;
let mut all_findings: Vec<Finding> = Vec::new();
for tool in &tools {
let findings = match tool.as_str() {
"cargo_audit" => run_cargo_audit(&container).await,
"gitleaks" => run_gitleaks(&container).await,
"trivy_fs" => run_trivy_fs(&container).await,
"semgrep" => run_semgrep(&container).await,
other => {
eprintln!("security_scan: unknown tool `{other}` — skipped");
Ok(Vec::new())
}
};
match findings {
Ok(f) => all_findings.extend(f),
Err(e) => {
// Surface tool failures as a `warning` task so the operator
// knows something needs installing/fixing, but keep going.
all_findings.push(Finding {
external_id: format!("{tool}:tool_error"),
title: format!("{tool} scan failed: {e}"),
tool: static_tool_name(tool),
});
}
}
}
for f in &all_findings {
cm_db::repo::missions::upsert_task(
pool,
UpsertTask {
mission_id,
phase_id,
external_id: &f.external_id,
title: &f.title,
assigned_agent_id: None,
status: "created",
run_id: None,
},
)
.await
.map_err(|e| format!("upsert_task {}: {e}", f.external_id))?;
}
Ok(all_findings.len())
}
// ── Per-tool runners ────────────────────────────────────────────
async fn run_cargo_audit(container: &str) -> Result<Vec<Finding>, String> {
let out = docker_exec_json(
container,
&[
"sh".into(),
"-c".into(),
"cargo audit --json 2>/dev/null || true".into(),
],
)
.await?;
let mut findings = Vec::new();
let vulns = out
.pointer("/vulnerabilities/list")
.or_else(|| out.get("vulnerabilities"))
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
for v in vulns {
let id = v
.pointer("/advisory/id")
.and_then(|x| x.as_str())
.unwrap_or("RUSTSEC-UNKNOWN")
.to_string();
let pkg = v
.pointer("/package/name")
.and_then(|x| x.as_str())
.unwrap_or("<unknown crate>");
let title = v
.pointer("/advisory/title")
.and_then(|x| x.as_str())
.unwrap_or("(no title)");
findings.push(Finding {
external_id: format!("cargo_audit:{id}"),
title: format!("{pkg}: {title}"),
tool: "cargo_audit",
});
}
Ok(findings)
}
async fn run_gitleaks(container: &str) -> Result<Vec<Finding>, String> {
let out = docker_exec_raw(
container,
&[
"sh".into(),
"-c".into(),
// gitleaks exits 1 on findings, 0 on none; `|| true` lets us always parse.
"gitleaks detect --report-format=json --report-path=/dev/stdout --no-banner 2>/dev/null || true".into(),
],
)
.await?;
// gitleaks emits a JSON array (or "[]").
let arr: Vec<Value> = serde_json::from_str(out.trim()).unwrap_or_default();
let mut findings = Vec::new();
for f in arr {
let fingerprint = f
.get("Fingerprint")
.and_then(|v| v.as_str())
.unwrap_or("unknown");
let rule = f
.get("RuleID")
.and_then(|v| v.as_str())
.unwrap_or("unknown_rule");
let file = f.get("File").and_then(|v| v.as_str()).unwrap_or("?");
let line = f.get("StartLine").and_then(|v| v.as_i64()).unwrap_or(0);
findings.push(Finding {
external_id: format!("gitleaks:{fingerprint}"),
title: format!("{rule} at {file}:{line}"),
tool: "gitleaks",
});
}
Ok(findings)
}
async fn run_trivy_fs(container: &str) -> Result<Vec<Finding>, String> {
let out = docker_exec_json(
container,
&[
"sh".into(),
"-c".into(),
"trivy fs --quiet --format=json --scanners=vuln . 2>/dev/null || true".into(),
],
)
.await?;
let mut findings = Vec::new();
let results = out
.get("Results")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
for r in results {
let vulns = r
.get("Vulnerabilities")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
for v in vulns {
let cve = v
.get("VulnerabilityID")
.and_then(|x| x.as_str())
.unwrap_or("CVE-UNKNOWN");
let pkg = v.get("PkgName").and_then(|x| x.as_str()).unwrap_or("?");
let severity = v
.get("Severity")
.and_then(|x| x.as_str())
.unwrap_or("UNKNOWN");
findings.push(Finding {
external_id: format!("trivy_fs:{cve}"),
title: format!("[{severity}] {pkg} — {cve}"),
tool: "trivy_fs",
});
}
}
Ok(findings)
}
async fn run_semgrep(container: &str) -> Result<Vec<Finding>, String> {
let out = docker_exec_json(
container,
&[
"sh".into(),
"-c".into(),
"semgrep --config=auto --json --quiet . 2>/dev/null || true".into(),
],
)
.await?;
let mut findings = Vec::new();
let results = out
.get("results")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
for r in results {
let rule = r
.get("check_id")
.and_then(|v| v.as_str())
.unwrap_or("semgrep_unknown");
let path = r.get("path").and_then(|v| v.as_str()).unwrap_or("?");
let line = r
.pointer("/start/line")
.and_then(|v| v.as_i64())
.unwrap_or(0);
let severity = r
.pointer("/extra/severity")
.and_then(|v| v.as_str())
.unwrap_or("INFO");
// Fingerprint = rule + file + line so re-scans dedup cleanly.
let fingerprint = format!("{rule}@{path}:{line}");
findings.push(Finding {
external_id: format!("semgrep:{fingerprint}"),
title: format!("[{severity}] {rule} at {path}:{line}"),
tool: "semgrep",
});
}
Ok(findings)
}
// ── Container plumbing ──────────────────────────────────────────
async fn load_phase_config(pool: &PgPool, phase_id: Uuid) -> Result<Value, String> {
let row = sqlx::query("SELECT config FROM mission_phases WHERE id = $1")
.bind(phase_id)
.fetch_optional(pool)
.await
.map_err(|e| format!("load phase: {e}"))?;
Ok(row
.map(|r| r.get::<Value, _>("config"))
.unwrap_or_else(|| json!({})))
}
async fn team_container_for_mission(pool: &PgPool, mission_id: Uuid) -> Result<String, String> {
let row = sqlx::query(
"SELECT t.zeroclaw_container
FROM missions m
JOIN teams t ON t.id = m.team_id
WHERE m.id = $1",
)
.bind(mission_id)
.fetch_optional(pool)
.await
.map_err(|e| format!("resolve container: {e}"))?;
row.and_then(|r| {
r.try_get::<Option<String>, _>("zeroclaw_container")
.ok()
.flatten()
})
.ok_or_else(|| "mission has no team container yet".to_string())
}
async fn docker_exec_raw(container: &str, cmd: &[String]) -> Result<String, String> {
let mut args = vec![
"exec".to_string(),
"-w".into(),
"/workspace/repo".into(),
container.to_string(),
];
args.extend(cmd.iter().cloned());
let out = tokio::process::Command::new("docker")
.args(&args)
.output()
.await
.map_err(|e| format!("spawn docker: {e}"))?;
Ok(String::from_utf8_lossy(&out.stdout).into_owned())
}
async fn docker_exec_json(container: &str, cmd: &[String]) -> Result<Value, String> {
let raw = docker_exec_raw(container, cmd).await?;
let trimmed = raw.trim();
if trimmed.is_empty() {
return Ok(json!({}));
}
serde_json::from_str(trimmed).map_err(|e| format!("parse tool json: {e}"))
}
fn static_tool_name(s: &str) -> &'static str {
match s {
"cargo_audit" => "cargo_audit",
"gitleaks" => "gitleaks",
"trivy_fs" => "trivy_fs",
"semgrep" => "semgrep",
_ => "unknown",
}
}
// Unused import silence + shape hint for a future artifact-write
// path that dumps the raw JSON outputs into mission_artifacts/security/.
#[allow(dead_code)]
fn future_artifact_root(mission_id: Uuid) -> PathBuf {
PathBuf::from(format!("/var/lib/clawmates-missions/{mission_id}/security"))
}
+11
View File
@@ -138,6 +138,17 @@ export const triggerBenchmark = (
body: JSON.stringify({ phase_id, slot, iteration }), body: JSON.stringify({ phase_id, slot, iteration }),
}); });
export interface SecurityScanResponse {
findings: number;
tasks: MissionTask[];
}
export const triggerSecurityScan = (missionId: string, phase_id: string) =>
api<SecurityScanResponse>(`/api/missions/${missionId}/security-scan`, {
method: "POST",
body: JSON.stringify({ phase_id }),
});
export interface PhaseSpec { export interface PhaseSpec {
kind: PhaseKind; kind: PhaseKind;
order_idx: number; order_idx: number;