slice 7: before/after benchmark runner
ci / gates (push) Successful in 4s
ci / frontend (push) Successful in 26s
ci / rust (push) Successful in 4m19s
ci / e2e (push) Skipped
ci / publish (push) Successful in 4m52s

Executes a benchmark harness inside the mission's team container
and records the resulting metrics as a benchmark_snapshots row keyed
on (phase_id, iteration). Baseline pass (iteration=0) captures
before_metrics; each post-iteration call captures after_metrics +
computes delta vs baseline.

Rust surface:
  - cm_db::repo::missions::upsert_benchmark_snapshot / benchmark_snapshots_for
  - cm_api::benchmark_runner::{baseline, after_iteration, run}
  - Harness enum: Auto | Criterion | CargoBench | VitestBench |
    PytestBench | Shell (each with a command() vector)
  - Auto detection peeks at the repo layout inside the container
    (Cargo.toml → CargoBench, package.json → VitestBench, pyproject
    → PytestBench). Falls back to a Shell echo when nothing
    identifiable.
  - Bencher-format line parser extracts (name, ns_per_iter,
    plusminus) so criterion + `cargo bench` output become structured
    samples the canvas can diff.
  - compute_delta pairs samples by name, emits {before_ns, after_ns,
    delta_pct, direction: improved|regressed}.

API:
  - POST /api/missions/{id}/benchmark { phase_id, slot, iteration? }
    triggers baseline or after run and returns the mission's full
    snapshot list.
  - GET /api/missions/{id} now includes `benchmarks[]` in the detail
    payload.

Frontend:
  - New Benchmarks tab on MissionCanvas with iteration + driver
    header, plus a 4-column grid (bench / before / after / Δ%) when
    delta samples are present. Improved deltas render green,
    regressions red.
  - TS types + triggerBenchmark() helper in lib/api/missions.ts.

Wiring notes:
  - team_container_for_mission reads teams.zeroclaw_container — that's
    populated by topology_worker::try_team_gateway_url on first run,
    so trigger baseline AFTER the mission's first phase spawns the
    container.
  - Not auto-fired yet by phase execution; that's the "template phase
    executor" work that spans Slices 4-8. Manual API trigger works
    today; automated hook is a follow-up.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
This commit is contained in:
Omar Sobh
2026-07-19 15:32:47 -07:00
co-authored by Claude Opus 4.7
parent 3ac3d53da7
commit f843c9ddb1
6 changed files with 705 additions and 5 deletions
+45 -1
View File
@@ -10,7 +10,8 @@ use axum::{
Json,
};
use cm_db::repo::missions::{
Mission, MissionArtifact, MissionPhase, MissionTask, NewMission, NewMissionPhase,
BenchmarkSnapshot, Mission, MissionArtifact, MissionPhase, MissionTask, NewMission,
NewMissionPhase,
};
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -71,6 +72,17 @@ pub struct MissionDetail {
pub phases: Vec<MissionPhase>,
pub tasks: Vec<MissionTask>,
pub artifacts: Vec<MissionArtifact>,
pub benchmarks: Vec<BenchmarkSnapshot>,
}
#[derive(Debug, Deserialize)]
pub struct BenchmarkTriggerRequest {
pub phase_id: Uuid,
/// Slot: "baseline" (records iteration 0) or "after"
/// (records iteration N + delta vs baseline).
pub slot: String,
#[serde(default)]
pub iteration: Option<i32>,
}
// ── Handlers ─────────────────────────────────────────────────────
@@ -135,14 +147,46 @@ pub async fn get(
let phases = cm_db::repo::missions::phases_for(&state.pool, id).await?;
let tasks = cm_db::repo::missions::tasks_for(&state.pool, id).await?;
let artifacts = cm_db::repo::missions::artifacts_for(&state.pool, id).await?;
let benchmarks = cm_db::repo::missions::benchmark_snapshots_for(&state.pool, id).await?;
Ok(Json(MissionDetail {
mission,
phases,
tasks,
artifacts,
benchmarks,
}))
}
/// POST /api/missions/{id}/benchmark — run the benchmark harness
/// against a phase. Slot='baseline' records iteration 0's
/// before_metrics; slot='after' with iteration=N records the
/// after_metrics + computes delta against baseline.
pub async fn trigger_benchmark(
State(state): State<AppState>,
Authed(user): Authed,
Path(id): Path<Uuid>,
Json(body): Json<BenchmarkTriggerRequest>,
) -> Result<Json<Vec<BenchmarkSnapshot>>, ApiError> {
// Workspace scope check on the mission — 404 if not visible.
let _ = cm_db::repo::missions::get(&state.pool, id, user.workspace_id.as_uuid())
.await?
.ok_or(ApiError::NotFound)?;
let result = match body.slot.as_str() {
"baseline" => crate::benchmark_runner::baseline(&state.pool, id, body.phase_id).await,
"after" => {
let iter = body.iteration.unwrap_or(1);
crate::benchmark_runner::after_iteration(&state.pool, id, body.phase_id, iter).await
}
_ => return Err(ApiError::BadRequest),
};
if let Err(e) = result {
eprintln!("benchmark trigger for mission {id}: {e}");
return Err(ApiError::Internal);
}
let snaps = cm_db::repo::missions::benchmark_snapshots_for(&state.pool, id).await?;
Ok(Json(snaps))
}
pub async fn set_status(
State(state): State<AppState>,
Authed(user): Authed,