Files
clawmates/crates/cm-api/tests/mission_orchestrator.rs
T
Omar SobhandClaude Opus 5 f7f3dfe495 feat(fleet): GLM as a real microVM backend, and per-role models for claws
Three threads, all of which end at the same place: a mission whose verifier does
not share a model with the coder it reviews.

**GLM has a credential contract now.** `microvm_credential_for` returned one env
var name, which quietly assumed every provider reads its secret from the same
place Anthropic does. It returns a `Credential { source, target }` instead —
z.ai's key lives in the server's `ZAI_API_KEY` and Claude Code reads it as
`ANTHROPIC_AUTH_TOKEN`, and collapsing those two names is what forces a guess at
the other end. A wrong guess here sends one provider's credential to another
provider's endpoint.

`images/agent-glm` is the same CLI at the same pinned version as `agent-claude`
with `ANTHROPIC_BASE_URL` baked in. The split is deliberate: the ENDPOINT is a
property of the image, the CREDENTIAL is a property of the turn. That makes the
dangerous mix-up unrepresentable — a GLM VM cannot be handed an Anthropic
subscription token, and a claude VM cannot be pointed at z.ai. Asserted both
ways, because "the GLM VM must not carry CLAUDE_CODE_OAUTH_TOKEN" is the
property that costs a credential if it ever stops holding.

Kimi stays refused. `KIMI_API_KEY` is set and Moonshot serves an
Anthropic-compatible API, but I have not verified its base URL against the
running service, and this function is precisely where guessing a URL is
expensive. It becomes an arm the day someone measures it.

`api.z.ai` joins the node's default egress allow-list. A default that cannot
run the images we ship is a trap rather than a policy — the alternative is an
operator discovering it as a hung agent with no model access.

**Per-role models for claws** (migration 0071). `template_roles` had no model
column, so `mint_team_from_template` bound every role of every mission team to
one literal — a template whose whole point is an independent reviewer minted a
reviewer sharing a model with the coder. A role may now name its own; roles that
say nothing still take the mint's default, so every template written before this
behaves exactly as it did. The literal is now that default rather than a
hardcode.

**A harness scenario for the roster flow.** `verify-mission-delivery.sh roster`
runs the whole Slice 5 loop — planner proposes, human approves, mission runs —
and asserts the roster LANDED on the mission row rather than trusting the API's
answer. That distinction is not theoretical: the first live approval returned an
error while leaving the proposal marked approved.

Built and proven on tank ahead of the deploy: `clawmates/agent-glm:dev` reports
`2.1.223` and `BASE=https://api.z.ai/api/anthropic`, and
`fc-build-rootfs.sh … glm 8G` boots a VM from it that has git, can write
/mission, and answers `claude --version`.

533 tests pass, clippy clean. Migration 0071.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-06 17:14:53 -07:00

377 lines
13 KiB
Rust

//! End-to-end coverage for `mission_orchestrator::on_launch` — the
//! draft→running transition that materializes a team from a template,
//! inserts agents, seeds brains, records template lineage, and binds
//! members via team_members. Runs against a real Postgres (cm-testkit).
//!
//! What this test locks in:
//! * Given a mission with `team_template_id` and no `team_id`,
//! `on_launch` materializes exactly one team per role in the
//! template.
//! * The materialized team carries `template_id` + `template_version`
//! + `risk_profile` + `mcp_bundles` from the template row.
//! * Every role produces one `agents` row + one
//! `agent_template_link` row (seeded=true).
//! * `team_members` binds every claw to a topology node id.
//! * The mission row's `team_id` gets updated.
//! * Re-invoking is a no-op (returns the existing team_id, doesn't
//! duplicate agents).
use cm_api::mission_orchestrator;
use cm_db::repo::{team_templates, users, workspaces};
use cm_domain::{Role, User, UserId, Workspace, WorkspaceId};
use serde_json::json;
use sqlx::Row;
use uuid::Uuid;
async fn seed_workspace(pool: &sqlx::PgPool) -> WorkspaceId {
let ws = Workspace {
id: WorkspaceId::new(),
name: "Missions Test".into(),
plan: "team".into(),
};
workspaces::insert(pool, &ws).await.unwrap();
ws.id
}
async fn seed_owner(pool: &sqlx::PgPool, ws: WorkspaceId) -> UserId {
let user = User {
id: UserId::new(),
workspace_id: ws,
email: "[email protected]".into(),
role: Role::Owner,
display_name: "Owner".into(),
created_at: time::OffsetDateTime::UNIX_EPOCH,
};
users::insert(pool, &user).await.unwrap();
user.id
}
async fn seed_test_template(pool: &sqlx::PgPool) -> Uuid {
let id = Uuid::now_v7();
team_templates::upsert_builtin(
pool,
team_templates::UpsertBuiltin {
id,
key: "test_backend",
name: "Test Backend Team",
stack: vec!["rust".into(), "postgres".into()],
default_topology: "pipeline",
risk_profile: "medium",
mcp_bundles: vec!["clawmates_skills".into()],
version: 1,
description: Some("Fixture template for orchestrator test"),
config: json!({}),
category: "development",
roles: vec![
team_templates::UpsertBuiltinRole {
slot: "planner",
order_idx: 0,
system_prompt: "Plan the feature. Break it into INT-XX items.",
skills: vec!["decompose-int-items".into()],
brain_seed: Some("# Planner\nBreak features into INT items."),
model: None,
},
team_templates::UpsertBuiltinRole {
slot: "coder",
order_idx: 1,
system_prompt: "Implement one INT item at a time.",
skills: vec!["write-rust-current-edition".into()],
brain_seed: Some("# Coder\nOne INT per commit."),
model: None,
},
team_templates::UpsertBuiltinRole {
slot: "reviewer",
order_idx: 2,
system_prompt: "Review each commit before merge.",
skills: vec!["code-review-checklist".into()],
brain_seed: None,
// The point of migration 0071: a reviewer that does NOT
// share a model with the coder it reviews.
model: Some("glm-4.7"),
},
],
},
)
.await
.unwrap();
id
}
async fn seed_mission(
pool: &sqlx::PgPool,
ws: WorkspaceId,
template_id: Uuid,
title: &str,
) -> Uuid {
let id = Uuid::now_v7();
sqlx::query(
"INSERT INTO missions
(id, workspace_id, title, template_kind, team_template_id, schedule, status, config)
VALUES ($1, $2, $3, 'research_and_code', $4, '{\"kind\":\"one_shot\"}'::jsonb,
'draft', '{}'::jsonb)",
)
.bind(id)
.bind(ws.as_uuid())
.bind(title)
.bind(template_id)
.execute(pool)
.await
.unwrap();
id
}
#[tokio::test]
async fn on_launch_materializes_team_from_template() {
let pool = cm_testkit::test_pool().await;
let ws = seed_workspace(&pool).await;
let owner = seed_owner(&pool, ws).await;
let template_id = seed_test_template(&pool).await;
let mission_id = seed_mission(&pool, ws, template_id, "Test Mission").await;
let team_id = mission_orchestrator::on_launch(&pool, ws, owner, mission_id, None)
.await
.expect("on_launch succeeds")
.expect("returns a team id");
// Team row exists with template lineage stamped.
let team_row = sqlx::query(
"SELECT template_id, template_version, risk_profile, mcp_bundles
FROM teams WHERE id = $1",
)
.bind(team_id)
.fetch_one(&pool)
.await
.unwrap();
let stamped_template_id: Uuid = team_row.get("template_id");
let stamped_version: i32 = team_row.get("template_version");
let stamped_risk: String = team_row.get("risk_profile");
assert_eq!(stamped_template_id, template_id);
assert_eq!(stamped_version, 1);
assert_eq!(stamped_risk, "medium");
// One agent per role — three total.
let agent_count: i64 =
sqlx::query_scalar("SELECT count(*)::bigint FROM team_members WHERE team_id = $1")
.bind(team_id)
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(agent_count, 3, "expected one member per role");
// Every member has a matching agents row + role_slot binding.
let member_slots: Vec<String> = sqlx::query_scalar(
"SELECT tm.role FROM team_members tm
JOIN agents a ON a.id = tm.claw_id
WHERE tm.team_id = $1
ORDER BY tm.role",
)
.bind(team_id)
.fetch_all(&pool)
.await
.unwrap();
assert_eq!(member_slots, vec!["coder", "planner", "reviewer"]);
// Every claw has an agent_template_link row with seeded=true.
let seeded_count: i64 = sqlx::query_scalar(
"SELECT count(*)::bigint FROM agent_template_link atl
JOIN team_members tm ON tm.claw_id = atl.agent_id
WHERE tm.team_id = $1
AND atl.template_id = $2",
)
.bind(team_id)
.bind(template_id)
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(
seeded_count, 3,
"expected every claw to have seeded lineage"
);
// Mission row was updated to point at the new team.
let bound_team_id: Uuid = sqlx::query_scalar("SELECT team_id FROM missions WHERE id = $1")
.bind(mission_id)
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(bound_team_id, team_id);
}
#[tokio::test]
async fn on_launch_is_idempotent() {
let pool = cm_testkit::test_pool().await;
let ws = seed_workspace(&pool).await;
let owner = seed_owner(&pool, ws).await;
let template_id = seed_test_template(&pool).await;
let mission_id = seed_mission(&pool, ws, template_id, "Idempotency Mission").await;
let team_a = mission_orchestrator::on_launch(&pool, ws, owner, mission_id, None)
.await
.unwrap()
.unwrap();
let team_b = mission_orchestrator::on_launch(&pool, ws, owner, mission_id, None)
.await
.unwrap()
.unwrap();
assert_eq!(
team_a, team_b,
"second invocation should return the same team_id"
);
// Still exactly three agents — no duplication.
let agent_count: i64 =
sqlx::query_scalar("SELECT count(*)::bigint FROM team_members WHERE team_id = $1")
.bind(team_a)
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(agent_count, 3);
}
#[tokio::test]
async fn on_launch_no_template_hard_fails() {
let pool = cm_testkit::test_pool().await;
let ws = seed_workspace(&pool).await;
let owner = seed_owner(&pool, ws).await;
// Mission with no team_template_id.
let mission_id = Uuid::now_v7();
sqlx::query(
"INSERT INTO missions
(id, workspace_id, title, template_kind, schedule, status, config)
VALUES ($1, $2, 'no template', 'refactor', '{\"kind\":\"one_shot\"}'::jsonb,
'draft', '{}'::jsonb)",
)
.bind(mission_id)
.bind(ws.as_uuid())
.execute(&pool)
.await
.unwrap();
let result = mission_orchestrator::on_launch(&pool, ws, owner, mission_id, None).await;
let err = result.expect_err("no template + no team must be a hard error");
assert!(
err.contains("no team_template_id") && err.contains("config.phase_teams"),
"unexpected error message: {err}"
);
// Mission stays with team_id NULL — no partial materialization.
let team_id: Option<Uuid> = sqlx::query_scalar("SELECT team_id FROM missions WHERE id = $1")
.bind(mission_id)
.fetch_one(&pool)
.await
.unwrap();
assert!(team_id.is_none());
}
/// Slice 5: an APPROVED roster outranks the team template.
///
/// The template gives every composed mission the same five roles on the same
/// image. A roster is the model's answer for THIS mission, and it is the only
/// path that carries a per-node backend — which is how a mission runs more than
/// one provider at all. If the template won, a heterogeneous roster would be
/// accepted, stored, and then silently ignored at launch.
#[tokio::test]
async fn an_approved_roster_outranks_the_template() {
let pool = cm_testkit::test_pool().await;
let ws = seed_workspace(&pool).await;
let template_id = seed_test_template(&pool).await;
let mission = seed_mission(&pool, ws, template_id, "roster beats template").await;
// With no roster, the shape comes from the template — the behaviour every
// composed mission had before this slice.
let from_template = mission_orchestrator::composed_graph(&pool, mission, &["mission"])
.await
.expect("template graph")
.expect("the template supplies a shape");
let template_nodes = from_template["nodes"].as_array().unwrap().len();
assert!(template_nodes >= 1);
assert!(
from_template["nodes"][0]["attrs"].get("backend").is_none(),
"a template cannot express a per-node backend — that is the gap the roster fills"
);
// Approve a roster the way the route does: the built graph under
// `config.roster`.
let roster = cm_api::mission_roster::Roster {
topology_kind: "pipeline".into(),
members: vec![
cm_api::mission_roster::RosterMember {
role: "implementer".into(),
backend: Some("claude".into()),
rationale: None,
},
cm_api::mission_roster::RosterMember {
role: "verifier".into(),
backend: Some("kimi".into()),
rationale: None,
},
],
};
let graph = roster.graph().expect("a runnable graph");
sqlx::query(
"UPDATE missions SET config = jsonb_set(config, '{roster}', $2::jsonb, true) WHERE id = $1",
)
.bind(mission)
.bind(&graph)
.execute(&pool)
.await
.unwrap();
let chosen = mission_orchestrator::composed_graph(&pool, mission, &["mission"])
.await
.expect("roster graph")
.expect("the roster supplies a shape");
let nodes = chosen["nodes"].as_array().unwrap();
assert_eq!(nodes.len(), 2, "the roster's two nodes, not the template's");
assert_eq!(nodes[0]["role"], "implementer");
assert_eq!(nodes[0]["attrs"]["backend"], "claude");
assert_eq!(nodes[1]["attrs"]["backend"], "kimi");
}
/// Migration 0071: a template role may name its own model, and the claw minted
/// for it must actually run on that model.
///
/// Before this, `mint_team_from_template` bound EVERY role to one literal — so a
/// template whose whole point is an independent reviewer minted a reviewer
/// sharing a model with the coder it reviews. That is the correlated failure the
/// cross-provider judge exists to break, reintroduced one layer down.
#[tokio::test]
async fn a_template_role_may_run_on_its_own_model() {
let pool = cm_testkit::test_pool().await;
let ws = seed_workspace(&pool).await;
let user = seed_owner(&pool, ws).await;
let template_id = seed_test_template(&pool).await;
let mission = seed_mission(&pool, ws, template_id, "per-role models").await;
mission_orchestrator::on_launch(&pool, ws, user, mission, None)
.await
.expect("launch");
let rows: Vec<(String, Option<String>)> = sqlx::query_as(
"SELECT job_title, model_binding FROM agents
WHERE workspace_id = $1 AND deleted_at IS NULL
ORDER BY job_title",
)
.bind(ws.as_uuid())
.fetch_all(&pool)
.await
.unwrap();
let by_role: std::collections::HashMap<_, _> = rows.into_iter().collect();
assert_eq!(
by_role.get("reviewer").and_then(|m| m.clone()).as_deref(),
Some("glm-4.7"),
"the reviewer must run the model its role names: {by_role:?}"
);
// And a role that names none still gets the mint's default, so every
// template written before 0071 behaves exactly as it did.
for silent in ["planner", "coder"] {
assert_eq!(
by_role.get(silent).and_then(|m| m.clone()).as_deref(),
Some("claude-sonnet-5"),
"{silent} named no model and must take the default"
);
}
}