feat(missions): Slice 4 — the two engines composed, with the file handoff proven
`team_engine='composed'` (the third name migration 0069 anticipated) runs a
mission as a durable ZeroClaw graph whose every node is a whole
Claude-Code-in-a-microVM session. Engine Z owns checkpoint/resume, cancellation
and per-node heterogeneity; Engine C owns shared context and cheap fan-out;
neither has the other's asset, which is why this is a composition and not a
compromise.
`MicroVmTurnExecutor` implements the existing `TurnExecutor`, so it inherits the
planners, the checkpoint, the stale-run recovery, `close_finished_phases`, the
evaluator, capture and delivery unchanged — the same trick `SubTopologyExecutor`
already plays with a heavy `run_turn`. Producer side emits ONE `queued` row
carrying the real graph and lets the worker claim it: the durability IS being
worker-driven, and the solo path's `tokio::spawn` has none of it. Still exactly
one `topology_runs` row per unit of work and one completion path — `finish()` is
now that one place, shared by every tier.
THE TRAP, solved and proven. A VM is inject → run → collect → destroy, so a
per-node VM with text-only handoff silently loses every file an earlier node
wrote: node 2 boots from the original checkout, sees nothing, and still reports
success. The mission's host checkout is the medium — every node injects from it
and collects back over it — and two properties make that safe rather than lucky:
`execute_resumable` is strictly sequential, so two VMs never write one directory;
and the vm id is deterministic per (phase, iteration, step), so a duplicate is
refused by the node ("vm already exists") instead of becoming a second writer.
NEGATIVE CONTROL, run rather than assumed: with `repo` swapped for a private
per-node workspace, `a_later_node_sees_an_earlier_nodes_files` FAILS with
`saw:[]`; restored, it passes. The `PhaseVm` seam exists for exactly this — it
models inject/collect through the real `mission_fs` tar path in milliseconds.
Two durability traps this tier walks into, both closed:
- `requeue_stale` fires at 180s on `updated_at`, and one node here can run for
an hour. `SubTopologyExecutor` keeps its parent alive from each leaf step;
there is nothing between the start and end of a VM turn, so the turn holds a
ticker that touches `updated_at` every 30s and aborts on drop. Without it a
healthy composed run is requeued mid-node and boots a second VM.
- the 15-minute stuck-run reaper asks "any step records since it was CREATED?",
which describes a healthy composed run as readily as a wedged one. Hence
`REAPABLE_TIERS` — worker-driven minus this tier. Reaping it would be #54 in
a different costume.
`on_launch` mints no team for a microVM mission, deliberately: claws in
containers are what a VM mission does not use. So `mission_orchestrator::
composed_graph` builds the shape from the team template directly — nodes, roles
and pattern, zero claws provisioned. Per-node `attrs["backend"]` and
`attrs["node_id"]` override the mission's, which is what makes a validator node
on another provider's image a first-class graph node; a malformed `node_id`
fails the node rather than quietly running it where the graph did not ask.
Refusals are recorded as a failed run, not returned as an error: `launch_phase`
is swept every ten seconds, so a returned error is a phase that retries forever
while the log repeats itself.
501 tests pass, clippy clean. NOT yet proven end to end: no composed mission has
run on the fleet, so the resume-after-a-killed-worker leg is argued from the DB
test and the step-numbering test, not from a real two-node run.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
e31688bac5
commit
12147a1e01
@@ -574,6 +574,73 @@ async fn mint_team_from_template(
|
||||
Ok(team_id)
|
||||
}
|
||||
|
||||
/// The graph a COMPOSED microVM mission runs, built from its team template
|
||||
/// without minting a single claw.
|
||||
///
|
||||
/// A composed mission needs the template's *shape* — how many nodes, in what
|
||||
/// pattern, playing what roles — and nothing else it carries. Its nodes are VMs,
|
||||
/// so provisioning claws for them would create agents, containers and `.brain`
|
||||
/// files that nothing ever dials; that is exactly why `on_launch` returns early
|
||||
/// for a microVM mission, and this is how the composed path gets its graph
|
||||
/// anyway rather than by undoing that.
|
||||
///
|
||||
/// `purposes` is the phase's purpose list, matched against `config.phase_teams`;
|
||||
/// missions using the legacy single `team_template_id` fall back to it.
|
||||
/// Returns `None` when the mission picked no template at all.
|
||||
pub async fn composed_graph(
|
||||
pool: &PgPool,
|
||||
mission_id: Uuid,
|
||||
purposes: &[&str],
|
||||
) -> Result<Option<serde_json::Value>, String> {
|
||||
let row: Option<(serde_json::Value, Option<Uuid>)> =
|
||||
sqlx::query_as("SELECT config, team_template_id FROM missions WHERE id = $1")
|
||||
.bind(mission_id)
|
||||
.fetch_optional(pool)
|
||||
.await
|
||||
.map_err(|e| format!("load mission {mission_id}: {e}"))?;
|
||||
let Some((config, legacy_template)) = row else {
|
||||
return Err(format!("mission {mission_id} not found"));
|
||||
};
|
||||
|
||||
let template_id = config
|
||||
.get("phase_teams")
|
||||
.and_then(|v| v.as_object())
|
||||
.and_then(|pt| {
|
||||
// First template named by any purpose this phase answers to, in the
|
||||
// phase's own preference order — the same order `launch_phase` uses
|
||||
// to pick teams, so a composed mission and a ZeroClaw one resolve the
|
||||
// same template for the same phase.
|
||||
purposes.iter().find_map(|p| {
|
||||
pt.get(*p)
|
||||
.and_then(|v| v.as_array())
|
||||
.and_then(|a| a.first())
|
||||
.and_then(|v| v.as_str())
|
||||
.and_then(|s| Uuid::parse_str(s).ok())
|
||||
})
|
||||
})
|
||||
.or(legacy_template);
|
||||
let Some(template_id) = template_id else {
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
let template = cm_db::repo::team_templates::get(pool, template_id)
|
||||
.await
|
||||
.map_err(|e| format!("load template {template_id}: {e}"))?
|
||||
.ok_or_else(|| format!("template {template_id} not found"))?;
|
||||
let roles: Vec<&str> = template.roles.iter().map(|r| r.slot.as_str()).collect();
|
||||
if roles.is_empty() {
|
||||
return Err(format!("template {template_id} defines no roles"));
|
||||
}
|
||||
let graph = cm_topology::build(
|
||||
parse_topology_kind(&template.template.default_topology),
|
||||
&roles,
|
||||
)
|
||||
.map_err(|e| format!("build topology graph for template {template_id}: {e}"))?;
|
||||
serde_json::to_value(&graph)
|
||||
.map(Some)
|
||||
.map_err(|e| format!("serialize topology graph: {e}"))
|
||||
}
|
||||
|
||||
fn parse_topology_kind(s: &str) -> cm_topology::TopologyKind {
|
||||
use cm_topology::TopologyKind;
|
||||
match s {
|
||||
|
||||
Reference in New Issue
Block a user