fix(microvm): a mission with no repository can run in a VM, and its work comes back

Two halves, and the first was worse than the plan assumed. `run_phase_in_vm`
packed `<missions_root>/<mission>/repo` unconditionally — a directory a
repo-less mission does not have — and then required `/mission/repo/.git` inside
the guest before spending a turn. So a repo-less microVM phase did not merely
go uncaptured: it failed before the agent ran.

A repo-less mission now gets an EMPTY workspace at the same guest path, created
host-side so the collect unpacks back over it with no special case, and the
readiness probe asks for what was actually sent — the directory rather than a
`.git` that was never going to be there.

`mission_outputs` then drops its `runtime_kind <> 'microvm'` exclusion, whose
stated reason ("a microVM mission always has a checkout") is exactly what
stopped being true. Where the files come from now depends on the runtime, and
the difference is not cosmetic: a container mission's output is still inside a
running container, while a VM's has already been unpacked onto the host by the
end-of-turn collect. Asking docker for a VM mission's files would query a
container that never existed.

The recursive copy skips symlinks rather than following them — a link out of
the tree would publish whatever it points at.

`research-vm` is the proof, added to the suite as well as the dispatch: the same
assertions as `research-only` with `runtime_kind: microvm`. A scenario nobody
runs is a scenario that does not exist.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-09 19:04:00 -07:00
co-authored by Claude Opus 5
parent e4bddeb1ba
commit 768e106614
6 changed files with 141 additions and 20 deletions
+35 -8
View File
@@ -407,6 +407,7 @@ pub async fn run_phase_in_vm(hub: &NodeHub, p: VmPhase<'_>) -> Result<VmOutcome,
&created, &created,
p.task, p.task,
p.repo, p.repo,
p.has_repo,
&env, &env,
p.team_engine, p.team_engine,
p.gate, p.gate,
@@ -444,6 +445,16 @@ pub struct VmPhase<'a> {
/// The host checkout, injected as a tar and collected back over the same /// The host checkout, injected as a tar and collected back over the same
/// path so `mission_delivery` needs no change. /// path so `mission_delivery` needs no change.
pub repo: &'a std::path::Path, pub repo: &'a std::path::Path,
/// Whether the mission has a repository at all.
///
/// A repo-less mission still gets a `/mission/repo` — the agents need
/// somewhere to write and the collect brings it back — but it is an empty
/// workspace rather than a checkout. Without this the inject packed a
/// directory that does not exist and the readiness probe demanded a `.git`
/// that never would, so a repo-less microVM phase failed before the agent
/// ran. It is the same `has_repo` `phase_runner` already threads through to
/// choose the prompt.
pub has_repo: bool,
/// `missions.team_engine` — `Some("claude_code")` asks the lead to form a /// `missions.team_engine` — `Some("claude_code")` asks the lead to form a
/// team. `None` is solo, which is the default. /// team. `None` is solo, which is the default.
pub team_engine: Option<&'a str>, pub team_engine: Option<&'a str>,
@@ -493,6 +504,9 @@ async fn run_inside(
created: &serde_json::Value, created: &serde_json::Value,
task: &str, task: &str,
repo: &std::path::Path, repo: &std::path::Path,
// See `VmPhase::has_repo`: a repo-less mission gets an EMPTY workspace at
// the same guest path, and is proven present differently.
has_repo: bool,
env: &[(String, String)], env: &[(String, String)],
engine: Option<&str>, engine: Option<&str>,
gate: Option<&crate::vm_stop_gate::StopGate>, gate: Option<&crate::vm_stop_gate::StopGate>,
@@ -516,6 +530,15 @@ async fn run_inside(
// The checkout, as a tar. `pack_dir` names the entry `repo`, and the guest // The checkout, as a tar. `pack_dir` names the entry `repo`, and the guest
// unpacks it under /mission, so it lands at /mission/repo. // unpacks it under /mission, so it lands at /mission/repo.
//
// A repo-less mission has no directory to pack. Create it — empty — rather
// than skipping the inject: the guest needs the workspace to exist before
// the agent writes into it, and creating it host-side means the collect
// unpacks back over the same path with no special case.
if !has_repo && !repo.exists() {
std::fs::create_dir_all(repo)
.map_err(|e| format!("create empty workspace {}: {e}", repo.display()))?;
}
let archive = crate::mission_fs::pack_dir(repo, "repo")?; let archive = crate::mission_fs::pack_dir(repo, "repo")?;
let injected = archive.len(); let injected = archive.len();
vm.inject("/mission", &archive).await?; vm.inject("/mission", &archive).await?;
@@ -523,18 +546,22 @@ async fn run_inside(
// Prove the guest actually has the checkout before spending an agent turn on // Prove the guest actually has the checkout before spending an agent turn on
// it. An inject that reports success while landing nothing would otherwise // it. An inject that reports success while landing nothing would otherwise
// become an agent reporting that the repository is empty. // become an agent reporting that the repository is empty.
// What "landed" means depends on what was sent. A checkout is proven by its
// `.git`; an empty workspace can only be proven by the directory itself,
// and demanding `.git` of it failed every repo-less microVM phase before
// the agent got a turn.
let want = if has_repo {
format!("{GUEST_REPO}/.git")
} else {
GUEST_REPO.to_string()
};
let probe = vm let probe = vm
.exec( .exec(&format!("test -d {want} && echo REPO-PRESENT"), None, 60, &[])
&format!("test -d {GUEST_REPO}/.git && echo REPO-PRESENT"),
None,
60,
&[],
)
.await?; .await?;
if !probe.stdout.contains("REPO-PRESENT") { if !probe.stdout.contains("REPO-PRESENT") {
return Err(format!( return Err(format!(
"the checkout did not land in the guest ({injected} bytes injected, \ "the workspace did not land in the guest ({injected} bytes injected, \
{GUEST_REPO}/.git is absent) — rc={} {}", {want} is absent) — rc={} {}",
probe.rc, probe.stderr probe.rc, probe.stderr
)); ));
} }
@@ -76,6 +76,9 @@ pub struct MicroVmTurnExecutor<V: PhaseVm> {
/// The mission's host checkout — injected into every node's VM and collected /// The mission's host checkout — injected into every node's VM and collected
/// back over, which is how file work survives a node boundary. /// back over, which is how file work survives a node boundary.
repo: PathBuf, repo: PathBuf,
/// Whether the mission has a repository. Carried so every graph node gets
/// the same workspace treatment as a solo phase — see `VmPhase::has_repo`.
has_repo: bool,
/// `missions.target_node_id`: the fleet node a mission was placed on. A node /// `missions.target_node_id`: the fleet node a mission was placed on. A node
/// may override it with `attrs["node_id"]`. /// may override it with `attrs["node_id"]`.
default_fleet_node: Option<Uuid>, default_fleet_node: Option<Uuid>,
@@ -102,6 +105,7 @@ pub struct ComposedRun {
pub phase_id: Uuid, pub phase_id: Uuid,
pub iteration: i32, pub iteration: i32,
pub repo: PathBuf, pub repo: PathBuf,
pub has_repo: bool,
pub target_node_id: Option<Uuid>, pub target_node_id: Option<Uuid>,
pub backend: Option<String>, pub backend: Option<String>,
pub team_engine: Option<String>, pub team_engine: Option<String>,
@@ -121,6 +125,7 @@ impl<V: PhaseVm> MicroVmTurnExecutor<V> {
phase_id: r.phase_id, phase_id: r.phase_id,
iteration: r.iteration, iteration: r.iteration,
repo: r.repo, repo: r.repo,
has_repo: r.has_repo,
default_fleet_node: r.target_node_id, default_fleet_node: r.target_node_id,
default_backend: r.backend, default_backend: r.backend,
team_engine: r.team_engine, team_engine: r.team_engine,
@@ -196,6 +201,7 @@ impl<V: PhaseVm> TurnExecutor for MicroVmTurnExecutor<V> {
task: &task, task: &task,
backend: backend.as_deref(), backend: backend.as_deref(),
repo: &self.repo, repo: &self.repo,
has_repo: self.has_repo,
team_engine: self.team_engine.as_deref(), team_engine: self.team_engine.as_deref(),
// Each node is its own agent session, so each carries the // Each node is its own agent session, so each carries the
// phase's gate. Threaded from the run rather than rebuilt here: // phase's gate. Threaded from the run rather than rebuilt here:
@@ -425,6 +431,7 @@ mod tests {
phase_id: Uuid::now_v7(), phase_id: Uuid::now_v7(),
iteration: 1, iteration: 1,
repo, repo,
has_repo: true,
target_node_id: Some(Uuid::now_v7()), target_node_id: Some(Uuid::now_v7()),
backend: Some("claude".into()), backend: Some("claude".into()),
team_engine: None, team_engine: None,
@@ -520,6 +527,7 @@ mod tests {
phase_id: Uuid::now_v7(), phase_id: Uuid::now_v7(),
iteration: 1, iteration: 1,
repo: d.path().join("repo"), repo: d.path().join("repo"),
has_repo: true,
target_node_id: Some(Uuid::now_v7()), target_node_id: Some(Uuid::now_v7()),
backend: None, backend: None,
team_engine: None, team_engine: None,
+55 -9
View File
@@ -66,14 +66,17 @@ const EMPTY_MARKER: &str = "NO-OUTPUT.md";
/// Capture the outputs of finished phases on missions that have no repo. /// Capture the outputs of finished phases on missions that have no repo.
pub async fn capture_repo_less_phases(pool: &PgPool) -> Result<(), String> { pub async fn capture_repo_less_phases(pool: &PgPool) -> Result<(), String> {
let rows = sqlx::query( let rows = sqlx::query(
"SELECT mp.id, mp.mission_id, mp.kind, mp.config "SELECT mp.id, mp.mission_id, mp.kind, mp.config, m.runtime_kind
FROM mission_phases mp FROM mission_phases mp
JOIN missions m ON m.id = mp.mission_id JOIN missions m ON m.id = mp.mission_id
WHERE mp.status IN ('completed', 'failed') WHERE mp.status IN ('completed', 'failed')
AND m.repo_id IS NULL AND m.repo_id IS NULL
-- A microVM mission always has a checkout (`run_phase_in_vm` refuses -- microVM used to be excluded here because `run_phase_in_vm`
-- to boot without one), so this path is container-only. -- refused to boot without a checkout. It no longer does: a
AND m.runtime_kind <> 'microvm' -- repo-less mission gets an empty workspace at the same guest path,
-- and the collect unpacks it back onto the host — so those files are
-- already on disk and `collect_into` reads them instead of asking a
-- container that never existed.
AND NOT EXISTS ( AND NOT EXISTS (
SELECT 1 FROM mission_artifacts a SELECT 1 FROM mission_artifacts a
WHERE a.mission_id = mp.mission_id WHERE a.mission_id = mp.mission_id
@@ -94,9 +97,10 @@ pub async fn capture_repo_less_phases(pool: &PgPool) -> Result<(), String> {
let mission_id: Uuid = row.get("mission_id"); let mission_id: Uuid = row.get("mission_id");
let kind: String = row.get("kind"); let kind: String = row.get("kind");
let config: serde_json::Value = row.get("config"); let config: serde_json::Value = row.get("config");
let runtime_kind: String = row.get("runtime_kind");
let dest = outputs_dir(mission_id, phase_id); let dest = outputs_dir(mission_id, phase_id);
let captured = match collect_into(mission_id, &dest).await { let captured = match collect_into(mission_id, &dest, &runtime_kind).await {
Ok(files) => files, Ok(files) => files,
Err(e) => { Err(e) => {
// Loud and retryable, never silently "captured nothing": the // Loud and retryable, never silently "captured nothing": the
@@ -239,18 +243,60 @@ async fn register_empty_marker(
.map_err(|e| format!("register empty marker: {e}")) .map_err(|e| format!("register empty marker: {e}"))
} }
/// Copy `/mission/repo` out of the mission's container and return the files kept. /// Gather the mission's produced files and return the ones worth keeping.
async fn collect_into(mission_id: Uuid, dest: &Path) -> Result<Vec<PathBuf>, String> { ///
let container = crate::mission_runtime::container_name(mission_id); /// Where they come from depends on the runtime, and the difference is not
let docker = crate::container_exec::connect()?; /// cosmetic: a container mission's files are still INSIDE a running container,
/// while a microVM's have already been unpacked onto the host by the collect at
/// the end of the turn (`microvm_executor` writes them over
/// `mission_workspace::checkout_path`). Asking docker for a VM mission's files
/// would query a container that never existed.
async fn collect_into(mission_id: Uuid, dest: &Path, runtime_kind: &str) -> Result<Vec<PathBuf>, String> {
// A stale copy from an earlier attempt would be registered as this pass's // A stale copy from an earlier attempt would be registered as this pass's
// output — the same "captured a tree nobody wrote" shape capture avoids. // output — the same "captured a tree nobody wrote" shape capture avoids.
let _ = std::fs::remove_dir_all(dest); let _ = std::fs::remove_dir_all(dest);
std::fs::create_dir_all(dest).map_err(|e| format!("create {}: {e}", dest.display()))?; std::fs::create_dir_all(dest).map_err(|e| format!("create {}: {e}", dest.display()))?;
if runtime_kind == "microvm" {
let src = crate::mission_workspace::checkout_path(mission_id);
if !src.is_dir() {
return Err(format!(
"{} is absent — the VM's collect did not land",
src.display()
));
}
copy_tree(&src, &dest.join("repo"))?;
return Ok(keep_files(&dest.join("repo")));
}
let container = crate::mission_runtime::container_name(mission_id);
let docker = crate::container_exec::connect()?;
crate::mission_fs::copy_out(&docker, &container, "/mission/repo", dest).await?; crate::mission_fs::copy_out(&docker, &container, "/mission/repo", dest).await?;
Ok(keep_files(&dest.join("repo"))) Ok(keep_files(&dest.join("repo")))
} }
/// Recursive file copy. Small on purpose — the alternative is a dependency or a
/// shell-out, and this runs as the server's own uid against its own directory.
fn copy_tree(src: &Path, dest: &Path) -> Result<(), String> {
std::fs::create_dir_all(dest).map_err(|e| format!("create {}: {e}", dest.display()))?;
let entries =
std::fs::read_dir(src).map_err(|e| format!("read {}: {e}", src.display()))?;
for entry in entries.flatten() {
let from = entry.path();
let to = dest.join(entry.file_name());
match entry.file_type() {
Ok(t) if t.is_dir() => copy_tree(&from, &to)?,
Ok(t) if t.is_file() => {
std::fs::copy(&from, &to).map_err(|e| format!("copy {}: {e}", from.display()))?;
}
// Symlinks and specials are skipped rather than followed: a link out
// of the tree would publish whatever it points at.
_ => {}
}
}
Ok(())
}
/// Every regular file worth keeping, recursively. /// Every regular file worth keeping, recursively.
fn keep_files(root: &Path) -> Vec<PathBuf> { fn keep_files(root: &Path) -> Vec<PathBuf> {
let mut out = Vec::new(); let mut out = Vec::new();
+6
View File
@@ -829,6 +829,7 @@ async fn launch_phase(
chosen_node, chosen_node,
p.team_engine, p.team_engine,
crate::vm_stop_gate::StopGate::for_phase(kind, p.config), crate::vm_stop_gate::StopGate::for_phase(kind, p.config),
has_repo,
) )
.await; .await;
} }
@@ -1087,6 +1088,10 @@ async fn launch_microvm_phase(
target_node_id: Option<Uuid>, target_node_id: Option<Uuid>,
team_engine: Option<&str>, team_engine: Option<&str>,
gate: Option<crate::vm_stop_gate::StopGate>, gate: Option<crate::vm_stop_gate::StopGate>,
// Whether the mission has a repository. A repo-less mission gets an EMPTY
// workspace at the same guest path instead of a checkout — see
// `VmPhase::has_repo`.
has_repo: bool,
) -> Result<(), String> { ) -> Result<(), String> {
sqlx::query( sqlx::query(
"DELETE FROM topology_runs "DELETE FROM topology_runs
@@ -1165,6 +1170,7 @@ async fn launch_microvm_phase(
task: &task, task: &task,
backend: backend.as_deref(), backend: backend.as_deref(),
repo: &repo, repo: &repo,
has_repo,
team_engine: team_engine.as_deref(), team_engine: team_engine.as_deref(),
gate: gate.as_ref(), gate: gate.as_ref(),
// The solo path is one VM for the whole phase; only a // The solo path is one VM for the whole phase; only a
+10 -2
View File
@@ -286,8 +286,11 @@ async fn run_composed(
OrchestratorError::Executor("a composed run must belong to a mission phase".into()) OrchestratorError::Executor("a composed run must belong to a mission phase".into())
})?; })?;
let mission: (Option<Uuid>, Option<String>, Option<String>) = let mission: (Option<Uuid>, Option<String>, Option<String>, bool) =
sqlx::query_as("SELECT target_node_id, backend, team_engine FROM missions WHERE id = $1") sqlx::query_as(
"SELECT target_node_id, backend, team_engine, (repo_id IS NOT NULL) \
FROM missions WHERE id = $1",
)
.bind(mission_id) .bind(mission_id)
.fetch_one(pool) .fetch_one(pool)
.await .await
@@ -312,6 +315,11 @@ async fn run_composed(
phase_id, phase_id,
iteration: job.iteration.unwrap_or(1), iteration: job.iteration.unwrap_or(1),
repo: crate::mission_workspace::checkout_path(mission_id), repo: crate::mission_workspace::checkout_path(mission_id),
// A repo-less composed mission gets an empty shared workspace, the
// same as a solo phase — the graph's whole property is that node 2
// sees node 1's files, and that holds whether or not it is a git
// checkout.
has_repo: mission.3,
target_node_id: mission.0, target_node_id: mission.0,
backend: mission.1, backend: mission.1,
team_engine: mission.2, team_engine: mission.2,
+27 -1
View File
@@ -468,6 +468,28 @@ RESEARCH_ONLY_BODY=$(cat <<JSON
JSON JSON
) )
# The same repo-less research, in a microVM.
#
# This path could not run at all until now: `run_phase_in_vm` packed a checkout
# directory that does not exist for a repo-less mission, and then demanded a
# `.git` inside the guest that never would. So it failed before the agent got a
# turn, and `mission_outputs` excluded microvm entirely on the grounds that a VM
# mission "always has a checkout". Both halves are gone; this is what proves it.
RESEARCH_VM_BODY=$(cat <<JSON
{"title":"verify: a repo-less research mission in a microVM",
"template_kind":"research_only",
"team_template_id":"$TEAM_TEMPLATE",
"runtime_kind":"microvm",
"backend":"claude",
"description":"Prove a mission with no repository runs in a VM and its output comes back.",
"phases":[
{"kind":"research","order_idx":0,"config":{"max_iterations":1,
"task":"Write exactly two markdown files under /mission/repo/research/: 01_findings.md and 02_notes.md. Each 5-10 lines about Rust error handling. Create no other files."}}
]}
JSON
)
assert_research_only() { # <token> <mission> <report> assert_research_only() { # <token> <mission> <report>
local token="$1" mission="$2" report="$3" docs names scaffold local token="$1" mission="$2" report="$3" docs names scaffold
while read -r idx status _f _p _b _c _e; do while read -r idx status _f _p _b _c _e; do
@@ -1493,6 +1515,9 @@ case "${1:-all}" in
research-only) research-only)
run_scenario research-only "$(echo "$RESEARCH_ONLY_BODY" | tr -d '\n')" assert_research_only no-checkout run_scenario research-only "$(echo "$RESEARCH_ONLY_BODY" | tr -d '\n')" assert_research_only no-checkout
;; ;;
research-vm)
run_scenario research-vm "$(echo "$RESEARCH_VM_BODY" | tr -d '\n')" assert_research_only no-checkout
;;
benchmark) benchmark)
run_scenario benchmark "$(echo "$BENCHMARK_BODY" | tr -d '\n')" assert_benchmark run_scenario benchmark "$(echo "$BENCHMARK_BODY" | tr -d '\n')" assert_benchmark
;; ;;
@@ -1527,6 +1552,7 @@ case "${1:-all}" in
scenario_microvm_unavailable_backend scenario_microvm_unavailable_backend
run_scenario gatecap "$(echo "$GATECAP_BODY" | tr -d '\n')" assert_gate_cap run_scenario gatecap "$(echo "$GATECAP_BODY" | tr -d '\n')" assert_gate_cap
run_scenario research-only "$(echo "$RESEARCH_ONLY_BODY" | tr -d '\n')" assert_research_only no-checkout run_scenario research-only "$(echo "$RESEARCH_ONLY_BODY" | tr -d '\n')" assert_research_only no-checkout
run_scenario research-vm "$(echo "$RESEARCH_VM_BODY" | tr -d '\n')" assert_research_only no-checkout
run_scenario benchmark "$(echo "$BENCHMARK_BODY" | tr -d '\n')" assert_benchmark run_scenario benchmark "$(echo "$BENCHMARK_BODY" | tr -d '\n')" assert_benchmark
run_scenario security "$(echo "$SECURITY_BODY" | tr -d '\n')" assert_security run_scenario security "$(echo "$SECURITY_BODY" | tr -d '\n')" assert_security
run_scenario refactor "$(echo "$REFACTOR_BODY" | tr -d '\n')" assert_refactor run_scenario refactor "$(echo "$REFACTOR_BODY" | tr -d '\n')" assert_refactor
@@ -1537,7 +1563,7 @@ case "${1:-all}" in
scenario_drain_midmission scenario_drain_midmission
;; ;;
*) *)
die "unknown scenario: $1 (selftest|uids|chain|multirole|noop|microvm|canary|gatecap|research-only|benchmark|security|refactor|composed|roster|local-ornith|capacity|drain-midmission|all)" die "unknown scenario: $1 (selftest|uids|chain|multirole|noop|microvm|canary|gatecap|research-only|research-vm|benchmark|security|refactor|composed|roster|local-ornith|capacity|drain-midmission|all)"
;; ;;
esac esac