fix(missions): stop three launch failures from passing as success
A verification run against the deployed stack found a chain mission whose phase 0 reported `completed` with zero files, no commit error and no push error — indistinguishable from a phase that correctly had nothing to do. Three separate defects had to line up, each of them the same shape: a failure sharing its representation with a legitimate negative result. 1. `pin_agent_workspaces` embedded the whole config in one `sh -c` argv. That works until the file grows — config gains a block per provisioned claw — then fails with `argument list too long`. Now written through the tar upload API, which has no argv limit, so the failure mode is gone rather than merely further away. 2. A failed pin was logged "(continuing)". Without the pin, agents write to their sandboxes and the committer finds nothing in /mission/repo — the mission cannot deliver, so the launch now fails where someone is still looking. The restart that applies the pin is fatal for the same reason. 3. `capture_phase_diff_at` swallowed `git diff` failures with `unwrap_or_default`, so an unreadable base landed `empty: true, files_changed: 0` — byte-identical to an honest no-op. The error is now recorded as `diff_error`, and an empty patch that came from a failed diff is no longer trusted to mean an unchanged tree. Adds scripts/verify-mission-delivery.sh, which found #1 and #2 on its first real run. Its probes are fail-closed: no placeholder values, a self-test that proves the uid probe can detect the split it looks for, and FAIL-NORUN for a scenario that never executed. Its own first version had this bug too — a `die` inside `$(...)` exited the subshell, so a run that could not authenticate printed "all checks passed" and exited 0. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
bb274d08c6
commit
1253595ba7
@@ -90,6 +90,60 @@ pub async fn copy_in(
|
||||
.map_err(|e| format!("copy into {container}:{CONTAINER_MISSION_DIR}: {e}"))
|
||||
}
|
||||
|
||||
/// Build a one-entry tar. Split out from [`put_file`] so the size-independence
|
||||
/// that is the whole point can be tested without Docker.
|
||||
fn single_file_archive(name: &str, contents: &[u8]) -> Result<Vec<u8>, String> {
|
||||
let mut header = tar::Header::new_gnu();
|
||||
header
|
||||
.set_path(name)
|
||||
.map_err(|e| format!("tar path {name}: {e}"))?;
|
||||
header.set_size(contents.len() as u64);
|
||||
header.set_mode(0o600);
|
||||
header.set_entry_type(tar::EntryType::Regular);
|
||||
header.set_cksum();
|
||||
|
||||
let mut builder = tar::Builder::new(Vec::new());
|
||||
builder
|
||||
.append(&header, contents)
|
||||
.map_err(|e| format!("tar {name}: {e}"))?;
|
||||
builder
|
||||
.into_inner()
|
||||
.map_err(|e| format!("finish archive for {name}: {e}"))
|
||||
}
|
||||
|
||||
/// Write one file into a container, at any size.
|
||||
///
|
||||
/// The obvious way to do this is `sh -c "printf … > file"`, and it works right
|
||||
/// up until the payload approaches `ARG_MAX`, at which point exec fails with
|
||||
/// `argument list too long`. That is a size-dependent failure in a code path
|
||||
/// whose payload grows with use, which makes it a bug that ships green and
|
||||
/// surfaces in production — as it did, silently unpinning every agent in
|
||||
/// mission `019fcf62`. Tar has no argv limit.
|
||||
///
|
||||
/// The write is not atomic. Callers that need it can upload beside the target
|
||||
/// and rename; the config writer does not, because the daemon reads its config
|
||||
/// once at boot and is restarted afterwards.
|
||||
pub async fn put_file(
|
||||
docker: &Docker,
|
||||
container: &str,
|
||||
path: &str,
|
||||
contents: &[u8],
|
||||
) -> Result<(), String> {
|
||||
let (dir, file) = path
|
||||
.rsplit_once('/')
|
||||
.ok_or_else(|| format!("{path} is not an absolute path"))?;
|
||||
let dir = if dir.is_empty() { "/" } else { dir };
|
||||
|
||||
let archive = single_file_archive(file, contents)?;
|
||||
let opts = bollard::query_parameters::UploadToContainerOptionsBuilder::default()
|
||||
.path(dir)
|
||||
.build();
|
||||
docker
|
||||
.upload_to_container(container, Some(opts), bollard::body_full(archive.into()))
|
||||
.await
|
||||
.map_err(|e| format!("upload {path} to {container}: {e}"))
|
||||
}
|
||||
|
||||
/// Copy a directory back out of a container onto the host.
|
||||
pub async fn copy_out(
|
||||
docker: &Docker,
|
||||
@@ -260,6 +314,39 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// The regression this exists for: a config large enough to blow `ARG_MAX`
|
||||
/// via `sh -c` must round-trip untouched. 2 MB is well past the ~128 KB
|
||||
/// limit that unpinned every agent in mission `019fcf62`.
|
||||
#[test]
|
||||
fn a_file_far_past_arg_max_round_trips() {
|
||||
let big = "workspace_path = \"/mission/repo\"\n".repeat(64 * 1024);
|
||||
assert!(big.len() > 2_000_000, "the fixture must exceed ARG_MAX");
|
||||
|
||||
let archive = single_file_archive("config.toml", big.as_bytes()).unwrap();
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
unpack_into(&archive, tmp.path()).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
std::fs::read_to_string(tmp.path().join("config.toml")).unwrap(),
|
||||
big,
|
||||
"a large config must survive byte-for-byte"
|
||||
);
|
||||
}
|
||||
|
||||
/// TOML holding quotes, newlines and backslashes went through a shell
|
||||
/// before; nothing may depend on quoting now.
|
||||
#[test]
|
||||
fn shell_metacharacters_survive_the_archive() {
|
||||
let nasty = "path = \"/a'b\\\"c\"\n$(rm -rf /) `id` \\\\ \n";
|
||||
let archive = single_file_archive("config.toml", nasty.as_bytes()).unwrap();
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
unpack_into(&archive, tmp.path()).unwrap();
|
||||
assert_eq!(
|
||||
std::fs::read_to_string(tmp.path().join("config.toml")).unwrap(),
|
||||
nasty
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_directory_packs_without_error() {
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
|
||||
Reference in New Issue
Block a user