fix(runtime): a mission could not build the repo it was given
`clawmates-runtime` shipped with `gcc` and `make` but no `cmake`, no `g++` and no `python3-dev`. Measured on clawhdf5, three probes: no cmake → "is `cmake` not installed?" exit 101 after 13s no python3-dev → "cannot find -lpython3.11" exit 101 at link with both → cargo test PASSES exit 0 after 69s This is not only the delivery gate. The AGENTS run in this image, so a coding phase was writing Rust it had no way to compile or test — which reframes the last run's 11 agent commits as unverifiable by construction. `images/agent-toolchain/Dockerfile` (the microVM path) has had `cmake build-essential` all along, and its own header warns about precisely this: "if `cargo` is present in one image and absent in another, the same mission passes or fails depending on which backend it landed on, and nothing says why." Both images now install the same set — it was missing `python3-dev` too. `images/runtime-toolchain.Dockerfile` is a thin local overlay so the laptop can run today without recompiling zeroclaw from the fork; it is meant to be deleted once a runtime image built from the corrected deploy/ Dockerfile is published. Also: a build failure is no longer reported as a red suite. Both are cargo exit 101, and `verify_tests` mapped every non-zero to `Failed(code)` — so a missing toolchain was recorded as the USER's tests failing. It now returns `CouldNotRun` with the reason when the output shows a compile or link failure. Deliberately narrow: a failing `assert!` still reads as red, because letting broken code past `on_green_tests` is the expensive direction to be wrong in. Both directions are pinned by tests built from today's two real samples. And the coding phase finally has a loop: `research_and_code.toml` declared `loop = "until_no_more_int_items"`, which `phase_config.rs` lists as DECLARED_BUT_UNREAD. Iteration is driven by `max_iterations` + `done_when`, and with `max_iterations = 1` and no `done_when` the phase ran ONCE and was never judged — reporting `completed` whatever it produced. Now 3 passes against a stated goal, wording per the measured rule (say what the tree must CONTAIN). Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
10341cf7fe
commit
53da4d7e6d
@@ -1029,6 +1029,39 @@ pub struct Publish {
|
||||
/// reason: this codebase has repeatedly found things reporting success while
|
||||
/// doing nothing, and a test suite that never ran must not license a push to a
|
||||
/// mission branch.
|
||||
/// Did the toolchain fail to BUILD the project, as opposed to building it and
|
||||
/// finding failing tests?
|
||||
///
|
||||
/// Deliberately narrow. These three phrases are emitted by cargo/rustc only
|
||||
/// when compilation or linking did not complete; a failing `assert!` produces
|
||||
/// none of them. Anything not matched here stays a red suite, because guessing
|
||||
/// "probably an environment problem" over a genuine test failure is the far
|
||||
/// more expensive mistake — it would let broken code through the gate.
|
||||
fn build_failed(output: &str) -> bool {
|
||||
let o = output.to_ascii_lowercase();
|
||||
o.contains("error: could not compile")
|
||||
|| o.contains("error: linking with")
|
||||
|| o.contains("error: failed to run custom build command")
|
||||
}
|
||||
|
||||
/// The first line that explains a build failure, for the artifact.
|
||||
fn build_failure_excerpt(output: &str) -> String {
|
||||
output
|
||||
.lines()
|
||||
.find(|l| {
|
||||
let l = l.to_ascii_lowercase();
|
||||
l.contains("error: could not compile")
|
||||
|| l.contains("error: linking with")
|
||||
|| l.contains("error: failed to run custom build command")
|
||||
|| l.contains("cannot find -l")
|
||||
|| l.contains("not installed")
|
||||
})
|
||||
.unwrap_or("")
|
||||
.chars()
|
||||
.take(300)
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub async fn verify_tests(repo: &Path, container: &str) -> TestOutcome {
|
||||
let Some(argv) = discover_test_command(repo) else {
|
||||
return TestOutcome::NoSuite;
|
||||
@@ -1046,14 +1079,32 @@ pub async fn verify_tests(repo: &Path, container: &str) -> TestOutcome {
|
||||
argv.join(" "),
|
||||
out.exit_code
|
||||
);
|
||||
let text = out.combined();
|
||||
match out.exit_code {
|
||||
Some(0) => TestOutcome::Passed,
|
||||
// A suite that never COMPILED is not a red suite. Both are
|
||||
// non-zero (cargo exits 101 either way), and calling the
|
||||
// difference is what stops a missing toolchain being reported
|
||||
// as the user's code being broken.
|
||||
//
|
||||
// Measured twice on clawhdf5 in one sitting: no `cmake` gave
|
||||
// "is `cmake` not installed?", and no `python3-dev` gave
|
||||
// "cannot find -lpython3.11" — both exit 101, both would have
|
||||
// been recorded as `tests_status: "failed"` on a repo whose
|
||||
// tests were never run. The branch suffix is `-wip` either way,
|
||||
// so nothing ships differently; what changes is that the
|
||||
// artifact now says which of the two happened.
|
||||
Some(_) if build_failed(&text) => TestOutcome::CouldNotRun(format!(
|
||||
"`{}` could not build the project: {}",
|
||||
argv.join(" "),
|
||||
build_failure_excerpt(&text)
|
||||
)),
|
||||
// An unreadable status is not a pass, and it is not a red
|
||||
// suite either — the command may never have started.
|
||||
None => TestOutcome::CouldNotRun(format!(
|
||||
"`{}` produced no exit status: {}",
|
||||
argv.join(" "),
|
||||
out.combined().chars().take(300).collect::<String>()
|
||||
text.chars().take(300).collect::<String>()
|
||||
)),
|
||||
Some(code) => TestOutcome::Failed(code),
|
||||
}
|
||||
@@ -1409,6 +1460,37 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// A missing toolchain must not be reported as the user's tests failing.
|
||||
/// Both are cargo exit 101; only the output distinguishes them, and this
|
||||
/// session produced both real samples on clawhdf5.
|
||||
#[test]
|
||||
fn a_build_failure_is_not_a_red_suite() {
|
||||
let no_cmake = "error: failed to run custom build command for `libz-ng-sys v1.1.29`\n\
|
||||
is `cmake` not installed?";
|
||||
let no_python = "= note: /usr/bin/ld: cannot find -lpython3.11: No such file or directory\n\
|
||||
error: could not compile `clawhdf5-py` (lib) due to 1 previous error";
|
||||
for sample in [no_cmake, no_python] {
|
||||
assert!(build_failed(sample), "must read as a build failure: {sample}");
|
||||
assert!(
|
||||
!build_failure_excerpt(sample).is_empty(),
|
||||
"the artifact needs a reason, not an empty string"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The half that protects the gate: a genuinely failing test must STAY a
|
||||
/// red suite. Mistaking one for an environment problem would let broken
|
||||
/// code past `on_green_tests`, which is the expensive direction to be
|
||||
/// wrong in.
|
||||
#[test]
|
||||
fn a_failing_test_is_still_a_red_suite() {
|
||||
let red = "running 3 tests\n\
|
||||
test math::adds ... FAILED\n\
|
||||
failures:\n math::adds\n\
|
||||
test result: FAILED. 2 passed; 1 failed; 0 ignored";
|
||||
assert!(!build_failed(red), "a failing assertion is not a build failure");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_command_is_discovered_from_the_tree() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
|
||||
Reference in New Issue
Block a user