fix(missions): make an unrunnable test suite legible, and check the runtime at boot
ci / gates (push) Failing after 6s
ci / rust (push) Skipped
ci / frontend (push) Skipped
ci / e2e (push) Skipped
ci / publish (push) Skipped

Two changes against the same defect: the platform could not tell a missing
capability from a legitimate negative result.

verify_tests returned Option<bool>, collapsing four outcomes into None:
no suite found, docker unreachable, exec failed, and no exit status. When
clawmates-runtime shipped without cargo, every on_green_tests phase
returned None and landed on -wip — identical to the reading for "this
repo has no tests", which is the conclusion I drew and reported. The gate
was correct throughout; it simply could not say why it was unproven.

TestOutcome now names the four cases. Gating is unchanged (only Passed
clears, unproven is never a pass), and tests_verified keeps its tri-state
meaning for existing readers. tests_status and tests_detail are new, so an
artifact distinguishes no_suite from could_not_run, and a CouldNotRun is
logged as the infrastructure fault it is rather than passing quietly.

runtime_preflight probes the runtime container at boot for every tool the
platform invokes inside it and names what each absence disables. This is
the check that was missing: the Dockerfile gained a toolchain, the image
was never built, gw-04 ran the old one for days, and the only symptoms
were an ungated suite and a security scan that scanned nothing. A report,
not a gate — a missing scanner should stop us believing a scan, not stop
the server. Its test guards the probes themselves, since a typo would
produce a permanent false "missing" and train operators to ignore it.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-02 17:22:47 -07:00
co-authored by Claude Opus 5
parent 9bdc3cd89b
commit f7e336ff5f
5 changed files with 350 additions and 16 deletions
+5
View File
@@ -402,6 +402,11 @@ async fn run() -> Result<(), String> {
.await
.map_err(|e| format!("bind {} failed: {e}", config.listen_addr))?;
println!("clawmates-server listening on {}", config.listen_addr);
// Say plainly whether the mission runtime carries the tools we invoke in
// it. The image on the host silently fell behind its Dockerfile once, and
// every consequence — an ungated test suite, a scan that scanned nothing —
// looked like a normal result rather than a broken deployment.
cm_api::runtime_preflight::report_at_boot();
// Graceful shutdown: on SIGTERM/Ctrl-C, stop accepting, finish in-flight
// requests, then DRAIN the sandbox managers so no container is left running.
let shutdown = async move {