fix(diag): "turn timed out" now says what the agent was doing

A research_and_code mission failed with:

  turn executor failed: turn timed out
  turn executor failed: turn timed out

and that is the entire record. Investigating it found: the run produced zero
steps and zero output, it died at exactly 700s (TURN_TIMEOUT), the node→claw
aliases were bound correctly, and the same zeroclaw team path passes in the
`multirole` scenario. So the platform path is fine and the agent simply never
finished a turn — but the one place the reason lived, the per-mission runtime
container, is torn down after the phase and takes its log with it. By the time
anyone looks, all that survives is the string.

The timeout now reads the last 40 lines out of that container while it still
exists, and reports which agent alias and which gateway it was driving.
Best-effort by construction: it runs on a path that is ALREADY failing, so a
docker error there degrades to a note rather than replacing the real failure
with a second one.

`container_name` derives the container from the gateway URL and returns None
rather than guessing, because this feeds a diagnostic — a wrong name would put a
different container's log under a failure and send the reader somewhere else
entirely.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-10 08:56:51 -07:00
co-authored by Claude Opus 5
parent e417247e7e
commit b2e2735583
2 changed files with 108 additions and 3 deletions
+66 -3
View File
@@ -219,13 +219,50 @@ impl ZeroClawDriveExecutor {
.await
.map_err(|e| OrchestratorError::Executor(format!("ws send failed: {e}")))?;
let outcome = tokio::time::timeout(TURN_TIMEOUT, Self::drain(&mut ws))
.await
.map_err(|_| OrchestratorError::Executor("turn timed out".into()))??;
let outcome = match tokio::time::timeout(TURN_TIMEOUT, Self::drain(&mut ws)).await {
Ok(res) => res?,
Err(_) => {
// "turn timed out" on its own is unactionable, and the one place
// the reason lives — the per-mission runtime container — is torn
// down after the phase, taking its log with it. Read the tail
// while it still exists.
//
// MEASURED: a research phase timed out at exactly 700s having
// produced zero steps and zero output, and the container was
// already gone by the time anyone looked. All that survived was
// the string.
let container = self.container_name();
let tail = match &container {
Some(c) => crate::container_exec::tail_logs(c, 40).await,
None => "(could not derive the container name from the gateway url)".into(),
};
return Err(OrchestratorError::Executor(format!(
"turn timed out after {}s driving agent {alias} on {} — the agent \
never finished a turn. Last lines from {}:\n{tail}",
TURN_TIMEOUT.as_secs(),
self.gateway_url,
container.as_deref().unwrap_or("its runtime container"),
)));
}
};
let _ = ws.close(None).await;
Ok(outcome)
}
/// The runtime container behind this executor, derived from its gateway URL
/// (`http://cm-runtime-mission-<hex>:42617`). Used only to fetch a log tail
/// for an error message, so an unparseable URL is `None` rather than a
/// failure.
fn container_name(&self) -> Option<String> {
let rest = self
.gateway_url
.split("://")
.nth(1)
.unwrap_or(&self.gateway_url);
let host = rest.split('/').next()?.split(':').next()?;
(!host.is_empty()).then(|| host.to_string())
}
/// Use a runtime agent as a governance judge: drive `alias` with the judge
/// prompt and parse the verdict (`DENY` anywhere ⇒ deny, else allow). This
/// lets a **subscription-only** model (e.g. Kimi via `kimi_cli`) be the judge
@@ -406,6 +443,32 @@ fn parse_agent_map(s: &str) -> HashMap<String, String> {
#[cfg(test)]
mod tests {
/// The container name comes out of the gateway URL, or nothing does.
///
/// This is only used to fetch a log tail for a failure message, so a URL
/// shape it does not recognise must degrade to "no log" rather than to a
/// second error on top of the first.
#[test]
fn the_container_name_is_derived_or_absent_never_wrong() {
let ex = |url: &str| ZeroClawDriveExecutor::new(
url.to_string(),
String::new(),
std::collections::HashMap::new(),
"scout".into(),
);
assert_eq!(
ex("http://cm-runtime-mission-019fec2d596f:42617").container_name().as_deref(),
Some("cm-runtime-mission-019fec2d596f")
);
assert_eq!(
ex("https://host.example:8443/base").container_name().as_deref(),
Some("host.example")
);
// No scheme is still a host.
assert_eq!(ex("clawmates-runtime:42617").container_name().as_deref(), Some("clawmates-runtime"));
assert_eq!(ex("").container_name(), None);
}
use super::*;
use axum::extract::ws::{Message as AxMsg, WebSocket, WebSocketUpgrade};
use axum::response::Response;