fix(llm): the chain preflight printed nothing at all

Deployed, and the report simply did not appear — from the tool built to stop
things failing silently. Two causes, both worth keeping:

There was no timeout anywhere in the probe, so one slow provider swallowed the
entire report. Each link is now bounded at 60s (generous: `complete_or` spends
up to 30s in its own backoff, so a tighter cap would report a merely throttled
link as hung) with `TimedOut` as its own state, and every line is emitted AS IT
RESOLVES rather than collected and printed at the end — a later link that hangs
must not be able to hide the ones already checked.

The first attempt at the timeout awaited the probe and then wrapped the result:

    let probe = complete_or(...).await;
    timeout(PROBE_TIMEOUT, async { probe }).await

That compiles, reads correctly, and bounds nothing. The timeout has to wrap the
future.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-09 14:51:51 -07:00
co-authored by Claude Opus 5
parent c7c3eeab46
commit 3c3d01c8d1
+24 -7
View File
@@ -222,6 +222,10 @@ pub async fn complete_with_fallback(
pub enum LinkStatus {
Answered,
Throttled(String),
/// Never came back. Its own state because it is the one that used to make
/// the whole report vanish: with no timeout, a single hung provider meant
/// silence from the tool built to prevent silence.
TimedOut,
/// The spec named a provider the registry does not have, so
/// `resolve_provider` silently fell back to the DEFAULT provider. The link
/// would "work" while running on entirely the wrong model.
@@ -237,6 +241,9 @@ impl LinkStatus {
match self {
LinkStatus::Answered => "ok".into(),
LinkStatus::Throttled(_) => "throttled (configured, no capacity now)".into(),
LinkStatus::TimedOut => {
format!("TIMED OUT after {}s — treat as down", PROBE_TIMEOUT.as_secs())
}
LinkStatus::Unregistered => "UNREGISTERED — resolves to the DEFAULT provider".into(),
LinkStatus::Broken(e) => format!("BROKEN: {}", e.chars().take(120).collect::<String>()),
}
@@ -251,6 +258,11 @@ impl LinkStatus {
/// back to the default provider for an unknown provider name, so a typo in
/// `kimi:` does not error — it quietly runs on Anthropic, and the chain reads
/// as five providers while being one.
/// Per-link ceiling. Generous on purpose: `complete_or` spends up to 30s in its
/// own backoff before giving up, so anything under that would report a merely
/// throttled link as hung.
const PROBE_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
pub async fn preflight(runtime: &cm_runtime::Runtime, head: &str) -> Vec<(String, LinkStatus)> {
let mut out = Vec::new();
for spec in std::iter::once(head.to_string()).chain(fallback_chain(head)) {
@@ -275,20 +287,25 @@ pub async fn preflight(runtime: &cm_runtime::Runtime, head: &str) -> Vec<(String
// `400 the message at position 0 with role 'system' must not be empty` —
// so an empty probe reported a healthy provider as BROKEN on the first
// live run. The probe must look like the traffic it stands in for.
let status = match complete_or(
// NOT awaited here — the timeout has to wrap the FUTURE. Awaiting first
// and wrapping the result compiles, reads correctly, and bounds nothing.
let probe = complete_or(
runtime,
"You are a reachability probe.",
"Reply with exactly: OK",
&spec,
8,
false,
)
.await
{
Ok(_) => LinkStatus::Answered,
Err(e) if is_capacity_failure(&e) => LinkStatus::Throttled(e),
Err(e) => LinkStatus::Broken(e),
);
let status = match tokio::time::timeout(PROBE_TIMEOUT, probe).await {
Err(_) => LinkStatus::TimedOut,
Ok(Ok(_)) => LinkStatus::Answered,
Ok(Err(e)) if is_capacity_failure(&e) => LinkStatus::Throttled(e),
Ok(Err(e)) => LinkStatus::Broken(e),
};
// Emitted as it resolves, not collected and printed at the end. A later
// link that hangs must not be able to hide the ones already checked.
eprintln!("fallback chain: {spec:<32} {}", status.label());
out.push((spec, status));
}
out