feat(llm): the subscription is the default provider, with a recorded fallback chain

Two changes so an empty metered account stops being a platform outage.

1. `build_provider` prefers the subscription token over ANTHROPIC_API_KEY.
   A bare model name resolves to whatever this returns, so making it the
   subscription means no server-side call can reach the metered key by
   construction — rather than by a source-grep test that already missed four
   call sites once. The metered key remains a fallback and now warns loudly
   when it is the one in use; boot no longer requires it at all.

2. `complete_with_fallback` walks a declared chain when a model has no
   capacity: opus -> haiku -> glm:glm-4.7 by default, overridable via
   CLAWMATES_MODEL_FALLBACK, empty to disable. Measured on gw-04 today: opus
   and sonnet return 429 on the subscription while haiku, GLM and Kimi all
   return 200, so a capped window no longer means "the planner is gone".

The chain returns the model that ANSWERED, and every caller persists it —
mission_plan_proposals.author_model, mission_team_proposals.author_model, and
the swarm's step role. A plan drafted by the third link and filed as an opus
plan is a silent quality change, which is the failure shape this project keeps
paying for. Two negative controls hold the design: the chain never retries the
model that just failed as its own fallback, and it steps down ONLY for a
capacity failure — walking it on a malformed prompt would ask three models the
same bad question and report the third one's confusion.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-08 22:56:01 -07:00
co-authored by Claude Opus 5
parent ee5a939ce6
commit 9c9439a271
5 changed files with 161 additions and 20 deletions
+104
View File
@@ -124,6 +124,74 @@ fn is_transient(e: &cm_llm::LlmError) -> bool {
}
}
/// Models to try, in order, when the requested one is rate limited.
///
/// Measured on this deployment 2026-08-08: opus and sonnet returned 429 on the
/// subscription while haiku returned 200, and GLM — funded separately, and
/// already the project's measured-best independent judge — returned 200. So the
/// default chain steps down within the subscription first, then leaves Anthropic
/// entirely rather than making a capped window mean "the planner is gone".
///
/// Override with `CLAWMATES_MODEL_FALLBACK` (comma-separated). An empty value
/// disables fallback and restores plain "503 and wait".
const DEFAULT_FALLBACK: &str = "claude-haiku-4-5-20251001,glm:glm-4.7";
/// The chain to walk after `requested`, with `requested` itself removed so a
/// capped model is never retried as its own fallback.
pub fn fallback_chain(requested: &str) -> Vec<String> {
let raw =
std::env::var("CLAWMATES_MODEL_FALLBACK").unwrap_or_else(|_| DEFAULT_FALLBACK.to_string());
raw.split(',')
.map(str::trim)
.filter(|m| !m.is_empty() && *m != requested.trim())
.map(str::to_string)
.collect()
}
/// Whether a failure means "this model has no capacity right now" as opposed
/// to "this request was wrong".
///
/// The distinction is the whole safety of the chain: walking it on a malformed
/// prompt would ask three models the same bad question and report the third
/// one's confusion, while walking it on a rate limit is exactly the point.
pub fn is_capacity_failure(err: &str) -> bool {
err.contains("rate_limit") || err.contains("429") || err.contains("credit balance")
}
/// One completion, stepping down `fallback_chain` when a model has no capacity.
///
/// Returns the text **and the model that actually produced it**. Callers must
/// persist that second value: a plan drafted by the third link in the chain and
/// filed as an opus plan is a silent quality change, which is the failure shape
/// this project keeps paying for. Every hop is logged.
pub async fn complete_with_fallback(
runtime: &cm_runtime::Runtime,
system: &str,
user: &str,
model: &str,
max_tokens: u32,
web_search: bool,
) -> Result<(String, String), String> {
let mut last = match complete_or(runtime, system, user, model, max_tokens, web_search).await {
Ok(text) => return Ok((text, model.to_string())),
Err(e) if is_capacity_failure(&e) => e,
// A real error. Do not launder it through two more models.
Err(e) => return Err(e),
};
for next in fallback_chain(model) {
eprintln!("model fallback: {model} has no capacity ({last}) — trying {next}");
match complete_or(runtime, system, user, &next, max_tokens, web_search).await {
Ok(text) => {
eprintln!("model fallback: {next} answered in place of {model}");
return Ok((text, next));
}
Err(e) if is_capacity_failure(&e) => last = e,
Err(e) => return Err(format!("fallback {next}: {e}")),
}
}
Err(last)
}
/// Turn a `complete_or` failure into the right API error.
///
/// A rate limit that outlived the backoff is not a bug in this server, and
@@ -275,6 +343,42 @@ mod tests {
assert!(!is_transient(&LlmError::Wire("bad json".into())));
}
/// The chain never retries the capped model as its own fallback.
///
/// Without the filter, asking for haiku while haiku is capped would try
/// haiku, fail, and try haiku again — a chain that looks like resilience
/// and delivers none.
#[test]
fn the_chain_excludes_the_model_that_just_failed() {
// No env override in scope: this asserts the SHIPPED default.
let chain = fallback_chain("claude-opus-4-8");
assert_eq!(chain, vec!["claude-haiku-4-5-20251001", "glm:glm-4.7"]);
assert!(!fallback_chain("claude-haiku-4-5-20251001")
.iter()
.any(|m| m == "claude-haiku-4-5-20251001"));
}
/// The chain is walked for "no capacity" and NOT for "bad request".
///
/// Walking it on a malformed prompt would ask three models the same bad
/// question and report the third one's confusion as the answer, burning
/// the two credentials that still work in order to hide the real error.
#[test]
fn only_a_capacity_failure_steps_down_the_chain() {
assert!(is_capacity_failure(
"subscription call: provider returned an error: 429 Too Many Requests"
));
assert!(is_capacity_failure("rate_limit_error"));
// The metered key's wall counts too — same meaning, different wording.
assert!(is_capacity_failure(
"400: Your credit balance is too low to access the Anthropic API"
));
assert!(!is_capacity_failure("400: messages.0: text content is empty"));
assert!(!is_capacity_failure("401: invalid x-api-key"));
assert!(!is_capacity_failure("404: model not found"));
}
/// An operator's explicit provider choice is never hijacked.
///
/// The swarm worker model is a configured `name:model` spec. Routing that