feat(llm): the subscription is the default provider, with a recorded fallback chain
Two changes so an empty metered account stops being a platform outage. 1. `build_provider` prefers the subscription token over ANTHROPIC_API_KEY. A bare model name resolves to whatever this returns, so making it the subscription means no server-side call can reach the metered key by construction — rather than by a source-grep test that already missed four call sites once. The metered key remains a fallback and now warns loudly when it is the one in use; boot no longer requires it at all. 2. `complete_with_fallback` walks a declared chain when a model has no capacity: opus -> haiku -> glm:glm-4.7 by default, overridable via CLAWMATES_MODEL_FALLBACK, empty to disable. Measured on gw-04 today: opus and sonnet return 429 on the subscription while haiku, GLM and Kimi all return 200, so a capped window no longer means "the planner is gone". The chain returns the model that ANSWERED, and every caller persists it — mission_plan_proposals.author_model, mission_team_proposals.author_model, and the swarm's step role. A plan drafted by the third link and filed as an opus plan is a silent quality change, which is the failure shape this project keeps paying for. Two negative controls hold the design: the chain never retries the model that just failed as its own fallback, and it steps down ONLY for a capacity failure — walking it on a malformed prompt would ask three models the same bad question and report the third one's confusion. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
ee5a939ce6
commit
9c9439a271
@@ -124,6 +124,74 @@ fn is_transient(e: &cm_llm::LlmError) -> bool {
|
||||
}
|
||||
}
|
||||
|
||||
/// Models to try, in order, when the requested one is rate limited.
|
||||
///
|
||||
/// Measured on this deployment 2026-08-08: opus and sonnet returned 429 on the
|
||||
/// subscription while haiku returned 200, and GLM — funded separately, and
|
||||
/// already the project's measured-best independent judge — returned 200. So the
|
||||
/// default chain steps down within the subscription first, then leaves Anthropic
|
||||
/// entirely rather than making a capped window mean "the planner is gone".
|
||||
///
|
||||
/// Override with `CLAWMATES_MODEL_FALLBACK` (comma-separated). An empty value
|
||||
/// disables fallback and restores plain "503 and wait".
|
||||
const DEFAULT_FALLBACK: &str = "claude-haiku-4-5-20251001,glm:glm-4.7";
|
||||
|
||||
/// The chain to walk after `requested`, with `requested` itself removed so a
|
||||
/// capped model is never retried as its own fallback.
|
||||
pub fn fallback_chain(requested: &str) -> Vec<String> {
|
||||
let raw =
|
||||
std::env::var("CLAWMATES_MODEL_FALLBACK").unwrap_or_else(|_| DEFAULT_FALLBACK.to_string());
|
||||
raw.split(',')
|
||||
.map(str::trim)
|
||||
.filter(|m| !m.is_empty() && *m != requested.trim())
|
||||
.map(str::to_string)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Whether a failure means "this model has no capacity right now" as opposed
|
||||
/// to "this request was wrong".
|
||||
///
|
||||
/// The distinction is the whole safety of the chain: walking it on a malformed
|
||||
/// prompt would ask three models the same bad question and report the third
|
||||
/// one's confusion, while walking it on a rate limit is exactly the point.
|
||||
pub fn is_capacity_failure(err: &str) -> bool {
|
||||
err.contains("rate_limit") || err.contains("429") || err.contains("credit balance")
|
||||
}
|
||||
|
||||
/// One completion, stepping down `fallback_chain` when a model has no capacity.
|
||||
///
|
||||
/// Returns the text **and the model that actually produced it**. Callers must
|
||||
/// persist that second value: a plan drafted by the third link in the chain and
|
||||
/// filed as an opus plan is a silent quality change, which is the failure shape
|
||||
/// this project keeps paying for. Every hop is logged.
|
||||
pub async fn complete_with_fallback(
|
||||
runtime: &cm_runtime::Runtime,
|
||||
system: &str,
|
||||
user: &str,
|
||||
model: &str,
|
||||
max_tokens: u32,
|
||||
web_search: bool,
|
||||
) -> Result<(String, String), String> {
|
||||
let mut last = match complete_or(runtime, system, user, model, max_tokens, web_search).await {
|
||||
Ok(text) => return Ok((text, model.to_string())),
|
||||
Err(e) if is_capacity_failure(&e) => e,
|
||||
// A real error. Do not launder it through two more models.
|
||||
Err(e) => return Err(e),
|
||||
};
|
||||
for next in fallback_chain(model) {
|
||||
eprintln!("model fallback: {model} has no capacity ({last}) — trying {next}");
|
||||
match complete_or(runtime, system, user, &next, max_tokens, web_search).await {
|
||||
Ok(text) => {
|
||||
eprintln!("model fallback: {next} answered in place of {model}");
|
||||
return Ok((text, next));
|
||||
}
|
||||
Err(e) if is_capacity_failure(&e) => last = e,
|
||||
Err(e) => return Err(format!("fallback {next}: {e}")),
|
||||
}
|
||||
}
|
||||
Err(last)
|
||||
}
|
||||
|
||||
/// Turn a `complete_or` failure into the right API error.
|
||||
///
|
||||
/// A rate limit that outlived the backoff is not a bug in this server, and
|
||||
@@ -275,6 +343,42 @@ mod tests {
|
||||
assert!(!is_transient(&LlmError::Wire("bad json".into())));
|
||||
}
|
||||
|
||||
/// The chain never retries the capped model as its own fallback.
|
||||
///
|
||||
/// Without the filter, asking for haiku while haiku is capped would try
|
||||
/// haiku, fail, and try haiku again — a chain that looks like resilience
|
||||
/// and delivers none.
|
||||
#[test]
|
||||
fn the_chain_excludes_the_model_that_just_failed() {
|
||||
// No env override in scope: this asserts the SHIPPED default.
|
||||
let chain = fallback_chain("claude-opus-4-8");
|
||||
assert_eq!(chain, vec!["claude-haiku-4-5-20251001", "glm:glm-4.7"]);
|
||||
assert!(!fallback_chain("claude-haiku-4-5-20251001")
|
||||
.iter()
|
||||
.any(|m| m == "claude-haiku-4-5-20251001"));
|
||||
}
|
||||
|
||||
/// The chain is walked for "no capacity" and NOT for "bad request".
|
||||
///
|
||||
/// Walking it on a malformed prompt would ask three models the same bad
|
||||
/// question and report the third one's confusion as the answer, burning
|
||||
/// the two credentials that still work in order to hide the real error.
|
||||
#[test]
|
||||
fn only_a_capacity_failure_steps_down_the_chain() {
|
||||
assert!(is_capacity_failure(
|
||||
"subscription call: provider returned an error: 429 Too Many Requests"
|
||||
));
|
||||
assert!(is_capacity_failure("rate_limit_error"));
|
||||
// The metered key's wall counts too — same meaning, different wording.
|
||||
assert!(is_capacity_failure(
|
||||
"400: Your credit balance is too low to access the Anthropic API"
|
||||
));
|
||||
|
||||
assert!(!is_capacity_failure("400: messages.0: text content is empty"));
|
||||
assert!(!is_capacity_failure("401: invalid x-api-key"));
|
||||
assert!(!is_capacity_failure("404: model not found"));
|
||||
}
|
||||
|
||||
/// An operator's explicit provider choice is never hijacked.
|
||||
///
|
||||
/// The swarm worker model is a configured `name:model` spec. Routing that
|
||||
|
||||
Reference in New Issue
Block a user