fix(models): coding runs on sonnet-5, judging on opus-5, haiku only as last resort
deploy / test (push) Successful in 7m38s
deploy / build (push) Successful in 6m55s

Operator model policy: haiku ONLY for genuine yes/no questions; anything
requiring thinking is opus-5; coding is sonnet-5.

The mission AGENTS were running haiku, and nothing in the product said so.
`provider_alias_for` maps every `claude-*` binding onto the single alias
`claude_cli.default`, so a crew whose `model_binding` reads `claude-sonnet-5`
— as this deployment's does — still ran whatever that alias pointed at, which
was `model = "haiku"` in the runtime config. The binding is cosmetic; the
alias is the truth.

Measured consequence on mission 01a00bbb: the coding agents claimed six INT
items complete and had committed three, and the done_when judge caught it by
auditing git history against the claims.

Model assignments, by what the component actually does:
  evaluator (done_when judge)  haiku  -> opus-5   reads evidence, audits it
                                                  against the repo, writes
                                                  guidance. The verdict is a
                                                  boolean; the work is not —
                                                  and this is the one component
                                                  whose failure mode is passing
                                                  work that was never done.
  judge_model                  4-8    -> opus-5
  mission_refiner              4-8    -> opus-5   composition
  phase_summarizer             4-8    -> opus-5   composition
  swarm planner                4-8    -> opus-5   planning
  subscription preflight head  4-8    -> opus-5
  fallback chain head          4-6    -> sonnet-5 haiku stays BELOW it as a
                                                  last-resort link, never a peer

Every value stays env-overridable; only the shipped defaults move.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-16 12:36:19 -07:00
co-authored by Claude Opus 5
parent 53da4d7e6d
commit 69c294addc
6 changed files with 29 additions and 13 deletions
+9 -4
View File
@@ -153,7 +153,12 @@ fn is_transient(e: &cm_llm::LlmError) -> bool {
///
/// Override with `CLAWMATES_MODEL_FALLBACK` (comma-separated). An empty value
/// disables fallback and restores plain "503 and wait".
const DEFAULT_FALLBACK: &str = "claude-sonnet-4-6,claude-haiku-4-5-20251001,\
///
/// Ordered by the operator's model policy: sonnet-5 is the working tier, and
/// haiku sits BELOW it as a last-resort Anthropic link rather than as a peer —
/// a degraded answer beats a 503, but it must never be reached while a capable
/// model has capacity.
const DEFAULT_FALLBACK: &str = "claude-sonnet-5,claude-haiku-4-5-20251001,\
kimi:kimi-k2.7-code,glm:glm-4.7,local:ornith-fleet:9b";
/// The chain to walk after `requested`, with `requested` itself removed so a
@@ -320,7 +325,7 @@ pub async fn preflight(runtime: &cm_runtime::Runtime, head: &str) -> Vec<(String
pub fn report_at_boot(runtime: cm_runtime::Runtime) {
tokio::spawn(async move {
let head = std::env::var("CLAWMATES_PREFLIGHT_HEAD")
.unwrap_or_else(|_| "claude-opus-4-8".to_string());
.unwrap_or_else(|_| "claude-opus-5".to_string());
let links = preflight(&runtime, &head).await;
let bad: Vec<_> = links.iter().filter(|(_, s)| !s.usable()).collect();
eprintln!(
@@ -593,11 +598,11 @@ mod tests {
#[test]
fn the_chain_excludes_the_model_that_just_failed() {
// No env override in scope: this asserts the SHIPPED default.
let chain = fallback_chain("claude-opus-4-8");
let chain = fallback_chain("claude-opus-5");
assert_eq!(
chain,
vec![
"claude-sonnet-4-6",
"claude-sonnet-5",
"claude-haiku-4-5-20251001",
"kimi:kimi-k2.7-code",
"glm:glm-4.7",