fix(models): coding runs on sonnet-5, judging on opus-5, haiku only as last resort
Operator model policy: haiku ONLY for genuine yes/no questions; anything
requiring thinking is opus-5; coding is sonnet-5.
The mission AGENTS were running haiku, and nothing in the product said so.
`provider_alias_for` maps every `claude-*` binding onto the single alias
`claude_cli.default`, so a crew whose `model_binding` reads `claude-sonnet-5`
— as this deployment's does — still ran whatever that alias pointed at, which
was `model = "haiku"` in the runtime config. The binding is cosmetic; the
alias is the truth.
Measured consequence on mission 01a00bbb: the coding agents claimed six INT
items complete and had committed three, and the done_when judge caught it by
auditing git history against the claims.
Model assignments, by what the component actually does:
evaluator (done_when judge) haiku -> opus-5 reads evidence, audits it
against the repo, writes
guidance. The verdict is a
boolean; the work is not —
and this is the one component
whose failure mode is passing
work that was never done.
judge_model 4-8 -> opus-5
mission_refiner 4-8 -> opus-5 composition
phase_summarizer 4-8 -> opus-5 composition
swarm planner 4-8 -> opus-5 planning
subscription preflight head 4-8 -> opus-5
fallback chain head 4-6 -> sonnet-5 haiku stays BELOW it as a
last-resort link, never a peer
Every value stays env-overridable; only the shipped defaults move.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
53da4d7e6d
commit
69c294addc
@@ -153,7 +153,12 @@ fn is_transient(e: &cm_llm::LlmError) -> bool {
|
||||
///
|
||||
/// Override with `CLAWMATES_MODEL_FALLBACK` (comma-separated). An empty value
|
||||
/// disables fallback and restores plain "503 and wait".
|
||||
const DEFAULT_FALLBACK: &str = "claude-sonnet-4-6,claude-haiku-4-5-20251001,\
|
||||
///
|
||||
/// Ordered by the operator's model policy: sonnet-5 is the working tier, and
|
||||
/// haiku sits BELOW it as a last-resort Anthropic link rather than as a peer —
|
||||
/// a degraded answer beats a 503, but it must never be reached while a capable
|
||||
/// model has capacity.
|
||||
const DEFAULT_FALLBACK: &str = "claude-sonnet-5,claude-haiku-4-5-20251001,\
|
||||
kimi:kimi-k2.7-code,glm:glm-4.7,local:ornith-fleet:9b";
|
||||
|
||||
/// The chain to walk after `requested`, with `requested` itself removed so a
|
||||
@@ -320,7 +325,7 @@ pub async fn preflight(runtime: &cm_runtime::Runtime, head: &str) -> Vec<(String
|
||||
pub fn report_at_boot(runtime: cm_runtime::Runtime) {
|
||||
tokio::spawn(async move {
|
||||
let head = std::env::var("CLAWMATES_PREFLIGHT_HEAD")
|
||||
.unwrap_or_else(|_| "claude-opus-4-8".to_string());
|
||||
.unwrap_or_else(|_| "claude-opus-5".to_string());
|
||||
let links = preflight(&runtime, &head).await;
|
||||
let bad: Vec<_> = links.iter().filter(|(_, s)| !s.usable()).collect();
|
||||
eprintln!(
|
||||
@@ -593,11 +598,11 @@ mod tests {
|
||||
#[test]
|
||||
fn the_chain_excludes_the_model_that_just_failed() {
|
||||
// No env override in scope: this asserts the SHIPPED default.
|
||||
let chain = fallback_chain("claude-opus-4-8");
|
||||
let chain = fallback_chain("claude-opus-5");
|
||||
assert_eq!(
|
||||
chain,
|
||||
vec![
|
||||
"claude-sonnet-4-6",
|
||||
"claude-sonnet-5",
|
||||
"claude-haiku-4-5-20251001",
|
||||
"kimi:kimi-k2.7-code",
|
||||
"glm:glm-4.7",
|
||||
|
||||
Reference in New Issue
Block a user