feat(judge): Kimi as a fallback independent judge when GLM cannot answer
GLM's plan limit ran out for the second time in a month, and with one judge every conditioned phase on every mission fails until it resets. When the primary independent judge errors, CLAWMATES_VALIDATOR_FALLBACK_MODEL (prod: kimi:kimi-for-coding) judges instead, through the same independence checks (one shared function) plus one more: never the primary's own family. Measured first: judge-eval 15 cases x 3, kimi-for-coding 44/45 vs glm-5.3 43/45; goodhart — the false positive that once ruled Kimi out — 3/3. When both fail, the primary's error leads so the phase runner still reads the z.ai plan-limit code and does not spend the pass. Prod's kimi provider moved to the anthropic format the eval used (host config, backed up). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
a13f01e09c
commit
abfba832e1
@@ -47,6 +47,11 @@
|
||||
# a verification plan before reading the evidence (evaluator.rs), neither of
|
||||
# which this script exercises.
|
||||
#
|
||||
# MEASURED 2026-09-23, kimi-for-coding, three draws of all fifteen: 44/45.
|
||||
# goodhart 3/3 (its false positive on the original five is gone); the one
|
||||
# miss is a should-panic-hack draw, the same shape GLM misses. Wired as the
|
||||
# fallback judge (CLAWMATES_VALIDATOR_FALLBACK_MODEL), not the primary.
|
||||
#
|
||||
# Usage:
|
||||
#
|
||||
# scripts/judge-eval.sh # the configured validator
|
||||
|
||||
Reference in New Issue
Block a user