feat(judge): an independent GLM judge, on the newest model z.ai publishes
Every phase verdict this session was Anthropic grading Anthropic, and the boot
log said so on each start:
validator_preflight: no CLAWMATES_VALIDATOR_MODEL — phase verdicts are
judged by the house model, which is NOT an independent check
`evaluator.rs` already preferred a cross-provider judge and refused to call a
same-family one `independent`; the local stack simply had no non-Anthropic
credential. It now carries the same `glm` provider gw-04 has had all along —
`format = "anthropic"` is load-bearing, since z.ai's OpenAI-compatible endpoint
is ToS-throttled for raw SDK access while its Anthropic-format one is not.
Model: glm-5.3, the newest z.ai lists (4.5, 4.5-air, 4.6, 4.7, 5, 5-turbo,
5.1, 5.2, 5.3 as of 2026-08-17). gw-04 still runs glm-4.7.
glm-5.3 is a REASONING model: it emits a `thinking` block before its JSON. Our
SSE parser ignores `thinking_delta` and keeps the text, so the wire shape is
compatible — but on a realistic phase-evidence prompt it spent 819 of the
evaluator's 1024 output tokens. A longer phase would truncate the verdict, and
a truncated verdict parses as empty and FAILS CLOSED, burning one of the
phase's passes on a judge that never answered — precisely how mission 01a00bbb
lost a pass. max_tokens raised to 2048.
Measured before wiring: asked to judge 25 commits claiming INT-01..INT-25 with
tests passing, glm-5.3 returned met=false because the evidence never
established what the brief actually required. That skepticism is the point of
an independent judge.
Boot now reports: `validator_preflight: independent validator glm:glm-5.3
answered`.
The key lives in .env (gitignored), never in this file.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
b5032a732a
commit
9c4b0722e8
@@ -625,7 +625,14 @@ async fn judge_with_tools(
|
|||||||
model: model.to_string(),
|
model: model.to_string(),
|
||||||
messages: messages.clone(),
|
messages: messages.clone(),
|
||||||
tools: tools.clone(),
|
tools: tools.clone(),
|
||||||
max_tokens: 1024,
|
// Headroom for a REASONING judge. glm-5.3 emits a `thinking`
|
||||||
|
// block before its JSON and spent 819 of 1024 tokens on a
|
||||||
|
// realistic phase-evidence prompt; a longer phase would truncate
|
||||||
|
// the verdict, and a truncated verdict parses as empty and fails
|
||||||
|
// closed — burning one of the phase's passes on a judge that
|
||||||
|
// never answered, exactly as the subscription judge did on
|
||||||
|
// mission 01a00bbb.
|
||||||
|
max_tokens: 2048,
|
||||||
web_search: false,
|
web_search: false,
|
||||||
};
|
};
|
||||||
let mut stream = provider.stream(request).await.map_err(|e| e.to_string())?;
|
let mut stream = provider.stream(request).await.map_err(|e| e.to_string())?;
|
||||||
|
|||||||
@@ -14,6 +14,23 @@ provider = "openai_compat"
|
|||||||
base_url = "http://local-llm:8000/v1"
|
base_url = "http://local-llm:8000/v1"
|
||||||
model = "qwen2.5-72b-instruct"
|
model = "qwen2.5-72b-instruct"
|
||||||
|
|
||||||
|
# GLM, so a phase verdict is judged by a DIFFERENT provider family than the one
|
||||||
|
# that did the work. `evaluator.rs` refuses to call a same-family judge
|
||||||
|
# `independent`, and until now this stack had only Anthropic credentials — so
|
||||||
|
# every verdict was Anthropic grading Anthropic and the boot log said so:
|
||||||
|
#
|
||||||
|
# validator_preflight: no CLAWMATES_VALIDATOR_MODEL — phase verdicts are
|
||||||
|
# judged by the house model, which is NOT an independent check
|
||||||
|
#
|
||||||
|
# `format = "anthropic"` is required, not cosmetic: z.ai's OpenAI-compatible
|
||||||
|
# endpoint is ToS-throttled for raw SDK access, while its Anthropic-format
|
||||||
|
# endpoint accepts them. Mirrors gw-04's clawmates.toml exactly.
|
||||||
|
[[llm.providers]]
|
||||||
|
name = "glm"
|
||||||
|
base_url = "https://api.z.ai/api/anthropic"
|
||||||
|
api_key_env = "ZAI_API_KEY"
|
||||||
|
format = "anthropic"
|
||||||
|
|
||||||
[auth]
|
[auth]
|
||||||
mode = "local"
|
mode = "local"
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user