perf(evaluator): judge with a bare API call instead of an agent (156x fewer tokens)

A phase verdict is a classification: fixed prompt, no tools, no memory, one
JSON answer. Routing it through a ZeroClaw agent charged 17,772 input tokens
to produce a 20-token reply, and at the runtime's 32k context that scaffolding
— role prompt, tool descriptors, memory, identity — consumed over half the
window before the judge read any evidence.

The same verdict as a direct Messages API call costs 114 input tokens, with
the real system prompt and evidence. Measured through the production seam via
`cargo run -p cm-llm --example oauth_probe`.

- cm-llm: teach AnthropicProvider subscription auth. A `sk-ant-oat…`
  credential switches to bearer auth, adds the Claude Code beta set, and
  prepends the identity line the API requires as the first system block —
  idempotently, so re-wrapping can't stack it or waste tokens.
- evaluator: prefer a direct provider call whenever ANTHROPIC_OAUTH_TOKEN is
  set, falling back to the configured spec (including `runtime:<alias>`)
  otherwise. Fail-closed parsing is untouched and still governs every path.
- The ANTHROPIC_API_KEY shape guard now points at the slot that understands
  bearer auth rather than only saying no.

Deleting the agent from this path is the ablation applied to our own harness:
the scaffolding was there because a judge was built like every other agent,
not because a judge needs it.

Co-Authored-By: Claude Opus 5 <[email protected]>

crates/cm-llm/src/anthropic.rs

@@ -9,6 +9,16 @@ use crate::provider::{
     ChatRequest, ChatRole, ContentPart, EventStream, LlmError, LlmEvent, LlmProvider, StopReason,
 };
 
+/// The Claude Code identity line Anthropic requires as the first system block
+/// when authenticating with a subscription OAuth token.
+const OAUTH_SYSTEM_PREAMBLE: &str = "You are Claude Code, Anthropic's official CLI for Claude.";
+
+/// Setup tokens minted by `claude setup-token` carry this prefix. API keys are
+/// `sk-ant-api…`, so the shape is enough to pick the auth scheme.
+fn is_setup_token(credential: &str) -> bool {
+    credential.trim().starts_with("sk-ant-oat")
+}
+
 pub struct AnthropicProvider {
     client: reqwest::Client,
     base_url: String,
@@ -29,6 +39,24 @@ impl AnthropicProvider {
         }
     }
 
+    /// Whether this provider is authenticating with a subscription token
+    /// rather than an API key. Callers that report cost attribution care.
+    pub fn is_subscription(&self) -> bool {
+        is_setup_token(&self.api_key)
+    }
+
+    /// Prepend the Claude Code identity line, unless the caller's system
+    /// prompt already opens with it (so repeated wrapping can't stack).
+    fn with_oauth_preamble(system: &str) -> String {
+        if system.trim_start().starts_with(OAUTH_SYSTEM_PREAMBLE) {
+            return system.to_string();
+        }
+        if system.trim().is_empty() {
+            return OAUTH_SYSTEM_PREAMBLE.to_string();
+        }
+        format!("{OAUTH_SYSTEM_PREAMBLE}\n\n{system}")
+    }
+
     fn wire_messages(request: &ChatRequest) -> Vec<Value> {
         request
             .messages
@@ -89,20 +117,38 @@ impl LlmProvider for AnthropicProvider {
             // Anthropic server-side web search — the model searches the web itself.
             tools.push(json!({"type": "web_search_20250305", "name": "web_search", "max_uses": 5}));
         }
+        // A subscription token authenticates as Claude Code: bearer auth, the
+        // Claude Code beta set, and a system prompt whose first line is the
+        // Claude Code identity. Sending it as `x-api-key` returns 401.
+        let oauth = is_setup_token(&self.api_key);
+        let system = if oauth {
+            Self::with_oauth_preamble(&request.system)
+        } else {
+            request.system.clone()
+        };
         let body = json!({
             "model": request.model,
             "max_tokens": request.max_tokens,
-            "system": request.system,
+            "system": system,
             "messages": AnthropicProvider::wire_messages(&request),
             "tools": tools,
             "stream": true,
         });
 
-        let response = self
+        let mut req = self
             .client
             .post(format!("{}/v1/messages", self.base_url))
-            .header("x-api-key", &self.api_key)
-            .header("anthropic-version", "2023-06-01")
+            .header("anthropic-version", "2023-06-01");
+        req = if oauth {
+            req.header("authorization", format!("Bearer {}", self.api_key))
+                .header(
+                    "anthropic-beta",
+                    "claude-code-20250219,oauth-2025-04-20,interleaved-thinking-2025-05-14",
+                )
+        } else {
+            req.header("x-api-key", &self.api_key)
+        };
+        let response = req
             .json(&body)
             .send()
             .await
@@ -191,3 +237,43 @@ impl LlmProvider for AnthropicProvider {
         Ok(Box::pin(stream))
     }
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn setup_tokens_are_distinguished_from_api_keys() {
+        assert!(is_setup_token("sk-ant-oat01-abc"));
+        assert!(is_setup_token("  sk-ant-oat01-abc  "), "trims first");
+        assert!(!is_setup_token("sk-ant-api03-abc"));
+        assert!(!is_setup_token(""));
+    }
+
+    #[test]
+    fn oauth_preamble_is_prepended_once() {
+        let once = AnthropicProvider::with_oauth_preamble("Judge the condition.");
+        assert!(once.starts_with(OAUTH_SYSTEM_PREAMBLE));
+        assert!(once.ends_with("Judge the condition."));
+        // Re-wrapping must not stack the identity line — the API rejects a
+        // system prompt that doesn't *start* with it, and duplicating it is
+        // pure token waste on a path whose whole point is being cheap.
+        let twice = AnthropicProvider::with_oauth_preamble(&once);
+        assert_eq!(once, twice);
+        assert_eq!(twice.matches(OAUTH_SYSTEM_PREAMBLE).count(), 1);
+    }
+
+    #[test]
+    fn oauth_preamble_handles_an_empty_system_prompt() {
+        assert_eq!(
+            AnthropicProvider::with_oauth_preamble("   "),
+            OAUTH_SYSTEM_PREAMBLE
+        );
+    }
+
+    #[test]
+    fn is_subscription_reports_the_credential_kind() {
+        assert!(AnthropicProvider::new("sk-ant-oat01-x".into()).is_subscription());
+        assert!(!AnthropicProvider::new("sk-ant-api03-x".into()).is_subscription());
+    }
+}