fix(llm): the preflight found two broken links on its first live run, one its own

fallback chain (6 link(s), 4 usable):
    claude-opus-4-8            ok
    claude-sonnet-4-6          ok
    claude-haiku-4-5-20251001  ok
    kimi:kimi-k2.7-code        BROKEN: 400 ... role 'system' must not be empty
    glm:glm-4.7                ok
    local:ornith-fleet:9b      UNREGISTERED — resolves to the DEFAULT provider

Neither link was actually broken.

The probe sent an EMPTY system prompt, which Kimi rejects outright. A probe has
to look like the traffic it stands in for, or it measures itself.

The second is the one worth keeping. `resolve_provider` returns a spec unchanged
when it does not recognise the provider, and the part after the FIRST colon when
it does — so the obvious test, "does the model half still contain a colon",
reads correctly and is wrong the moment a model id has one. `ornith-fleet:9b`
has one. The probe reported a provider the server had just finished registering
as UNREGISTERED.

`evaluator::cross_provider_judge` had the identical check, and would therefore
have refused a local judge as "not independent" — silently falling back to a
same-family one, which is the exact claim that path exists to make honestly.
Both now compare against the whole spec.

That bug was written into the codebase before a model name with a colon existed,
was correct at the time, and became wrong when one arrived. Nothing would have
reported it; a boot-time probe of every link did, on its first run.

Co-Authored-By: Claude Opus 5 <[email protected]>

crates/cm-api/src/evaluator.rs

@@ -347,9 +347,9 @@ async fn cross_provider_judge(
     // `independent = true`. An Anthropic judge grading Anthropic work, labelled
     // independent, which is the one claim this whole path exists to make honestly.
     //
-    // The colon check below caught the same fallback for `glm:glm-4.7` when the
+    // The check below caught the same fallback for `glm:glm-4.7` when the `glm`
-    // `glm` provider was missing, because the unrouted spec comes back WITH its
+    // provider was missing, because an unrouted spec comes back WHOLE. It could
-    // colon. It could never catch a bare name.
+    // never catch a bare name.
     if !names_a_provider(spec) {
         eprintln!(
             "evaluator: CLAWMATES_VALIDATOR_MODEL={spec} is not a registry spec \
@@ -359,7 +359,16 @@ async fn cross_provider_judge(
         return None;
     }
     let (provider, model) = runtime.resolve_provider(spec);
-    if model.contains(':') {
+    // Compared against the WHOLE spec, not tested for a colon.
+    //
+    // `resolve_provider` returns the spec unchanged when it does not recognise
+    // the provider, and returns the part after the FIRST colon when it does. The
+    // old test — "does the model half still contain a colon" — assumed model
+    // names never do. `local:ornith-fleet:9b` resolves correctly to provider
+    // `local`, model `ornith-fleet:9b`, and was rejected as unregistered. The
+    // chain preflight found it by reporting a provider it had just registered as
+    // UNREGISTERED.
+    if model == spec {
         eprintln!(
             "evaluator: no provider registered for {spec} — refusing to judge with the \
              default provider and call it independent"

crates/cm-api/src/subscription.rs

@@ -257,15 +257,33 @@ pub async fn preflight(runtime: &cm_runtime::Runtime, head: &str) -> Vec<(String
         // A qualified spec whose provider is missing resolves to the default —
         // detected the same way `cross_provider_judge` does it, by asking what
         // the model half came back as.
-        if let Some((name, _)) = spec.split_once(':') {
+        if spec.contains(':') {
+            // Unrouted specs come back WHOLE; routed ones come back as the part
+            // after the FIRST colon. Testing "does it still contain a colon"
+            // reads the same and is wrong: `local:ornith-fleet:9b` resolves
+            // correctly to model `ornith-fleet:9b`, which does. This probe
+            // reported a provider the server had just registered as
+            // UNREGISTERED on its first live run, which is how the same latent
+            // bug was found in `evaluator::cross_provider_judge`.
             let (_, resolved) = runtime.resolve_provider(&spec);
-            if resolved.contains(':') || resolved == spec {
+            if resolved == spec {
-                let _ = name;
                 out.push((spec.clone(), LinkStatus::Unregistered));
                 continue;
             }
         }
-        let status = match complete_or(runtime, "", "Reply with exactly: OK", &spec, 8, false).await
+        // A non-empty system prompt. Kimi rejects an empty one outright —
+        // `400 the message at position 0 with role 'system' must not be empty` —
+        // so an empty probe reported a healthy provider as BROKEN on the first
+        // live run. The probe must look like the traffic it stands in for.
+        let status = match complete_or(
+            runtime,
+            "You are a reachability probe.",
+            "Reply with exactly: OK",
+            &spec,
+            8,
+            false,
+        )
+        .await
         {
             Ok(_) => LinkStatus::Answered,
             Err(e) if is_capacity_failure(&e) => LinkStatus::Throttled(e),
@@ -502,6 +520,33 @@ mod tests {
         }
     }
 
+    /// A model name may contain a colon, and "unregistered" must not mean that.
+    ///
+    /// `resolve_provider` returns the spec unchanged when it does not recognise
+    /// the provider and the part after the FIRST colon when it does. The obvious
+    /// test — "does the model half still contain a colon" — reads the same and
+    /// is wrong the moment a model id has one. `ornith-fleet:9b` has one, and
+    /// the live preflight reported a provider the server had just registered as
+    /// UNREGISTERED. The identical bug was in `cross_provider_judge`, where it
+    /// would have refused a perfectly good independent judge.
+    #[test]
+    fn a_colon_in_the_model_name_is_not_a_missing_provider() {
+        // What `resolve_provider` returns, in both cases.
+        let routed = |spec: &str| spec.split_once(':').map(|(_, m)| m).unwrap_or(spec);
+        let unrouted = |spec: &str| spec;
+
+        for spec in ["local:ornith-fleet:9b", "glm:glm-4.7", "kimi:kimi-k2.7-code"] {
+            assert_ne!(
+                routed(spec),
+                spec,
+                "{spec} routed must not equal the whole spec"
+            );
+            assert_eq!(unrouted(spec), spec, "{spec} unrouted comes back whole");
+        }
+        // And the one that made the naive check look correct for so long.
+        assert!(routed("local:ornith-fleet:9b").contains(':'));
+    }
+
     /// A throttled link is usable; an unregistered one is not.
     ///
     /// The second is the dangerous one and the reason `preflight` checks