feat(decide): cm-decide — typed calibrated decisions; Jev + local NLI backends; skill triage in shadow

A third kind of decision-maker between deterministic code and a full LLM
call: Choice / Score / Noul questions answered as probability
distributions with a confidence, behind one Decider trait, with the
composition patterns (confidence gating, composite scoring, rerank) as
code. Two backends: TypeSafe's Jev over HTTP, and a DeBERTa-v3 MNLI
cross-encoder run in-process with candle (feature nli; metal/cuda).

decide-eval measures a backend on labelled cases the way judge-eval
measures the judge. eval/skill-triage.json: 20 mission tasks × 53 skills,
75 positives, hand-labelled. Measured 2026-09-21:

  lexical overlap        AUROC 0.851  [email protected] 0.47  top-k 48/75  ECE 0.095
  jev (named wording)    AUROC 0.989  [email protected] 0.84  top-k 63/75  ECE 0.064  213 ms
  jev (plain wording)    AUROC 0.970  [email protected] 0.66  top-k 52/75
  nli mnli-base          AUROC 0.790  [email protected] 0.28  top-k 38/75  ECE 0.263  1.5 s
  nli zeroshot-v2        AUROC 0.782  [email protected] 0.43  top-k 39/75  ECE 0.054  1.2 s

The vendor's calibration claim survives our data; the local cross-encoder
ranks below keyword overlap on either checkpoint or wording and is kept
as the measured negative, not shipped. A local backend would need the
logit-readout route over the fleet's 9B model — a separate spike.

Shadow: one Jev call per phase launch (spawned, 10 s cap, silent without
TYPESAFE_API_KEY) records a skill.triage event; the Skill-Use report
carries triage_p beside each skill's Trigger verdict. It selects nothing.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz

crates/cm-api/src/skill_use.rs

@@ -99,6 +99,13 @@ pub struct SkillUse {
     pub trigger: Verdict,
     pub compliance: Verdict,
     pub boundary: Verdict,
+    /// What the shadow triage said BEFORE the phase ran: the probability
+    /// that this skill applies to the task (`skill_triage`). `None` when no
+    /// triage was recorded. Read next to `trigger`: a high probability with a
+    /// skipped skill is a miss by the agent or by the oracle, and only real
+    /// missions say which.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub triage_p: Option<f64>,
 }
 
 /// The skills a prompt actually delivered.
@@ -305,6 +312,7 @@ pub fn score(
                 compliance,
                 boundary,
                 skill,
+                triage_p: None,
             }
         })
         .collect()
@@ -1120,14 +1128,19 @@ pub async fn score_mission(
     .into_iter()
     .collect();
 
-    Ok(score(&prompts, &Evidence::new(&outputs, &tools), &|name| {
+    let triage = crate::skill_triage::recorded(pool, mission_id).await;
+    let mut scores = score(&prompts, &Evidence::new(&outputs, &tools), &|name| {
         kinds
             .get(name)
             .cloned()
             // A skill in a prompt with no catalogue row was delivered and then
             // deleted. Naming that explicitly beats defaulting it to builtin.
             .unwrap_or_else(|| "unknown (no catalogue row)".to_string())
-    }))
+    });
+    for s in &mut scores {
+        s.triage_p = triage.get(&s.skill).copied();
+    }
+    Ok(scores)
 }
 
 #[cfg(test)]