rtx-csm: Sprint 2 Phase B — steering vector extractor

Closes the loop on Phase A (apply hook). Adds:

- LlamaModel capture buffer + start/take_capture API. Pushes one
  mean-pooled-over-seq activation per layer into a per-call Vec when
  active. Steering apply runs first, so captures reflect post-steering
  state when both are on (extractor disables steering for the duration
  of the call to capture baseline activations).
- Model::capture_backbone_activations: teacher-forced forward over a
  built prompt, returns per-layer (embed_dim,) activation tensors.
- ModelBackend passthrough; FP-only (Quantized errors out).
- examples/steering_extract: reads emotion-labeled JSONL, accumulates
  per-emotion sums on CPU f32, writes per-layer
  (mean(target) - mean(baseline)) as `layer_<i>_steering` safetensors.

Smoke run on carlini2 manifest (excited vs surprised, 10 samples each):
- Vector norms grow monotonically with depth (layer 0: 2.25, layer 15:
  12.61) — consistent with deeper layers carrying richer
  emotion/style signal.
- Loaded into examples/generate at scales 0.5/1.0/2.0; quality_eval
  shows WER hits 1.0 immediately. This is the EmoSteer paper's warning
  ("large α may produce unintelligible speech") triggering at small
  α — diagnosis: the corpus is the problem, not the infrastructure.
  The emotion_tag labels in our existing manifests are noisy
  (emotion2vec output on lecture audio collapses to [surprised] /
  [excited] without a clean neutral pool), and 10 samples per pool
  is well short of the paper's 1000/emotion.

What this validates:
- End-to-end extraction → save → load → apply pathway works.
- quality_eval (Sprint 1) cleanly catches the regression — the metric
  foundation does its job.

What's next (a future session):
- Real emotion-labeled dataset (CREMA-D, ESD, RAVDESS) for proper
  pools with a true neutral baseline.
- Layer-subset experiments (paper steers layers 1,6,11,16,21 of 32;
  for our 16-layer backbone the analogue is roughly 1, 4, 8, 12).
- Listening test alongside the metric numbers.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

crates/models/rtx-csm/src/csm_fork.rs

@@ -422,6 +422,12 @@ pub struct LlamaModel {
     /// `apply()` hook runs after each Layer's forward to add a steering
     /// vector to the residual stream. See `crate::steering`.
     steering: Option<crate::steering::LayerSteering>,
+    /// When `Some`, every forward pass pushes one mean-pooled-over-seq
+    /// activation vector per layer into this buffer. Used by
+    /// `examples/steering_extract` to derive ActAdd-style steering
+    /// vectors from emotion-labeled audio. Caller calls `start_capture`
+    /// before forward and `take_capture` after.
+    capture: Option<Vec<Tensor>>,
 }
 
 impl LlamaModel {
@@ -440,6 +446,7 @@ impl LlamaModel {
             device: vb.device().clone(),
             dtype: vb.dtype(),
             steering: None,
+            capture: None,
         })
     }
 
@@ -454,6 +461,20 @@ impl LlamaModel {
         self.steering.as_ref()
     }
 
+    /// Begin capturing per-layer mean-pooled activations on the next
+    /// `forward()` call. After `forward` returns, retrieve them via
+    /// `take_capture`. No-op if `forward` is not called between these.
+    pub fn start_capture(&mut self) {
+        self.capture = Some(Vec::with_capacity(self.layers.len()));
+    }
+
+    /// Drain captured activations and disable capture. Returns one
+    /// `(embed_dim,)` tensor per layer (in layer order) when capture was
+    /// active and a forward pass ran, else `None`.
+    pub fn take_capture(&mut self) -> Option<Vec<Tensor>> {
+        self.capture.take()
+    }
+
     pub fn clear_kv_cache(&mut self) {
         for layer in self.layers.iter_mut() {
             layer.clear_kv_cache()
@@ -493,6 +514,17 @@ impl LlamaModel {
             if let Some(steering) = self.steering.as_ref() {
                 xs = steering.apply(layer_idx, &xs)?;
             }
+            if let Some(capture) = self.capture.as_mut() {
+                // Mean over seq dim → (batch, embed_dim) → squeeze batch=1 → (embed_dim,).
+                let pooled = xs.mean(1)?;
+                let pooled = if pooled.dims().first().copied() == Some(1) {
+                    pooled.squeeze(0)?
+                } else {
+                    // batch>1: pool over batch too so we always store (embed_dim,).
+                    pooled.mean(0)?
+                };
+                capture.push(pooled);
+            }
         }
         if std::env::var("CSM_LORA_DEBUG").is_ok() {
             eprintln!("LlamaModel: post-loop xs.track_op={}", xs.track_op());
@@ -761,6 +793,31 @@ impl Model {
         self.backbone.layers.len()
     }
 
+    /// Teacher-forced forward through the backbone over a built-prompt
+    /// `(tokens, mask)` and return one mean-pooled-over-seq activation
+    /// vector per layer. Used by `examples/steering_extract` to derive
+    /// per-emotion difference-of-means steering vectors.
+    ///
+    /// Steering is disabled for the duration of this call so the captured
+    /// activations are baseline (not already-steered).
+    pub fn capture_backbone_activations(
+        &mut self,
+        tokens: &Tensor,
+        tokens_mask: &Tensor,
+    ) -> Result<Vec<Tensor>> {
+        let saved_steering = self.backbone.steering.take();
+        self.backbone.clear_kv_cache();
+        self.backbone.start_capture();
+        let embeds = self.build_embeds(tokens, tokens_mask)?;
+        let _h = self.backbone.forward(&embeds, 0)?;
+        let captured = self
+            .backbone
+            .take_capture()
+            .ok_or_else(|| crate::error::CsmError::Config("capture buffer empty".into()))?;
+        self.backbone.steering = saved_steering;
+        Ok(captured)
+    }
+
     /// Build the per-frame embedding tensor `(B, S, D)` from packed token slots.
     /// Shared by `generate_frame` and `generate_frame_cfg`.
     fn build_embeds(&self, tokens: &Tensor, tokens_mask: &Tensor) -> Result<Tensor> {