rtx-csm: depth-decoder steering API

Adds set_decoder_steering on Model + Generator and
--decoder-steering-vec / --decoder-steering-scale on examples/generate.
The decoder is already a LlamaModel under the hood, so the existing
LayerSteering hook in csm_fork::Layer::forward applies as-is — only
the public surface needed wiring.

Architectural hypothesis being tested: backbone carries semantic
content (what the model says), depth decoder carries acoustic detail
(how it sounds). Backbone steering shifts character at the cost of
text fidelity (Sprint 2 finding); decoder steering should shift
prosody/timbre without disturbing word content.

Smoke test with random Gaussian decoder vectors (4 layers × 1024
embed_dim, stddev 0.1, scale 0.5):

  case      cos    WER    transcript
  baseline  0.72   1.0    "No."
  backbone  0.83   1.4    "That's for on-beat for bee..."
  decoder   0.76   1.0    "So" (premature EOT)
  both      0.81   3.0    "I'm going to go to the next one..."

Decoder steering DOES alter output (cosine 0.72 → 0.76, transcript
changes) but random vectors trigger premature EOT — same pattern as
random backbone vectors. The infrastructure works; getting the real
emotion-from-acoustic-codebooks signal needs decoder activation
capture, which the current Model::capture_backbone_activations
doesn't do (it captures the backbone forward only).

Decoder capture is the next-session item. With it we can extract
real per-emotion decoder vectors from RAVDESS and test the
hypothesis properly.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-04-29 12:39:49 -07:00
co-authored by Claude Opus 4.7
parent 5a9d37bf4f
commit a1fa72d151
3 changed files with 80 additions and 1 deletions
+22
View File
@@ -793,6 +793,28 @@ impl Model {
self.backbone.layers.len()
}
/// Install activation-steering vectors on the depth decoder. The
/// decoder generates the acoustic codebooks (c1..N-1) given a
/// sampled c0 + the backbone hidden state — so steering here should
/// affect prosody/timbre without changing word-level content. The
/// decoder for CSM-1B is Llama100M (4 layers × 1024 embed_dim);
/// vectors must match that shape.
pub fn set_decoder_steering(&mut self, steering: Option<crate::steering::LayerSteering>) {
self.decoder.set_steering(steering);
}
pub fn decoder_num_layers(&self) -> usize {
self.decoder.layers.len()
}
pub fn decoder_embed_dim(&self) -> usize {
LlamaConfig::from_flavor(self.config.decoder_flavor).embed_dim
}
pub fn backbone_embed_dim(&self) -> usize {
LlamaConfig::from_flavor(self.config.backbone_flavor).embed_dim
}
/// Teacher-forced forward through the backbone over a built-prompt
/// `(tokens, mask)` and return one mean-pooled-over-seq activation
/// vector per layer. Used by `examples/steering_extract` to derive