rtx-csm: emotional_speech.sh — per-emotion scale defaults

Capstone wrapper for the Phase 9 recipe (Selective CFG + RAVDESS
steering + mid-layer subset). Picks the steering scale automatically
from an empirical per-emotion map:

  - happy:    0.30
  - angry:    0.20
  - fearful:  0.20
  - sad:      0.20 (note: sad is unreliable — see below)

These came from a follow-up sweep after the multi-emotion demo
revealed the recipe is emotion-sensitive: scale 0.3 works for happy
("The police are, if you're, I can't recite this film") but pushes
angry / fearful past the speech manifold (Mimi emits non-speech /
music tokens, Moonshine transcribes as 🎵). Dropping to 0.2 recovers
fluent speech for both:

  - [email protected]:    "The next disorder is completing kashim for more."
  - [email protected]:  "You just heard a little bit about this decision,
                   though."
  - [email protected]:    "The police are, if you're, I can't recite this
                   film. I"

Sad is the outlier — model resists "sad" steering at every scale
between 0.15 and 0.3. Likely a corpus issue (sad RAVDESS clips are
the lowest-energy subset). Documented as a known limitation rather
than worked around.

perf_history.md updated with the per-emotion sensitivity finding.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

crates/models/rtx-csm/docs/perf_history.md

@@ -136,6 +136,15 @@ shipped on the FP backbone; quantized path unchanged.
   ramping…") but causes premature EOT on generic prompts ("Today I
   want to talk about…"). Treat the schedule as one tool to bench per
   use case, not a global default.
+- *Steering scale is **emotion-sensitive** at the same recipe*: the
+  scale=0.3 default that works for `happy` pushes `angry` and
+  `fearful` past the speech manifold (Mimi emits non-speech / music
+  tokens; Moonshine transcribes as 🎵). Empirical sweet spots:
+  `happy` 0.3, `angry` 0.2, `fearful` 0.2, `sad` ≈ no robust scale
+  found (the model resists "sad" steering — produces one-word
+  outputs at every tested 0.15–0.3). The wrapper script
+  `emotional_speech.sh` picks these per-emotion defaults
+  automatically.
 - *RAVDESS corpus shape biases output length*: clips are ~3 s
   ("Kids are talking by the door") so the steering biases the
   model toward terse outputs (one or two words). Longer-utterance

crates/models/rtx-csm/scripts/emotional_speech.sh

@@ -0,0 +1,130 @@
+#!/usr/bin/env bash
+# One-shot emotional voice synthesis using the Phase 9 recipe:
+#   - Selective CFG schedule `linear:3.0:1.0:25` (robust default)
+#   - Activation steering @ scale 0.3 on mid layers [8, 10, 12]
+#   - Reference-clip context conditioning
+#
+# This is the codified version of the empirical findings logged in
+# `docs/perf_history.md` Phase 9. Recipe was tuned for cross-character
+# emotional speech (e.g. take a neutral lecture voice, render it angry).
+#
+# Steering vectors must be pre-extracted via
+# `examples/steering_extract` from a labeled emotion corpus (RAVDESS,
+# CREMA-D, ESD, etc). Use `scripts/build_ravdess_manifest.sh` to
+# convert RAVDESS into a manifest first.
+#
+# Usage:
+#   scripts/emotional_speech.sh \
+#       --text "What you want said." \
+#       --context-wav <reference voice>.wav \
+#       --context-text "Transcript of the reference clip." \
+#       --emotion angry \
+#       --steering-dir /tmp/ravdess_steering \
+#       --out /tmp/out.wav \
+#       [--scale 0.3]                 # steering scale, lower = more text fidelity
+#       [--cfg-schedule linear:3.0:1.0:25]
+#       [--layers 8,10,12]
+#
+# Recipe defaults are deliberate; override only when you have a reason.
+# Sprint 2 measurements:
+#   - speaker_cosine vs angry-actor reference rises 0.55 → 0.846 with
+#     scale=0.3 stacked on Selective CFG
+#   - scale ≥ 0.5 corrupts the output (Moonshine WER 1.0 on Unicode noise)
+#   - layers including 0–7 also corrupt (input embedding fidelity loss)
+#
+# Per-emotion scale sweet spots (RAVDESS-extracted, mid-layer recipe,
+# linear:3.0:1.0:25 CFG, single-prompt single-seed measurement — treat
+# as starting points, not guarantees):
+#   - happy:    0.30 → fluent speech
+#   - angry:    0.20 → fluent speech (0.30 produces music-token outputs)
+#   - fearful:  0.20 → fluent speech (0.30 produces music-token outputs)
+#   - sad:      no robust scale found — model resists "sad" steering on
+#               most prompts; one-word outputs at all tested scales
+# We leave --scale defaulted to 0.3 because the wrapper can't know which
+# emotion is being requested at default-construction time. Override
+# with --scale 0.2 when using angry or fearful.
+
+set -euo pipefail
+
+TEXT=""
+CTX_WAV=""
+CTX_TEXT=""
+EMOTION=""
+STEERING_DIR=""
+OUT=""
+SCALE=""  # if unset, picked per-emotion below from the empirical map
+CFG_SCHEDULE="linear:3.0:1.0:25"
+LAYERS="8,10,12"
+SPEAKER=0
+SEED=42
+
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        --text) TEXT="$2"; shift 2 ;;
+        --context-wav) CTX_WAV="$2"; shift 2 ;;
+        --context-text) CTX_TEXT="$2"; shift 2 ;;
+        --emotion) EMOTION="$2"; shift 2 ;;
+        --steering-dir) STEERING_DIR="$2"; shift 2 ;;
+        --out) OUT="$2"; shift 2 ;;
+        --scale) SCALE="$2"; shift 2 ;;
+        --cfg-schedule) CFG_SCHEDULE="$2"; shift 2 ;;
+        --layers) LAYERS="$2"; shift 2 ;;
+        --speaker) SPEAKER="$2"; shift 2 ;;
+        --seed) SEED="$2"; shift 2 ;;
+        -h|--help) sed -n '2,32p' "$0"; exit 0 ;;
+        *) echo "unknown arg: $1" >&2; exit 1 ;;
+    esac
+done
+
+for v in TEXT CTX_WAV CTX_TEXT EMOTION STEERING_DIR OUT; do
+    if [[ -z "${!v}" ]]; then
+        echo "error: --${v,,} (or one of the required flags) is unset" >&2
+        echo "usage: $0 --text T --context-wav W --context-text C --emotion E --steering-dir D --out O" >&2
+        exit 1
+    fi
+done
+
+STEERING_FILE="$STEERING_DIR/${EMOTION}.safetensors"
+if [[ ! -f "$STEERING_FILE" ]]; then
+    echo "error: no steering file for emotion '$EMOTION' at $STEERING_FILE" >&2
+    echo "available emotions in $STEERING_DIR:" >&2
+    ls "$STEERING_DIR" 2>/dev/null | sed 's/\.safetensors$//' | sed 's/^/  /' >&2 || true
+    exit 1
+fi
+
+# Per-emotion scale defaults (Sprint 2 sweet-spot measurements).
+if [[ -z "$SCALE" ]]; then
+    case "$EMOTION" in
+        angry|fearful) SCALE=0.2 ;;
+        happy)         SCALE=0.3 ;;
+        sad)           SCALE=0.2 ;;  # rough fallback; sad isn't reliable
+        *)             SCALE=0.3 ;;  # generic default
+    esac
+fi
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+WORKSPACE_DIR="$(cd "$SCRIPT_DIR/../../../.." && pwd)"
+GEN_BIN="$WORKSPACE_DIR/target/release/examples/generate"
+if [[ ! -x "$GEN_BIN" ]]; then
+    echo "error: missing $GEN_BIN" >&2
+    echo "build with: cargo build -p rtx-csm --release --example generate" >&2
+    exit 1
+fi
+
+echo "→ recipe: emotion=$EMOTION scale=$SCALE cfg=$CFG_SCHEDULE layers=$LAYERS"
+echo "  context: $(basename "$CTX_WAV")"
+
+"$GEN_BIN" \
+    --text "$TEXT" \
+    --speaker "$SPEAKER" \
+    --seed "$SEED" \
+    --context-wav "$CTX_WAV" \
+    --context-text "$CTX_TEXT" \
+    --context-speaker 1 \
+    --enable-cfg --cfg-schedule "$CFG_SCHEDULE" \
+    --steering-vec "$STEERING_FILE" \
+    --steering-scale "$SCALE" \
+    --steering-layers "$LAYERS" \
+    --out "$OUT"
+
+echo "✓ wrote $OUT"