Commit Graph
1 Commits
Author SHA1 Message Date
osobhandClaude Opus 4.7 808e79dc40 rtx-csm: RAVDESS-derived steering — first real emotion signal
Adds a clean labeled-emotion data path now that the auto-tagged
manifests have proven inadequate for steering extraction.

scripts/build_ravdess_manifest.sh: parses the RAVDESS speech-only
distribution (Audio_Speech_Actors_01-24.zip from Zenodo, 208 MB) into
a manifest.jsonl with proper {neutral, calm, happy, sad, angry,
fearful, disgust, surprised} labels and the two canonical statements
("Kids are talking by the door", "Dogs are sitting by the door"). 1440
clips, balanced 192/emotion (96 neutral — RAVDESS lacks the 'strong'
intensity for neutral).

examples/steering_extract Mimi reload-every-10: the streaming state
counter overflows 8192 frames after ~80 encodes even with
reset_state(). Same fix training/audio_to_manifest/converse_server use
(commit 0568dd3); now applied here too.

A/B with angry-vs-neutral steering @ 50 samples per pool, matched
against an angry RAVDESS reference clip:

  case        cos    WER   transcript
  baseline    0.55   0.92  "It is a very important thing to do."
  [email protected]   0.61   1.00  "You"
  [email protected]     0.74   1.00  "So"          ← best speaker_cosine
  [email protected]     0.62   1.00  "You"
  [email protected]     0.60   1.00  "You"
  [email protected]    0.64   6.15  "the Lord, the Lord, the Lord..."

Speaker cosine 0.55 → 0.74 with mid-layer steering at scale 0.5 — a
35 % jump, the largest empirical gain we've measured. The model is
clearly migrating toward the angry actor's voice character. Side
effect: premature EOT (output reduces to one or two words). Likely
because RAVDESS clips themselves are very short ("Kids are talking
by the door", ~3s) so the steering biases toward terse outputs.
That's a corpus-shape artifact, not a code bug — different emotion
corpora with longer utterances should fix it.

Tightest single result of Sprint 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-29 10:33:18 -07:00