Wires emotion2vec into quality_eval so per-row metrics include
target_emotion_prob, top_emotion, top_emotion_prob. Pure inference,
optional via --emotion2vec / --target-emotion flags.
Critical empirical finding documented in code + user guide: the
emotion2vec_plus_base checkpoint classifies every input as
"Surprised" with prob ≥ 0.99, INCLUDING ground-truth RAVDESS clips
with explicit emotion labels. Real angry-RAVDESS → "Surprised"
(0.9999999). Real neutral-RAVDESS → "Surprised" (0.9999996).
The metric implementation is correct (matches the trait's
EmotionDetector::classify code path with same per-utterance zero-
mean unit-variance normalization); the underlying classifier
collapses to a dominant class on most input — likely the same
"9→5 fold collapse" the project already documented in the data-
labeling path.
Practical implication: target_emotion_prob is near-zero for almost
every (target, output) that isn't "surprised", so it can't be used
as a picker score. The emotion2vec metric still works as a
diagnostic ("did the model produce something that classifies as
audio at all?") but not as a generation-quality validator.
Doc'd in:
- examples/quality_eval.rs CLI doc (caveat block on --emotion2vec)
- docs/emotional_speech_guide.md (Known limitations section with
full sanity-check table)
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Brief addition to the recipe section explaining the WER + length
floor scoring used by emotional_speech_n.sh (committed in b7b267b).
Validates that the new scoring preserves canonical winners on
happy and calm while flipping surprised to the long-and-correct
candidate.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
3-seed picker run (7, 42, 100) on calm/disgust/surprised refines the
single-shot characterization:
calm: winner seed=100, WER 0.64
"It's a good reflection. Not that that. I want to
share somet..."
(recipe lands the prompt — single-shot at seed 42 only
produced hesitation markers; the picker found a seed
with actual content)
disgust: no reliable seed
(all 3 seeds WER ≥ 0.93; likely RAVDESS corpus issue —
disgust clips are low-energy / acoustically close to
neutral. Try CREMA-D or ESD for this emotion.)
surprised: picker chose seed=7 (WER 0.93, short "You can.") over
seed=100 (WER 1.0, "...Today I want to share something")
— WER weighting issue: deletions and insertions count
uniformly, so terse-but-mostly-wrong beat long-and-
mostly-right. Manual selection or weighting WER less
heavily would help here.
Updated per-emotion table marks disgust as ✗ (corpus limitation),
surprised as ⚠ (picker scoring artifact), calm as ✓ (works with
N-seed picker).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Extracted decoder steering vectors for the remaining 3 RAVDESS
emotions (calm, disgust, surprised). Single-seed bench at
seed=42 on Amini context, decoder route, recipe defaults:
emotion cos_ctx WER transcript
calm 0.70 0.86 "I'm sorry. Um, I don't know."
(natural hesitation markers — the
recipe produces semantically-emotion-
matched content, not just acoustic
shift)
disgust 0.81 0.93 "For that, that..." (truncated)
surprised 0.95 2.57 "Too couple, sorry, and that's saying,
even a premier and super driver..."
(long rambling; voice migrates well,
text drifts)
All 7 RAVDESS emotions now produce coherent English on the decoder
route — calm is solid first-shot, disgust truncates, surprised
rambles. Roll N seeds via emotional_speech_n.sh for the latter two.
emotional_speech_guide.md updated with the per-emotion table now
covering all 7. Voice character preservation (cos vs context > 0.7)
holds for every emotion in the pack.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Phase 9 produced 25 commits and a complex pipeline; perf_history is
the engineering log but new users coming to this cold need a clean
"how do I make CSM speak with emotion" handbook.
Sections:
- What this gets you (single-shot WER 0.07–0.21, voice cosine ≥ 0.95)
- One-liner quickstart (RAVDESS download → extract → use)
- The recipe explained — every flag and why it's there
- Per-emotion notes (works/best-seed/caveats per emotion)
- When it works / when it doesn't
- Troubleshooting (music tokens, premature EOT, repetition, etc.)
- Architecture cheat sheet (backbone=semantic, decoder=acoustic)
References perf_history.md for the full empirical log; this doc is
the user-facing distillation.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
