8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk
Whisper-LV3:
target RAVDESS CREMA-D
happy happy (0.999) ✓ happy (0.999) ✓
angry neutral (0.92) sad (0.99)
fearful happy (0.998) fearful (0.984) ✓
sad angry (0.99) fearful (0.99)
CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus
produces more class-pure fearful direction. Neither corpus solves
angry or sad — recipe shifts into 'vague expressivity' rather than
class-specific corners.
Practical: prefer CREMA-D when available; A/B both per emotion if
class precision matters.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Three real bugs found while running a YouTube → train → eval pipeline
end-to-end on real corpora:
1. emotion2vec was producing near-constant logits regardless of input.
Per config.yaml `normalize: true` — data2vec2/emotion2vec expects
per-utterance zero-mean unit-variance normalization on the raw
waveform before the local_encoder. Added inside the EmotionDetector
trait impl so all callers get it.
Verified empirically: 4 different audio inputs (Carlini talk,
audience question, McConaughey speech) now produce different argmax
classes. Before fix: all 4 produced identical logits.
2. The 9→5 emotion fold was collapsing every real-world clip to
[excited]. happy / surprised / other all mapped to Excited covered
~95% of natural speech. Replaced with a direct 9-class identity
mapping; EmotionLabel gained Disgusted, Fearful, Happy, Surprised,
Unk variants. Now: 132 [surprised] + 12 [excited] across the
Carlini corpus instead of 144 [excited].
3. lora_train_emotional --peak-lr / --epochs flags. The canned 3-stage
recipe over-fits on small (~100 clip) corpora at extended rank 8;
users need to tune. (The recipe stays as defaults; flags are pure
overrides.)
Plus diagnostic: examples/emotion2vec_probe — feed real audio files
into emotion2vec and dump per-class logits. Used to find bug #1.
Lib suite still 131/131 (the test that locked the 9→5 fold updated
to lock the new identity mapping).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
