rustyverse/rustytorch
2
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

rtx-csm: emotional_speech_guide — N-seed bench for new 3 emotions

Browse Source 
3-seed picker run (7, 42, 100) on calm/disgust/surprised refines the
single-shot characterization:

  calm:      winner seed=100, WER 0.64
             "It's a good reflection. Not that that. I want to
              share somet..."
             (recipe lands the prompt — single-shot at seed 42 only
              produced hesitation markers; the picker found a seed
              with actual content)

  disgust:   no reliable seed
             (all 3 seeds WER ≥ 0.93; likely RAVDESS corpus issue —
              disgust clips are low-energy / acoustically close to
              neutral. Try CREMA-D or ESD for this emotion.)

  surprised: picker chose seed=7 (WER 0.93, short "You can.") over
             seed=100 (WER 1.0, "...Today I want to share something")
             — WER weighting issue: deletions and insertions count
             uniformly, so terse-but-mostly-wrong beat long-and-
             mostly-right. Manual selection or weighting WER less
             heavily would help here.

Updated per-emotion table marks disgust as ✗ (corpus limitation),
surprised as ⚠ (picker scoring artifact), calm as ✓ (works with
N-seed picker).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-04-29 19:45:15 -07:00
co-authored by Claude Opus 4.7
parent e6f8d77328
commit 8190b26e93
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/models/rtx-csm/docs/emotional_speech_guide.md
+3 -3
View File
@@ -99,9 +99,9 @@ seeds rolled = 5
| angry | ✓ | 100 | Voice migrates well; text drift on some prompts (URL spam observed once). | | angry | ✓ | 100 | Voice migrates well; text drift on some prompts (URL spam observed once). |
| fearful | ✓ | 7 | Stable on Amini context. | | fearful | ✓ | 7 | Stable on Amini context. |
| sad | ✓ | 7 | Works on decoder route only. **Failed on the backbone route** at every scale we tested 0.15–0.3. The architectural switch was the unlock. | | sad | ✓ | 7 | Works on decoder route only. **Failed on the backbone route** at every scale we tested 0.15–0.3. The architectural switch was the unlock. |
| calm | ✓ | 42 (1-seed) | First bench produced "I'm sorry. Um, I don't know." (WER 0.86) — natural hesitation markers semantically match the emotion. Roll N seeds for fidelity. | | calm | ✓ | 100 | 3-seed picker produced "It's a good reflection. Not that that. I want to share somet..." (WER 0.64) — recipe lands prompt content with hesitation/reflection prosody. Single-shot at seed 42 gave hesitation markers ("I'm sorry. Um, I don't know."). |
| disgust | ⚠ | 42 (1-seed) | First bench produced "For that, that..." (WER 0.93, truncated). Probably needs different scale or seed. | | disgust | ✗ | (none reliable) | All 3 picker seeds (7, 42, 100) produced WER ≥ 0.93. Recipe doesn't work for this emotion on RAVDESS — likely corpus issue (RAVDESS disgust clips are low-energy / acoustically close to neutral). Try with CREMA-D or ESD. |
| surprised | ⚠ | 42 (1-seed) | First bench produced fluent but rambling output (cos 0.95, WER 2.57). Voice migrates well; text drifts. Consider N-seed sampling. | | surprised | ⚠ | 100 (manual pick) | Picker chose seed=7 ("You can.", WER 0.93). Seed 100 produced "...Today I want to share something" but WER scored higher because of prefix tokens — picker's WER metric weights deletions/insertions uniformly. Manual seed selection (or weighting WER less heavily) recommended for this emotion. |
The per-emotion best seed is **context-specific**. Seed 42 wins for happy on The per-emotion best seed is **context-specific**. Seed 42 wins for happy on
both Amini and Carlini contexts, but seed 7 wins for sad/fearful on Amini both Amini and Carlini contexts, but seed 7 wins for sad/fearful on Amini