8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk
Whisper-LV3:
target RAVDESS CREMA-D
happy happy (0.999) ✓ happy (0.999) ✓
angry neutral (0.92) sad (0.99)
fearful happy (0.998) fearful (0.984) ✓
sad angry (0.99) fearful (0.99)
CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus
produces more class-pure fearful direction. Neither corpus solves
angry or sad — recipe shifts into 'vague expressivity' rather than
class-specific corners.
Practical: prefer CREMA-D when available; A/B both per emotion if
class precision matters.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Composable orchestrator stitching the LLM-output side of the
conversational stack:
prompt + history -> LlmClient.generate_stream -> sentence buffer
-> per-sentence Generator.generate -> post-process -> watermark
-> Vec<Utterance> stream of (text, audio, latency)
CSM is sentence-level (best prosody on full sentences), so the buffer
flushes when terminal punctuation appears anywhere in the buffer
(Punctuation policy: . ! ? \n) or at any clause boundary
(Eager policy: + , ; :).
src/converse.rs:
- Converse<L: LlmClient> orchestrator
- Utterance { text, audio, tts_latency_ms } per sentence
- FlushPolicy { Punctuation, Eager }
- find_first_boundary scans the whole buffer (not just the last char)
so "Sentence one. Word two" emits "Sentence one." immediately rather
than waiting for the next terminal mark
- 3 unit tests for boundary detection + policy modes
examples/converse.rs:
- --mock mode: hardcoded 20-token "sleepy turtle" stream, no API key
- live mode: any OpenAI-compatible endpoint via OpenAiCompatibleClient
- writes the concatenated audio to a single WAV
Verified mock end-to-end on Metal: 2 utterances emitted as expected
(sentence 1 hits max_audio_ms cap at 6s; sentence 2 EOTs naturally at
4.88s), total 10.88s of audio in 31.5s wall-clock.
Phase 6c.1 ships the half-duplex (text-in -> voice-out) pipeline. Full
duplex (audio-in -> voice-out) is 6c.2, blocked on 6a's STT word
emission landing.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
