8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk
Whisper-LV3:
target RAVDESS CREMA-D
happy happy (0.999) ✓ happy (0.999) ✓
angry neutral (0.92) sad (0.99)
fearful happy (0.998) fearful (0.984) ✓
sad angry (0.99) fearful (0.99)
CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus
produces more class-pure fearful direction. Neither corpus solves
angry or sad — recipe shifts into 'vague expressivity' rather than
class-specific corners.
Practical: prefer CREMA-D when available; A/B both per emotion if
class precision matters.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Wires --whisper flag in converse_server using the existing whisper-rs
asr feature. AsrEngine enum (Kyutai default + Whisper variant gated on
"asr" feature) lets the receive loop branch on backend. Whisper path:
buffer audio during receive, transcribe full buffer at EOT — batch-only,
no VAD, no incremental words.
Adds examples/whisper_profile binary measuring Whisper-tiny in
isolation against the same audio used for stt_profile.
Standalone profile findings (M-series):
Kyutai STT 1B : 80.8 ms / 80 ms audio 1.01x realtime
Whisper-tiny : 209 ms / 10.43 s audio 0.020x (~50x faster)
But the full-stack bench reveals a critical regression: linking
whisper-rs's C++ runtime into the same binary as candle/CSM costs
2-3x across ALL CSM inference (recv_phase, tts_per_utterance,
total_turn) even when --whisper is NOT used. Build flag matters.
Build recv tts/u total
--features metal 4196 3113 18707
--features metal,asr (Kyutai) 10019 7803 43366 <- linkage cost
--features metal,asr +whisper 0 12921 54028 <- worse
Suspected cause: ggml/whisper.cpp's BLAS or Metal context init
conflicts with candle's. Production verdict: build WITHOUT asr
feature; accept Kyutai's 1x realtime STT cost. The standalone
whisper_profile binary still works for batch transcribe measurement.
Real Whisper integration would need a sidecar process pattern (whisper
running as a separate binary, IPC to converse_server). Documented in
the --whisper CLI help. Flag stays as opt-in with explicit warning.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
