New examples/stt_profile binary. Loads kyutai/stt-1b-en_fr, feeds PCM
in fixed-size chunks, reports per-call latency p50/p95/min/max plus
realtime factor.
Findings (M-series Metal, 10.43s LibriSpeech in):
frame_batch=1 (80ms): mean=80.8ms p50=81.7ms RT=1.01x
frame_batch=3 (240ms): mean=308.6ms p50=298ms RT=1.29x
Headline: Kyutai STT 1B on Metal saturates at ~1.0x real-time. There
is no slack in the existing model on this hardware. Per-call overhead
amortizes poorly when batching frames (3 frames takes 3.8x single
frame, not 3x). To go faster requires a smaller model (Whisper-tiny
via the existing whisper-rs feature) or a Kyutai variant if available.
Note: converse_server's measured recv_phase (~4-5s for 10.4s audio)
is faster than this profile predicts (~10s). Discrepancy not yet
resolved but the optimization conclusion stands: STT model swap is
the only lever for the receive phase.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
