Original plan: gate STT work via Silero V5 VAD using
voice_activity_detector / ort to skip silent chunks. Phase 8.1.2's
ort_conflict_probe initially passed (no Metal regression). But on
first server boot, sentencepiece-sys's protobuf 3.14 collides with
ort's bundled protobuf 3.21 at process startup:
[libprotobuf FATAL ...] This program was compiled against version
3.14.0 of the Protocol Buffer runtime library, which is not
compatible with the installed version (3.21.12) ... in
sentencepiece-sys-0.13.1
Updated the probe binary to also load Kyutai STT (which links
sentencepiece) — now correctly catches the conflict at probe time.
Same risk class as the whisper-rs/ggml issue from Phase 7.6: external
ML runtimes don't always coexist in the same Rust process.
Pivoted to a pure-Rust energy-based VAD: RMS amplitude threshold per
incoming chunk. ~30 LOC, no external runtime, no protobuf risk.
Catches obvious silence (room tone, pauses) — misses quiet speech
(whispering). Acceptable for typical mic-distance voice loops.
Wired into examples/converse_server as `--vad-gate` /
`--vad-gate-threshold 0.01`. The receive loop checks RMS before each
binary chunk's STT call; silence chunks skip step_pcm entirely.
Orthogonal to existing `--vad` (Kyutai semantic-VAD-for-EOT).
Bench (mock LLM, Q8+stream, 10s LibriSpeech in, ~90% speech):
baseline: recv_phase = 4834 ms
--vad-gate: recv_phase = 4509 ms (-7%, matches the ~10% silence
in this audio)
Real-world voice-agent audio is 30-50% silence; expect proportional
savings (~30-50% recv_phase reduction). Larger silence content =
larger VAD win.
Cargo.toml: keeps the optional `vad` feature + voice_activity_detector
dep wired (path remains for a future Silero-via-candle port). The
ort_conflict_probe example continues to require `--features vad` so
the conflict-detection path stays exercised.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Adds optional `vad` feature pulling voice_activity_detector v0.2 (Silero
V5 via the `ort` ONNX runtime). Gates wiring VAD into converse_server
on a regression check: does linking ort into the same binary as candle
slow CSM Metal inference the way whisper-rs (ggml) did?
examples/ort_conflict_probe.rs times 10 CSM Q8 forwards, loads ort +
runs Silero V5 a few times, then times 10 more forwards. Compares.
Result on M-series Metal:
before ort load: 645.8 ms mean
after ort load: 633.2 ms mean
ratio: 0.981 (-1.9%, within noise threshold ±5%)
PASS — ort coexists cleanly with candle/Metal. The whisper-rs/ggml
regression doesn't generalize to all C++ ML runtimes; ort's Metal
backend (via WebGPU EP) doesn't appear to fight with candle's. Safe
to ship Silero-VAD gating in Phase 8.1.3.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
