8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk
Whisper-LV3:
target RAVDESS CREMA-D
happy happy (0.999) ✓ happy (0.999) ✓
angry neutral (0.92) sad (0.99)
fearful happy (0.998) fearful (0.984) ✓
sad angry (0.99) fearful (0.99)
CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus
produces more class-pure fearful direction. Neither corpus solves
angry or sad — recipe shifts into 'vague expressivity' rather than
class-specific corners.
Practical: prefer CREMA-D when available; A/B both per emotion if
class precision matters.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Phase 8.1.3 deferred Silero V5 VAD because the only Rust crate path
(`voice_activity_detector` via `ort`) collides with `sentencepiece-sys`
on protobuf 3.14 vs 3.21 and panics at process startup. This commit
closes that gap with a NATIVE candle port.
Direct port from `Snakers4/silero-vad/src/silero_vad/tinygrad_model.py`
(71 LOC reference). Architecture:
stft_conv Conv1d(1, 258, k=256, s=128) no bias
conv1 Conv1d(129, 128, k=3, p=1)
conv2 Conv1d(128, 64, k=3, s=2, p=1)
conv3 Conv1d(64, 64, k=3, s=2, p=1)
conv4 Conv1d(64, 128, k=3, p=1)
lstm_cell LSTMCell(128, 128)
final_conv Conv1d(128, 1, k=1)
Forward: reflect-pad input by 64, STFT-as-conv1d, sqrt(real² + imag²),
4-layer Conv1d feature stack with ReLU, single LSTM step (state across
chunks), 1x1 conv + sigmoid -> speech probability.
Files added:
src/silero_vad.rs ~310 LOC (incl. LSTM cell + downloader)
docs/silero_vad_port_notes.md architecture + port plan
examples/silero_vad_smoke.rs real-audio discrimination test
Plus a new `ureq` direct dep (transport already pulled in via hf-hub).
Weights ship via download-on-first-run from the upstream GitHub raw
URL into `~/.cache/rtx-csm/silero_vad_16k.safetensors` (1.24 MB). No
repo bloat; no .gitignore wrestling.
End-to-end smoke (synthetic 50/50 silence/speech WAV at 16 kHz):
load (cold): download + parse, < 100 ms after first run
VAD sweep: 170 ms over 9.99 s of audio = 0.017x realtime (59x faster)
unit test: passes (load weights + run one step)
Probability output (per 32 ms chunk):
0-1.5 s: p ~ 0.01-0.07 silence
1.5-5 s: p ~ 1.000 speech (clean ramp at speech onset)
5-10 s: p ~ 0.001 silence
Speech-chunk fraction 33% on the 50/50 layout — matches expected.
Production angle: dramatically better silence/speech discrimination
than the Phase 8.1.3b energy VAD (which only catches obvious silence).
Silero V5 catches whisper-quiet speech, breath/lip noise, music vs
speech distinction. Drop-in candidate for `--vad-gate` in a future
iteration.
The ort/protobuf conflict that blocked this for two months is now
permanently resolved by NOT using ort.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
