rustytorch

451 Commits 1 Branch 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
osobhandClaude Opus 4.7	a5cedfb46a	rtx-csm: emotional_speech_guide — CREMA-D vs RAVDESS firdhokk verdict 8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk Whisper-LV3: target RAVDESS CREMA-D happy happy (0.999) ✓ happy (0.999) ✓ angry neutral (0.92) sad (0.99) fearful happy (0.998) fearful (0.984) ✓ sad angry (0.99) fearful (0.99) CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus produces more class-pure fearful direction. Neither corpus solves angry or sad — recipe shifts into 'vague expressivity' rather than class-specific corners. Practical: prefer CREMA-D when available; A/B both per emotion if class precision matters. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>	2026-04-30 00:01:02 -07:00
osobhandClaude Opus 4.7	1bb400a886	rtx-csm: speaker_sim::WavLmSimilarity is now production Wraps the WavLM-SV port from Phase 5c. Full 100M-param X-vector head running in-process; previously returned a typed error. - WavLmSimilarity::load(path, device) loads converted safetensors. - WavLmSimilarity::embed(samples) caches a 512-d embedding for repeat comparisons. - score(a, b) embeds both inputs and cosines them. - Module docs updated; SpectralCentroidSimilarity kept as a weak-baseline check. Caller-facing change: any code using the SpeakerSimilarity trait now gets a real speaker model with one constructor swap. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>	2026-04-25 19:59:10 -07:00
osobhandClaude Opus 4.7	15dd3575d4	Add rtx-csm: Rust-native port of Sesame CSM-1B with LoRA voice cloning A new model crate at crates/models/rtx-csm implementing end-to-end inference, quantization, and fine-tuning for Sesame's Conversational Speech Model (CSM-1B). Built on candle 0.9 + Kyutai Mimi codec. Key capabilities: - Inference (FP F16 on Metal, F32 on CPU, BF16 on CUDA) - Quantized inference (Q8_0 / Q4_K_M GGUF, ~3x speedup, ~50% memory) - Streaming Mimi decode with proper StreamTensor state machine - In-context voice cloning via SpeakerProfile - Classifier-Free Guidance (Koel-TTS recipe) - Long-form chunked generation with rolling context - Audio post-processing (HPF + declick + EBU R128 LUFS) - Text input normalization (brackets, times, unicode, length caps) - Frame-level repetition guard (loop-escape) - Top-k + top-p sampling - LoRA fine-tuning end-to-end (training + inference, on FP and Q8 bases) - In-process Whisper ASR via whisper-rs (under --features asr) - Standalone TTS HTTP server (Axum) - Bench harness with manifest export + per-prompt WER Phases delivered: quantization, ASR/WER eval, LoRA voice cloning, HTTP service. AudioSeal/WavLM/Unmute remain as documented future work. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>	2026-04-25 18:33:57 -07:00

Author

SHA1

Message

Date

osobhandClaude Opus 4.7

a5cedfb46a

rtx-csm: emotional_speech_guide — CREMA-D vs RAVDESS firdhokk verdict

8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk
Whisper-LV3:

  target    RAVDESS              CREMA-D
  happy     happy (0.999) ✓      happy (0.999) ✓
  angry     neutral (0.92)       sad (0.99)
  fearful   happy (0.998)        fearful (0.984) ✓
  sad       angry (0.99)         fearful (0.99)

CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus
produces more class-pure fearful direction. Neither corpus solves
angry or sad — recipe shifts into 'vague expressivity' rather than
class-specific corners.

Practical: prefer CREMA-D when available; A/B both per emotion if
class precision matters.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

2026-04-30 00:01:02 -07:00

osobhandClaude Opus 4.7

1bb400a886

rtx-csm: speaker_sim::WavLmSimilarity is now production

Wraps the WavLM-SV port from Phase 5c. Full 100M-param X-vector head
running in-process; previously returned a typed error.

- WavLmSimilarity::load(path, device) loads converted safetensors.
- WavLmSimilarity::embed(samples) caches a 512-d embedding for repeat
  comparisons.
- score(a, b) embeds both inputs and cosines them.
- Module docs updated; SpectralCentroidSimilarity kept as a weak-baseline
  check.

Caller-facing change: any code using the SpeakerSimilarity trait now
gets a real speaker model with one constructor swap.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

2026-04-25 19:59:10 -07:00

osobhandClaude Opus 4.7

15dd3575d4

Add rtx-csm: Rust-native port of Sesame CSM-1B with LoRA voice cloning

A new model crate at crates/models/rtx-csm implementing end-to-end
inference, quantization, and fine-tuning for Sesame's Conversational
Speech Model (CSM-1B). Built on candle 0.9 + Kyutai Mimi codec.

Key capabilities:
- Inference (FP F16 on Metal, F32 on CPU, BF16 on CUDA)
- Quantized inference (Q8_0 / Q4_K_M GGUF, ~3x speedup, ~50% memory)
- Streaming Mimi decode with proper StreamTensor state machine
- In-context voice cloning via SpeakerProfile
- Classifier-Free Guidance (Koel-TTS recipe)
- Long-form chunked generation with rolling context
- Audio post-processing (HPF + declick + EBU R128 LUFS)
- Text input normalization (brackets, times, unicode, length caps)
- Frame-level repetition guard (loop-escape)
- Top-k + top-p sampling
- LoRA fine-tuning end-to-end (training + inference, on FP and Q8 bases)
- In-process Whisper ASR via whisper-rs (under --features asr)
- Standalone TTS HTTP server (Axum)
- Bench harness with manifest export + per-prompt WER

Phases delivered: quantization, ASR/WER eval, LoRA voice cloning, HTTP
service. AudioSeal/WavLM/Unmute remain as documented future work.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

2026-04-25 18:33:57 -07:00