rustyverse/rustytorch
2
0
Fork 0
Code Issues Pull Requests Actions 41 Packages Projects Releases Wiki Activity
502 Commits 1 Branch 0 Tags
fdfb6da76918c59be45d04fa465c6904e41baf9a
Commit Graph
3 Commits
Author SHA1 Message Date
osobhandClaude Opus 4.7 9161b32a91 rtx-csm: per-request watermark message in tts_server 
Watermarker trait gains embed_with_message(audio, message) with a
default impl forwarding to embed (no-op for watermarkers without a
payload). AudioSealWatermarker overrides to use the requested message
instead of self.message; ResampledWatermarker forwards through the
resample dance.

TtsRequest gains optional watermark_message: Option<String> (decimal or
0xHEX). Useful for clawsample to tag each generation with a unique ID
(e.g. job_id mod 0x10000) for audit trails. When omitted, falls back
to the server-startup --audioseal-message default.

Verified end-to-end: override "0xBEEF" -> detect 0xBEEF (mean_presence
0.9995, 16/16 bits). Default fallback also decodes correctly.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
 2026-04-26 04:01:30 -07:00
osobhandClaude Opus 4.7 63979eab45 rtx-csm: Generator inline watermarker + ResampledWatermarker adapter 
A single \`generate\` invocation now produces a watermarked WAV when
AudioSeal weights are passed via CLI. End-to-end verified on real CSM
speech: mean_presence=1.0000, 16/16 message bits decoded.

- Generator gains \`watermarker: Option<Box<dyn Watermarker>>\` slot;
  \`generate_to_wav\` runs \`wm.embed(&pcm)\` after post-process, before
  WAV write. Field is Send+Sync so the existing Arc<Mutex<Generator>>
  tts_server pattern still works.
- watermark.rs ships ResampledWatermarker<W> adapter for handling rate
  mismatches (CSM 24 kHz ↔ AudioSeal 16 kHz). Output length is normalized
  to input length so it's a transparent drop-in.
- examples/generate.rs gains --watermark-generator/--watermark-detector/
  --watermark-message flags. Loads AudioSeal, wraps in resampler, installs.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
 2026-04-25 19:36:17 -07:00
osobhandClaude Opus 4.7 15dd3575d4 Add rtx-csm: Rust-native port of Sesame CSM-1B with LoRA voice cloning 
A new model crate at crates/models/rtx-csm implementing end-to-end
inference, quantization, and fine-tuning for Sesame's Conversational
Speech Model (CSM-1B). Built on candle 0.9 + Kyutai Mimi codec.

Key capabilities:
- Inference (FP F16 on Metal, F32 on CPU, BF16 on CUDA)
- Quantized inference (Q8_0 / Q4_K_M GGUF, ~3x speedup, ~50% memory)
- Streaming Mimi decode with proper StreamTensor state machine
- In-context voice cloning via SpeakerProfile
- Classifier-Free Guidance (Koel-TTS recipe)
- Long-form chunked generation with rolling context
- Audio post-processing (HPF + declick + EBU R128 LUFS)
- Text input normalization (brackets, times, unicode, length caps)
- Frame-level repetition guard (loop-escape)
- Top-k + top-p sampling
- LoRA fine-tuning end-to-end (training + inference, on FP and Q8 bases)
- In-process Whisper ASR via whisper-rs (under --features asr)
- Standalone TTS HTTP server (Axum)
- Bench harness with manifest export + per-prompt WER

Phases delivered: quantization, ASR/WER eval, LoRA voice cloning, HTTP
service. AudioSeal/WavLM/Unmute remain as documented future work.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
 2026-04-25 18:33:57 -07:00