osobhandClaude Opus 4.7
15dd3575d4
A new model crate at crates/models/rtx-csm implementing end-to-end inference, quantization, and fine-tuning for Sesame's Conversational Speech Model (CSM-1B). Built on candle 0.9 + Kyutai Mimi codec. Key capabilities: - Inference (FP F16 on Metal, F32 on CPU, BF16 on CUDA) - Quantized inference (Q8_0 / Q4_K_M GGUF, ~3x speedup, ~50% memory) - Streaming Mimi decode with proper StreamTensor state machine - In-context voice cloning via SpeakerProfile - Classifier-Free Guidance (Koel-TTS recipe) - Long-form chunked generation with rolling context - Audio post-processing (HPF + declick + EBU R128 LUFS) - Text input normalization (brackets, times, unicode, length caps) - Frame-level repetition guard (loop-escape) - Top-k + top-p sampling - LoRA fine-tuning end-to-end (training + inference, on FP and Q8 bases) - In-process Whisper ASR via whisper-rs (under --features asr) - Standalone TTS HTTP server (Axum) - Bench harness with manifest export + per-prompt WER Phases delivered: quantization, ASR/WER eval, LoRA voice cloning, HTTP service. AudioSeal/WavLM/Unmute remain as documented future work. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
49 lines
1.4 KiB
Rust
49 lines
1.4 KiB
Rust
use crate::error::Result;
|
|
use candle_core::Tensor;
|
|
use candle_transformers::generation::{LogitsProcessor, Sampling};
|
|
|
|
pub const DEFAULT_TEMPERATURE: f64 = 0.9;
|
|
pub const DEFAULT_TOPK: usize = 50;
|
|
pub const DEFAULT_TOPP: f64 = 0.9;
|
|
|
|
pub struct CsmSampler {
|
|
inner: LogitsProcessor,
|
|
}
|
|
|
|
impl CsmSampler {
|
|
/// Build a sampler with explicit knobs. `top_p == 0.0` (or `>= 1.0`) disables
|
|
/// nucleus filtering and falls back to pure top-k. `temperature <= 0.0` selects
|
|
/// argmax (deterministic) regardless of top_k/top_p.
|
|
pub fn new(seed: u64, temperature: f64, top_k: usize, top_p: f64) -> Self {
|
|
let sampling = if temperature <= 0.0 {
|
|
Sampling::ArgMax
|
|
} else if top_p > 0.0 && top_p < 1.0 {
|
|
Sampling::TopKThenTopP {
|
|
k: top_k,
|
|
p: top_p,
|
|
temperature,
|
|
}
|
|
} else {
|
|
Sampling::TopK {
|
|
k: top_k,
|
|
temperature,
|
|
}
|
|
};
|
|
Self {
|
|
inner: LogitsProcessor::from_sampling(seed, sampling),
|
|
}
|
|
}
|
|
|
|
pub fn default_for_csm(seed: u64) -> Self {
|
|
Self::new(seed, DEFAULT_TEMPERATURE, DEFAULT_TOPK, DEFAULT_TOPP)
|
|
}
|
|
|
|
pub fn sample(&mut self, logits: &Tensor) -> Result<u32> {
|
|
Ok(self.inner.sample(logits)?)
|
|
}
|
|
|
|
pub fn inner_mut(&mut self) -> &mut LogitsProcessor {
|
|
&mut self.inner
|
|
}
|
|
}
|