osobhandClaude Opus 4.7
15dd3575d4
A new model crate at crates/models/rtx-csm implementing end-to-end inference, quantization, and fine-tuning for Sesame's Conversational Speech Model (CSM-1B). Built on candle 0.9 + Kyutai Mimi codec. Key capabilities: - Inference (FP F16 on Metal, F32 on CPU, BF16 on CUDA) - Quantized inference (Q8_0 / Q4_K_M GGUF, ~3x speedup, ~50% memory) - Streaming Mimi decode with proper StreamTensor state machine - In-context voice cloning via SpeakerProfile - Classifier-Free Guidance (Koel-TTS recipe) - Long-form chunked generation with rolling context - Audio post-processing (HPF + declick + EBU R128 LUFS) - Text input normalization (brackets, times, unicode, length caps) - Frame-level repetition guard (loop-escape) - Top-k + top-p sampling - LoRA fine-tuning end-to-end (training + inference, on FP and Q8 bases) - In-process Whisper ASR via whisper-rs (under --features asr) - Standalone TTS HTTP server (Axum) - Bench harness with manifest export + per-prompt WER Phases delivered: quantization, ASR/WER eval, LoRA voice cloning, HTTP service. AudioSeal/WavLM/Unmute remain as documented future work. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
35 lines
1.1 KiB
Rust
35 lines
1.1 KiB
Rust
//! Diagnostic: read a GGUF file and print tensor names + shapes + dtypes.
|
|
|
|
use anyhow::Result;
|
|
use candle_core::quantized::gguf_file;
|
|
|
|
fn main() -> Result<()> {
|
|
let path = std::env::args().nth(1).expect("usage: inspect_gguf <file>");
|
|
let mut f = std::fs::File::open(&path)?;
|
|
let ct = gguf_file::Content::read(&mut f)?;
|
|
println!("metadata entries: {}", ct.metadata.len());
|
|
println!("tensor entries: {}", ct.tensor_infos.len());
|
|
let mut keys: Vec<_> = ct.tensor_infos.iter().collect();
|
|
keys.sort_by_key(|(k, _)| k.clone());
|
|
for (name, info) in keys.iter().take(20) {
|
|
println!(
|
|
" {:<60} shape={:?} dtype={:?}",
|
|
name, info.shape, info.ggml_dtype
|
|
);
|
|
}
|
|
if keys.len() > 20 {
|
|
println!(" ... ({} more)", keys.len() - 20);
|
|
}
|
|
// Specifically check a known weight that exists in csm
|
|
if let Some((_, info)) = keys
|
|
.iter()
|
|
.find(|(k, _)| k.contains("backbone.layers.0.attn.q_proj.weight"))
|
|
{
|
|
println!(
|
|
"\nbackbone.layers.0.attn.q_proj.weight: shape={:?} dtype={:?}",
|
|
info.shape, info.ggml_dtype
|
|
);
|
|
}
|
|
Ok(())
|
|
}
|