osobhandClaude Opus 4.7
6b69fb68c7
Per-frame CFG scale schedule (arXiv 2509.19668, Zheng & Maleki). Pure
inference-time. Standard CFG uses one fixed scale for the whole
sequence; this lets the scale vary across frames so early frames
(speaker character) get full CFG and later frames (text adherence)
get a lower scale.
What lands:
- src/cfg_schedule.rs: CfgSchedule enum (Constant / Step /
LinearRamp), scale_at(frame_idx), parser for CLI form
`step:E:L:T | linear:S:E:R | const:X`. 6 unit tests.
- src/generator.rs: GenerateOptions::cfg_schedule (takes precedence
over legacy cfg_scale; fixed-f64 path is preserved as
Constant(s) for back-compat). Generation loop reads
schedule.scale_at(frame_idx) and passes per-frame to
generate_frame_cfg.
- examples/generate.rs: --cfg-schedule, --cfg-scale, --enable-cfg
flags. Loading via load_csm_1b_with_cfg when --enable-cfg.
A/B with 6s output on Amini context, prompt about Selective CFG:
case cos WER transcript
no-CFG baseline 0.944 1.50 "Okay, the M.U. worked..." (off)
const:2.0 0.854 0.92 "On the right side." (short)
step:3.0:1.5:12 0.938 1.00 "The officer for the selective
C.F.D. paper recommends" (best)
linear:3.0:1.0:25 0.854 1.08 "On the surface..." (off)
Step schedule produces the transcript closest to the input ("the
selective CFG paper recommends..."). WER stays at 1.0 because
Moonshine doesn't know "CFG" as a word, but qualitatively this is
the only one that's coherently following the prompt. Speaker cosine
stays ≈ baseline (0.94) instead of dropping to 0.85 like the
constant and linear cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
64 lines
1.7 KiB
Rust
64 lines
1.7 KiB
Rust
//! Rust-native port of Sesame CSM-1B (Conversational Speech Model).
|
|
//!
|
|
//! Built on candle + moshi (Kyutai's Mimi neural audio codec). Lives inside
|
|
//! the rustytorch workspace; will migrate to rtx-tensor in a later stage.
|
|
|
|
pub mod asr;
|
|
pub mod audio_io;
|
|
pub mod audioseal;
|
|
pub mod audioseal_convert;
|
|
pub mod cfg_schedule;
|
|
pub mod config;
|
|
pub mod converse;
|
|
pub mod csm_fork;
|
|
pub mod csm_quantized;
|
|
pub mod diarize;
|
|
pub mod emotion2vec;
|
|
pub mod error;
|
|
pub mod generator;
|
|
pub mod hub;
|
|
pub mod llm_client;
|
|
pub mod longform;
|
|
pub mod lora;
|
|
pub mod mimi;
|
|
pub mod model;
|
|
pub mod moonshine;
|
|
pub mod post;
|
|
pub mod prompt;
|
|
pub mod quantize;
|
|
pub mod repetition;
|
|
pub mod sampler;
|
|
pub mod ser;
|
|
pub mod silentcipher;
|
|
pub mod silero_vad;
|
|
pub mod speaker;
|
|
pub mod speaker_sim;
|
|
pub mod steering;
|
|
pub mod stt;
|
|
pub mod text_norm;
|
|
pub mod tokenizer;
|
|
pub mod training;
|
|
pub mod util;
|
|
pub mod watermark;
|
|
pub mod wav2vec2;
|
|
pub mod wavlm_sv;
|
|
pub mod wavlm_sv_convert;
|
|
pub mod wer;
|
|
|
|
pub use audioseal::{AudioSealWatermarker, DetectionResult};
|
|
pub use config::{BackboneFlavor, DecoderFlavor, ModelConfig};
|
|
pub use error::{CsmError, Result};
|
|
pub use generator::{GenerateOptions, Generator};
|
|
pub use longform::{LongFormConfig, split_sentences};
|
|
pub use post::PostProcess;
|
|
pub use prompt::Segment;
|
|
pub use quantize::{QuantPolicy, TensorQuant};
|
|
pub use repetition::{RepetitionConfig, RepetitionGuard};
|
|
pub use speaker::{DEFAULT_PROFILE_BUDGET_TOKENS, SpeakerProfile};
|
|
pub use speaker_sim::{
|
|
CosineSimilarityFromEmbeddings, SpeakerSimilarity, SpectralCentroidSimilarity, WavLmSimilarity,
|
|
};
|
|
pub use text_norm::TextNormalize;
|
|
pub use watermark::{NoopWatermarker, ResampledWatermarker, Watermarker};
|
|
pub use wer::{WerResult, wer as compute_wer};
|