rustyverse/rustytorch
2
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

rtx-csm: drop Mimi reload cadence to every-10 clips

Browse Source 
Empirical: with 25-clip cadence the cumulative state still tipped over
once at position 8102 mid-window during a 167-clip lora_eval load.
Drop to 10 — costs ~200ms×(N/10) at load time but eliminates the
state-leak panic across every corpus size we've tried up to 1505 clips.

Found while running the first successful 3-lecture Amini corpus through
the full data-prep + train + eval pipeline.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-04-28 16:33:55 -07:00
co-authored by Claude Opus 4.7
parent d42aba0f1b
commit 0568dd3653
1 changed files with 5 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/models/rtx-csm/src/training.rs
+5 -4
View File
@@ -193,10 +193,11 @@ impl TrainingDataset {
let speaker = row.speaker.unwrap_or(default_speaker); let speaker = row.speaker.unwrap_or(default_speaker);
let audio = audio_io::load_mono_24k(&wav_path)?; let audio = audio_io::load_mono_24k(&wav_path)?;
// Mimi's transformer carries a position counter across encode() // Mimi's transformer carries a position counter across encode()
// calls that `reset_state()` does NOT fully clear. Without an // calls that `reset_state()` does NOT fully clear. Reload every
// explicit reload it overflows after ~80-100 clips with // 10 clips — empirical floor; 25 still tipped over occasionally
// `narrow invalid args [8192, 32]`. Reload every 50 clips. // (single-window cumulative reaches ~8102 frames before the
if !examples.is_empty() && examples.len() % 50 == 0 { // 25-clip mark). Cost: ~200ms × N/10 reload overhead at load.
if !examples.is_empty() && examples.len() % 10 == 0 {
generator.mimi.reload().map_err(|e| { generator.mimi.reload().map_err(|e| {
CsmError::Config(format!("mimi reload at clip {}: {e}", examples.len())) CsmError::Config(format!("mimi reload at clip {}: {e}", examples.len()))
})?; })?;