rtx-csm: Phase 12.4 — inference-time LoRA loading + extended-lora flag

Closes the train→generate loop the lora_train comment has promised
since Phase 3 ("forthcoming --lora flag on generate").

apply_lora_adapter(generator, path, rank, alpha, extended, device)
shared helper in src/training.rs wraps add_lora_to_backbone +
load_lora_adapter + refresh_lora. Both examples/generate and
examples/converse_server now call it instead of inlining their own
versions, and both now take --extended-lora to opt into Phase 12.1
coverage. Classic q+v adapters still load without the flag.

lora_train.rs now prints the exact `--lora <path> --lora-rank N
--lora-alpha N [--extended-lora]` command-line you need to apply the
trained adapter at inference, replacing the (forthcoming) message.

End-to-end verified: a Phase 12.3 curriculum-trained adapter loaded
into generate with identical seed/text produces different audio
(92KB vs 61KB, EOT @ frame 24 vs 16) — confirming the adapter takes
effect through to the sampled output. The 3-utterance smoke adapter
hasn't learned anything meaningful but the wiring is sound.

Phase 12 emotional voice stack now complete end-to-end:
12.1 capacity → 12.2 control tokens → 12.3 curriculum → 12.4 inference.

Lib suite 96/96.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

crates/models/rtx-csm/src/training.rs

@@ -514,6 +514,58 @@ fn clip_grads(
     Ok(())
 }
 
+/// One-shot inference-time LoRA loader: inject adapters with the matching
+/// shape, populate from a safetensors file, and refresh the model's tensor
+/// handles so the next forward pass picks up the trained weights.
+///
+/// Pass `extended = true` when the adapter was trained with Phase 12.1's
+/// extended coverage (q+k+v+output_proj + MLP); otherwise it loads the
+/// classic q+v recipe. Adapters trained with the narrow recipe still load
+/// cleanly when `extended = true` — the unmatched k/o/MLP slots are
+/// initialized B=0 and stay no-ops, at the cost of a small extra VarMap.
+///
+/// Returns the `VarMap` so callers can keep it alive (the model holds plain
+/// tensor handles refreshed from this VarMap; dropping it doesn't break a
+/// loaded model but `refresh_lora` after another `optim.step` would). For
+/// inference-only usage callers can drop the VarMap immediately after this
+/// returns — `refresh_lora` snapshots the values into the model's owned
+/// tensors.
+pub fn apply_lora_adapter<P: AsRef<Path>>(
+    generator: &mut Generator,
+    path: P,
+    rank: usize,
+    alpha: f32,
+    extended: bool,
+    device: &candle_core::Device,
+) -> Result<VarMap> {
+    let base = if extended {
+        crate::lora::LoraConfig::extended()
+    } else {
+        crate::lora::LoraConfig::default()
+    };
+    let cfg = crate::lora::LoraConfig { rank, alpha, ..base };
+    let vm = VarMap::new();
+    generator
+        .model
+        .inner
+        .add_lora_to_backbone(&cfg, &vm)
+        .map_err(|e| CsmError::Other(anyhow::anyhow!("add_lora_to_backbone: {e}")))?;
+    load_lora_adapter(&vm, path.as_ref(), device)?;
+    generator
+        .model
+        .inner
+        .refresh_lora(&vm)
+        .map_err(|e| CsmError::Other(anyhow::anyhow!("refresh_lora: {e}")))?;
+    tracing::info!(
+        "applied LoRA adapter from {} (rank={} alpha={} extended={})",
+        path.as_ref().display(),
+        rank,
+        alpha,
+        extended,
+    );
+    Ok(vm)
+}
+
 /// Save the LoRA adapter parameters (A,B for every layer) as a safetensors
 /// file. Reload via `load_lora_adapter`.
 pub fn save_lora_adapter<P: AsRef<Path>>(vm: &VarMap, out: P) -> Result<()> {