rtx-csm: Phase 8.8 — Moonshine end-to-end transcription works

Full encoder-decoder Moonshine v2 transcribing real audio in pure
candle 0.9 + Metal. No ort, no ggml, no protobuf. The path that
whisper-rs (Phase 7.6) and Silero V5 via ort (Phase 8.1.3) couldn't
deliver due to in-process linkage conflicts.

End-to-end on /tmp/asr_test.flac (LibriSpeech, 10.42 s):

  encode:    10 ms
  decode:   348 ms (49 tokens, 7.1 ms/token greedy, no KV cache)
  realtime factor: 0.068x  (~14x faster than realtime)

Output transcript:
  "He hoped there would be stew for dinner, turnips and carrots and
   bruised potatoes, and fat, mutton pieces to be ladled out in thick,
   peppered, flour-fat and sauce."

Ground truth:
  "He hoped there would be stew for dinner, turnips and carrots and
   bruised potatoes and fat mutton pieces to be ladled out in thick
   peppered flour-fattened sauce."

Near-perfect (a few punctuation tweaks, "flour-fat and sauce" vs
"flour-fattened sauce"). WER very low.

Compared to other STT backends in this crate:
  Kyutai STT 1B   :  1.01x realtime  (3 GB, hardware-bound)
  Whisper-tiny    :  0.020x realtime (in-process ggml -> CSM regression)
  **Moonshine-tiny: 0.068x realtime  (pure candle, no runtime conflict)**

Components shipped this commit:
  - Decoder::generate(encoder_output, cfg, max_tokens) — greedy
    autoregressive loop. No KV cache yet (each step re-runs the full
    growing token sequence — O(T^2) total). For 49-token transcripts
    at <500 ms total, KV cache isn't urgent.
  - load_tokenizer() — wraps tokenizers::Tokenizer::from_file for
    Moonshine's HF tokenizer.json (BPE, vocab 32_768).
  - examples/moonshine_transcribe — full pipeline: audio -> 16 kHz
    PCM -> encode -> decode -> detokenize -> transcript text.

Critical bug fixed: SwiGLU gate/up split direction. HF
modeling_moonshine.py says:
    hidden, gate = fc1(x).chunk(2, dim=-1)
    out = silu(gate) * hidden
The FIRST half of the fused fc1 output is `up` (multiplied), the
SECOND half is `gate` (silu-activated). I had it reversed in Phase
8.7 — the symptom was a degenerate "tt tt tt" repetition loop after
the first 2 tokens. Reversing the split unlocked the working
transcription. Captured in the code comment.

Remaining for Moonshine readiness in production:
  Phase 8.9 — KV cache for sub-200ms latency on long transcripts,
              plus a standalone moonshine_profile binary for the
              full A/B against Kyutai/Whisper.
  Phase 8.10 — wire as a third AsrEngine variant in converse_server
               (gated on English-only acceptance for the deploy).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

crates/models/rtx-csm/src/moonshine.rs

@@ -569,10 +569,14 @@ impl DecoderMlp {
         let h = self.fc1.forward(xs)?;
         let dims = h.dims();
         let last = dims.len() - 1;
-        // Split on the last dim into gate and up.
-        let gate = h.narrow(last, 0, self.intermediate)?;
-        let up = h.narrow(last, self.intermediate, self.intermediate)?;
-        // silu(gate) * up
+        // HF Moonshine MLP (verified against modeling_moonshine.py):
+        //   hidden, gate = fc1(x).chunk(2, dim=-1)
+        //   out = silu(gate) * hidden
+        // i.e., FIRST half is `up` (hidden), SECOND half is `gate`.
+        // We had this reversed initially, which produced a degenerate
+        // repetition loop after 1-2 tokens.
+        let up = h.narrow(last, 0, self.intermediate)?;
+        let gate = h.narrow(last, self.intermediate, self.intermediate)?;
         let activated = candle_nn::ops::silu(&gate)?.mul(&up)?;
         self.fc2.forward(&activated)
     }
@@ -686,6 +690,55 @@ pub fn load_full(
     Ok((encoder, decoder))
 }
 
+// ---------------------------------------------------------------------
+// Phase 8.8 — generation loop + tokenizer
+// ---------------------------------------------------------------------
+
+impl Decoder {
+    /// Greedy autoregressive decode given an encoder output. Stops on
+    /// `eos_token_id` or after `max_tokens`. Returns the predicted
+    /// token ids (excluding the initial start token).
+    ///
+    /// **No KV cache** — each step re-runs the decoder on the full
+    /// growing token sequence (O(T^2) total work). For Moonshine-tiny
+    /// at ~85 ms / step prefill, a 30-token transcript takes a few
+    /// seconds. KV-cached `step()` is a follow-up if needed.
+    pub fn generate(
+        &self,
+        encoder_output: &Tensor,
+        cfg: &MoonshineConfig,
+        max_tokens: usize,
+    ) -> Result<Vec<u32>> {
+        let device = encoder_output.device();
+        let mut tokens: Vec<u32> = vec![cfg.decoder_start_token_id];
+        let mut out = Vec::with_capacity(max_tokens);
+        for _ in 0..max_tokens {
+            let input = Tensor::from_vec(tokens.clone(), (1, tokens.len()), device)?;
+            let logits = self.forward(&input, encoder_output)?;
+            // logits: (1, T_so_far, vocab). Take last position.
+            let last_t = tokens.len() - 1;
+            let last = logits.narrow(1, last_t, 1)?.squeeze(1)?; // (1, vocab)
+            let argmax = last.argmax(1)?;
+            let next_id: u32 = argmax.to_dtype(DType::U32)?.to_vec1::<u32>()?[0];
+            if next_id == cfg.eos_token_id {
+                break;
+            }
+            out.push(next_id);
+            tokens.push(next_id);
+            if tokens.len() >= cfg.max_position_embeddings {
+                break;
+            }
+        }
+        Ok(out)
+    }
+}
+
+/// Convenience: load the HF tokenizer.json for Moonshine. Caller passes
+/// the path returned by hf_hub.
+pub fn load_tokenizer(path: &std::path::Path) -> std::result::Result<tokenizers::Tokenizer, Box<dyn std::error::Error + Send + Sync>> {
+    tokenizers::Tokenizer::from_file(path)
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;