rtx-csm: Phase 10.4 — SilentCipher detect + Watermarker trait + apply CLI

End-to-end SilentCipher: bit-perfect round-trip on real LibriSpeech
audio. Sesame's actual production watermarker now works in pure
candle 0.9 + Metal.

New components in src/silentcipher.rs:

  detect(samples_16k) -> DetectResult
    1. RMS-normalize to VCTK baseline (matches embed pre-conditioning)
    2. STFT -> magnitude
    3. dec_m_0(magnitude) -> (B, message_dim, 1, T) logits
    4. argmax along message_dim -> (T,) per-frame predictions
    5. Truncate to multiple of message_len
    6. Reshape to (n_patches, message_len), per-column mode
    7. Find terminator (value 0), rotate so payload follows it
    8. Subtract +1 offset -> original codes

  encode_bits / decode_bits  (Phase 10.4 fix)
    Switched from base-4 (2 bits per code) to base-`(message_dim - 1)`.
    The 16 kHz model has message_dim=4 = 3 carrier values (1,2,3) +
    terminator (0), NOT 4 carrier values. Original base-4 packing
    occasionally produced value 3, which Python's
    `np.identity(4)[index+1]` would have crashed on. Real capacity:
    15 codes x log2(3) ~= 23.78 bits per patch.

  SilentCipherWatermark (impl Watermarker)
    Wraps a SilentCipherWatermarker with a fixed default_payload so
    it satisfies the existing Watermarker trait. Maps confidence ->
    DetectionResult.mean_presence and the lower-16-bits of the
    decoded payload -> DetectionResult.message (None below confidence
    0.7 to suppress false positives).

  examples/silentcipher_apply
    Mirrors audioseal_apply: --in / --out / --payload / --detect-only.
    Loads from sony/silentcipher HF repo, embeds, optionally
    resamples back to source rate, optionally re-detects to verify.

Verified end-to-end (LibriSpeech /tmp/asr_test.flac, 10.42 s @ 16 kHz):

  Build:        29 ms (3 .ckpt files from HF cache)
  Embed:      1213 ms = 0.116x realtime
  Detect:     1838 ms = 0.18x realtime
  payload:        0x00BC614E (in)
  recovered:      0x00BC614E (out)
  codes match:    15 / 15
  confidence:     1.0000

Clean (un-watermarked) audio: confidence 0.475, codes mostly 0 -
strong signal-vs-noise discrimination at the 0.7 threshold.

This closes the most surprising gap from the Sesame stack analysis:
rtx-csm now has the *literal* Sesame watermarker (not Meta's
AudioSeal) working in pure candle. AudioSeal stays available for
callers that prefer it.

Phase 10.5 (next): wire as a third option in converse_server alongside
AudioSeal, and a 24/16 kHz ResampledWatermarker for the CSM path.
Plus an A/B bench (SilentCipher vs AudioSeal).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

crates/models/rtx-csm/Cargo.toml

@@ -273,3 +273,7 @@ path = "examples/silentcipher_inspect.rs"
 [[example]]
 name = "silentcipher_smoke"
 path = "examples/silentcipher_smoke.rs"
+
+[[example]]
+name = "silentcipher_apply"
+path = "examples/silentcipher_apply.rs"

crates/models/rtx-csm/examples/silentcipher_apply.rs

@@ -0,0 +1,177 @@
+//! Phase 10.4 — apply SilentCipher to an arbitrary WAV. Mirror of
+//! `audioseal_apply`. Loads the released 16 kHz checkpoint from
+//! `sony/silentcipher`, embeds a payload, optionally re-detects to
+//! verify round-trip.
+//!
+//! Usage (embed + verify):
+//! ```bash
+//! cargo run -p rtx-csm --release --features metal --example silentcipher_apply -- \
+//!     --in /tmp/asr_test.flac --out /tmp/asr_test_silent.wav --payload 12345678
+//! ```
+//!
+//! Usage (detect only):
+//! ```bash
+//! cargo run -p rtx-csm --release --features metal --example silentcipher_apply -- \
+//!     --in /tmp/asr_test_silent.wav --detect-only
+//! ```
+
+use anyhow::{Context, Result};
+use candle_core::Device;
+use clap::Parser;
+use hf_hub::api::sync::Api;
+use rtx_csm::{
+    audio_io,
+    silentcipher::{SilentCipherConfig, SilentCipherWatermarker},
+};
+use std::path::PathBuf;
+use std::time::Instant;
+
+const REPO: &str = "sony/silentcipher";
+const CKPT_DIR: &str = "16_khz/97561_iteration";
+const SR: u32 = 16_000;
+
+#[derive(Debug, Parser)]
+struct Cli {
+    /// Input WAV (any rate, mono OK; resampled to 16 kHz).
+    #[arg(long = "in")]
+    input: PathBuf,
+    /// Output WAV. Optional in `--detect-only` mode.
+    #[arg(long)]
+    out: Option<PathBuf>,
+    /// 30-bit payload to embed. Default 0 (no payload). 16 kHz model
+    /// can carry up to ~23.78 bits (15 base-3 codes).
+    #[arg(long, default_value_t = 0)]
+    payload: u32,
+    /// Skip embedding; just run detect on the input. Useful for
+    /// verifying that a previously-watermarked file still carries the
+    /// signature.
+    #[arg(long)]
+    detect_only: bool,
+    /// Source sample rate of the input WAV. Default = read from file.
+    #[arg(long)]
+    source_rate: Option<u32>,
+    #[arg(long)]
+    cpu: bool,
+}
+
+fn main() -> Result<()> {
+    tracing_subscriber::fmt().with_max_level(tracing::Level::WARN).init();
+    let cli = Cli::parse();
+
+    let device = if cli.cpu {
+        Device::Cpu
+    } else if candle_core::utils::metal_is_available() {
+        Device::new_metal(0)?
+    } else {
+        Device::Cpu
+    };
+    eprintln!("device: {device:?}");
+
+    // Load model.
+    let api = Api::new()?;
+    let repo = api.model(REPO.to_string());
+    let enc_c = repo.get(&format!("{CKPT_DIR}/enc_c.ckpt"))?;
+    let dec_c = repo.get(&format!("{CKPT_DIR}/dec_c.ckpt"))?;
+    let dec_m_0 = repo.get(&format!("{CKPT_DIR}/dec_m_0.ckpt"))?;
+    let cfg = SilentCipherConfig::sixteen_khz();
+    let load_t = Instant::now();
+    let wm = SilentCipherWatermarker::from_ckpts(cfg, &enc_c, &dec_c, &dec_m_0, &device)?;
+    eprintln!("model loaded in {} ms", load_t.elapsed().as_millis());
+
+    // Load + resample input to 16 kHz.
+    let src_rate = cli.source_rate.unwrap_or_else(|| {
+        // Read native rate via hound where possible — but we just use
+        // load_mono_at_rate which resamples to whatever target we
+        // pass. For round-trip output at the original rate, we read
+        // the source rate manually.
+        match hound::WavReader::open(&cli.input) {
+            Ok(r) => r.spec().sample_rate,
+            Err(_) => SR, // assume 16 kHz if we can't read header
+        }
+    });
+    let src = audio_io::load_mono_at_rate(&cli.input, src_rate)?;
+    let src_16k = audio_io::resample(&src, src_rate, SR)?;
+    eprintln!(
+        "loaded {} ({} samples @ {src_rate} Hz, {} samples @ {SR} Hz)",
+        cli.input.display(),
+        src.len(),
+        src_16k.len()
+    );
+
+    if cli.detect_only {
+        // Detect on the input directly.
+        let t = Instant::now();
+        let r = wm
+            .detect(&src_16k)
+            .context("silentcipher detect")?;
+        let ms = t.elapsed().as_millis();
+        let recovered = wm.decode_bits(&r.codes);
+        println!();
+        println!("=== detect-only ===");
+        println!("input:        {}", cli.input.display());
+        println!("audio:        {} samples ({:.2}s) at 16 kHz", src_16k.len(), src_16k.len() as f32 / SR as f32);
+        println!("detect:       {ms} ms");
+        println!("confidence:   {:.4}", r.confidence);
+        println!("payload:      0x{recovered:08X} ({recovered})");
+        println!("codes:        {:?}", r.codes);
+        return Ok(());
+    }
+
+    // Embed.
+    let codes = wm.encode_bits(cli.payload);
+    eprintln!(
+        "payload 0x{:08X} -> {} codes (each in 0..{}): {:?}",
+        cli.payload,
+        codes.len(),
+        wm.cfg.message_dim,
+        codes
+    );
+    let embed_t = Instant::now();
+    let watermarked_16k = wm
+        .embed(&src_16k, &codes)
+        .context("silentcipher embed")?;
+    let embed_ms = embed_t.elapsed().as_millis();
+    eprintln!(
+        "embed: {embed_ms} ms ({:.3}× realtime)",
+        embed_ms as f32 / (src_16k.len() as f32 / SR as f32 * 1000.0)
+    );
+
+    // Resample back to source rate and write output.
+    let out_path = cli
+        .out
+        .as_ref()
+        .context("--out is required unless --detect-only is set")?;
+    let watermarked = audio_io::resample(&watermarked_16k, SR, src_rate)?;
+    let mut watermarked = watermarked;
+    watermarked.truncate(src.len());
+    audio_io::write_wav_mono(out_path.as_path(), &watermarked, src_rate)?;
+    eprintln!("wrote {} ({} samples @ {} Hz)", out_path.display(), watermarked.len(), src_rate);
+
+    // Round-trip detect on the resampled output (the realistic test).
+    let probe_16k = audio_io::resample(&watermarked, src_rate, SR)?;
+    let detect_t = Instant::now();
+    let r = wm
+        .detect(&probe_16k)
+        .context("silentcipher detect")?;
+    let detect_ms = detect_t.elapsed().as_millis();
+    let recovered = wm.decode_bits(&r.codes);
+    let matching = r
+        .codes
+        .iter()
+        .zip(codes.iter())
+        .filter(|(a, b)| a == b)
+        .count();
+    println!();
+    println!("=== round-trip ===");
+    println!("payload:      0x{:08X}  (expected)", cli.payload);
+    println!("recovered:    0x{recovered:08X}");
+    println!("codes match:  {matching} / {}", codes.len());
+    println!("confidence:   {:.4}", r.confidence);
+    println!("detect:       {detect_ms} ms");
+    if matching == codes.len() && recovered == cli.payload {
+        println!("PASS");
+    } else {
+        println!("WEAK — bit accuracy {:.0}%", 100.0 * matching as f32 / codes.len() as f32);
+    }
+    Ok(())
+}

crates/models/rtx-csm/examples/silentcipher_smoke.rs

@@ -63,8 +63,9 @@ fn main() -> Result<()> {
     eprintln!("=== synthetic sine ===");
     eprintln!("input samples: {}", samples.len());
 
-    // Encode a deliberate payload (5 bytes -> 15 codes via encode_bits).
+    // 16 kHz model carries ~23.78 bits per patch (15 base-3 codes).
-    let payload = 0xCAFEBABEu32;
+    // Pick a payload that's well below 3^15 = 14_348_907.
+    let payload = 12345678u32;
     let codes = wm.encode_bits(payload);
     eprintln!(
         "payload 0x{payload:08X} -> {} codes (each in 0..{}): {:?}",
@@ -145,6 +146,70 @@ fn main() -> Result<()> {
             16_000,
         )?;
         eprintln!("wrote /tmp/silentcipher_smoke.wav for ear test");
+
+        // Phase 10.4 round-trip: detect on the watermarked audio,
+        // verify we recover the embedded codes.
+        let detect_t = Instant::now();
+        let result = wm.detect(&watermarked).context("detect speech")?;
+        let detect_ms = detect_t.elapsed().as_millis();
+        eprintln!();
+        eprintln!("=== detect round-trip ===");
+        eprintln!(
+            "detect: {detect_ms} ms ({:.4}× realtime)",
+            detect_ms as f32 / (speech.len() as f32 / 16_000.0 * 1000.0)
+        );
+        eprintln!("recovered codes:    {:?}", result.codes);
+        eprintln!("confidence:         {:.3}", result.confidence);
+        let recovered_payload = wm.decode_bits(&result.codes);
+        eprintln!("recovered payload:  0x{recovered_payload:08X}");
+        eprintln!("expected payload:   0x{payload:08X}");
+        let matching_codes = result
+            .codes
+            .iter()
+            .zip(codes.iter())
+            .filter(|(a, b)| a == b)
+            .count();
+        eprintln!(
+            "matching codes:     {} / {} ({:.1}%)",
+            matching_codes,
+            codes.len(),
+            100.0 * matching_codes as f32 / codes.len() as f32
+        );
+        if matching_codes == codes.len() {
+            println!("PASS: full round-trip recovers the embedded payload");
+        } else if matching_codes >= codes.len() * 3 / 4 {
+            println!("PARTIAL: most codes recovered ({matching_codes}/{}); good enough for confidence-based detection", codes.len());
+        } else {
+            println!(
+                "WEAK: only {matching_codes}/{} codes recovered. Likely a normalization bug (Phase 10.5 to fix)",
+                codes.len()
+            );
+        }
+
+        // Also detect on UN-watermarked audio — should NOT recover the
+        // payload (or low confidence).
+        let result_clean = wm.detect(&speech).context("detect clean")?;
+        eprintln!();
+        eprintln!("=== detect on un-watermarked audio ===");
+        eprintln!("recovered codes:    {:?}", result_clean.codes);
+        eprintln!("confidence:         {:.3}", result_clean.confidence);
+        let clean_matching = result_clean
+            .codes
+            .iter()
+            .zip(codes.iter())
+            .filter(|(a, b)| a == b)
+            .count();
+        if clean_matching < codes.len() / 2 {
+            println!(
+                "PASS: clean audio does NOT recover the payload ({clean_matching}/{} match)",
+                codes.len()
+            );
+        } else {
+            println!(
+                "WARN: clean audio matches {clean_matching}/{} codes — confidence threshold needed for false-positive control",
+                codes.len()
+            );
+        }
     }
     Ok(())
 }

crates/models/rtx-csm/src/silentcipher.rs

@@ -673,18 +673,213 @@ impl SilentCipherWatermarker {
         Ok(out)
     }
 
-    /// Convenience: pack a `u32` into `message_len - 1` 2-bit codes.
+    /// Pack a `u32` payload into `message_len - 1` codes for embedding.
-    /// Output values are in `0..message_dim` (i.e., `0..4` for the
+    /// Each code is in `0..(message_dim - 1)`, i.e. `0..3` for the
-    /// 16 kHz model). Up to 30 bits of payload fit (15 codes × 2 bits).
+    /// 16 kHz model (3-way alphabet, since the 4th identity column is
+    /// reserved for the terminator). Capacity for 16 kHz: 15 codes ×
+    /// log₂(3) ≈ **23.78 bits per patch** (max payload < 14_348_907).
+    ///
+    /// Encoding: base-`(message_dim - 1)` representation, MSB first.
     pub fn encode_bits(&self, payload: u32) -> Vec<u32> {
         let n = self.cfg.message_len - 1;
-        let mut codes = Vec::with_capacity(n);
+        let radix = (self.cfg.message_dim - 1).max(2) as u32;
-        for i in 0..n {
+        let mut codes = vec![0u32; n];
-            let shift = (n - 1 - i) * 2;
+        let mut p = payload;
-            codes.push((payload >> shift) & 0x3);
+        for i in (0..n).rev() {
+            codes[i] = p % radix;
+            p /= radix;
         }
         codes
     }
+
+    /// Inverse of `encode_bits`: pack codes back into a `u32` payload
+    /// using base-`(message_dim - 1)`.
+    pub fn decode_bits(&self, codes: &[u32]) -> u32 {
+        let radix = (self.cfg.message_dim - 1).max(2) as u32;
+        let mut out = 0u32;
+        for &c in codes {
+            out = out.saturating_mul(radix).saturating_add(c % radix);
+        }
+        out
+    }
+
+    /// Detect: recover the embedded codes + a confidence score from a
+    /// (potentially-watermarked) audio signal. Mirrors `decode_wav` in
+    /// the Python source line-for-line:
+    ///
+    /// 1. RMS-normalize to VCTK baseline (same pre-conditioning as embed)
+    /// 2. STFT -> magnitude
+    /// 3. dec_m_0(magnitude) -> (1, message_dim, 1, T) logits
+    /// 4. argmax along message_dim -> (T,) predicted codes
+    /// 5. Truncate to multiple of message_len, reshape to (n_patches, message_len)
+    /// 6. Per-column mode -> (message_len,) consensus codes
+    /// 7. Find the terminator (0) -> rotate so it ends the sequence
+    /// 8. Subtract 1 (the +1 offset added during encode) -> original codes
+    pub fn detect(&self, samples_16k: &[f32]) -> Result<DetectResult> {
+        if samples_16k.is_empty() {
+            return Ok(DetectResult {
+                codes: vec![],
+                confidence: 0.0,
+            });
+        }
+        // Pre-condition energy (same as embed).
+        let original_power = samples_16k.iter().map(|&s| s * s).sum::<f32>()
+            / samples_16k.len() as f32;
+        if original_power < 1e-12 {
+            return Ok(DetectResult {
+                codes: vec![],
+                confidence: 0.0,
+            });
+        }
+        let pre_scale = (AVERAGE_ENERGY_VCTK / original_power).sqrt();
+        let normalized: Vec<f32> =
+            samples_16k.iter().map(|&s| s * pre_scale).collect();
+
+        // STFT.
+        let (mag, _phase, n_frames) = self.stft.forward(&normalized);
+        let n_freq = self.cfg.n_fft / 2 + 1;
+        let mag_t = Tensor::from_vec(mag, (1, 1, n_freq, n_frames), &self.device)?;
+
+        // dec_m_0 forward -> (1, message_dim, 1, T).
+        let logits = self.dec_m.forward(&mag_t)?;
+        // Squeeze the singleton dims to get (message_dim, T).
+        let logits_2d = logits.squeeze(0)?.squeeze(1)?; // (message_dim, T)
+
+        // Argmax along message_dim (axis 0).
+        let predicted = logits_2d.argmax(0)?; // (T,)
+        let pred: Vec<u32> = predicted
+            .to_dtype(DType::U32)?
+            .to_vec1::<u32>()?;
+
+        // Truncate to a multiple of message_len.
+        let m_len = self.cfg.message_len;
+        let n_patches = pred.len() / m_len;
+        if n_patches == 0 {
+            return Ok(DetectResult {
+                codes: vec![],
+                confidence: 0.0,
+            });
+        }
+        let pred = &pred[..n_patches * m_len];
+
+        // Per-column mode + confidence.
+        let mut consensus = vec![0u32; m_len];
+        let mut confidences = vec![0.0f32; m_len];
+        for col in 0..m_len {
+            let mut counts = std::collections::HashMap::<u32, u32>::new();
+            for row in 0..n_patches {
+                let v = pred[row * m_len + col];
+                *counts.entry(v).or_insert(0) += 1;
+            }
+            let (best_val, best_count) = counts.iter().max_by_key(|&(_, &c)| c).unwrap();
+            consensus[col] = *best_val;
+            confidences[col] = *best_count as f32 / n_patches as f32;
+        }
+        let confidence = confidences.iter().sum::<f32>() / m_len as f32;
+
+        // Find the terminator (0) and rotate so the message starts after it.
+        let end_char = consensus.iter().position(|&v| v == 0);
+        let codes: Vec<u32> = match end_char {
+            Some(idx) if idx + 1 < m_len => {
+                // Rotate: tail (after terminator) + head (before terminator)
+                let mut rotated: Vec<u32> = consensus[idx + 1..].to_vec();
+                rotated.extend_from_slice(&consensus[..idx]);
+                rotated.iter().map(|&v| v.saturating_sub(1)).collect()
+            }
+            _ => {
+                // No terminator found OR it's at the very end: take first
+                // message_len-1 values, subtract 1.
+                consensus[..m_len - 1]
+                    .iter()
+                    .map(|&v| v.saturating_sub(1))
+                    .collect()
+            }
+        };
+
+        Ok(DetectResult { codes, confidence })
+    }
+}
+
+/// Output of [`SilentCipherWatermarker::detect`].
+#[derive(Debug, Clone)]
+pub struct DetectResult {
+    /// Recovered codes, length `message_len - 1`. Each value is in
+    /// `0..message_dim` (i.e. `0..4` for the 16 kHz model).
+    pub codes: Vec<u32>,
+    /// Mean per-column agreement across patches (range `[0, 1]`).
+    pub confidence: f32,
+}
+
+// ---------------------------------------------------------------------
+// Phase 10.4 — `Watermarker` trait integration
+// ---------------------------------------------------------------------
+
+/// SilentCipher with a fixed payload, ready to drop into
+/// `Generator::set_watermarker`. Use [`ResampledWatermarker`] to
+/// adapt for a 24 kHz signal (CSM TTS native rate).
+///
+/// `payload` is the value to embed when [`Watermarker::embed`] is
+/// called without an explicit message. Up to ~23.78 bits fit in the
+/// 16 kHz model (15 base-3 codes); use `embed_with_message` for the
+/// trait's u16 override.
+pub struct SilentCipherWatermark {
+    inner: SilentCipherWatermarker,
+    default_payload: u32,
+}
+
+impl SilentCipherWatermark {
+    pub fn new(inner: SilentCipherWatermarker, default_payload: u32) -> Self {
+        Self {
+            inner,
+            default_payload,
+        }
+    }
+
+    /// Detect a watermark in the audio. Returns the AudioSeal-shaped
+    /// `DetectionResult` so callers can use the same downstream
+    /// reporting (mean_presence, message lower-16-bits, etc.).
+    pub fn detect(
+        &self,
+        samples_16k: &[f32],
+    ) -> crate::error::Result<crate::audioseal::DetectionResult> {
+        let r = self
+            .inner
+            .detect(samples_16k)
+            .map_err(|e| crate::CsmError::Config(format!("silentcipher detect: {e}")))?;
+        let payload = self.inner.decode_bits(&r.codes);
+        // Map our 30-bit-ish payload to the AudioSeal `Option<u16>`
+        // contract: lower 16 bits, present iff confidence is high.
+        let message = if r.confidence >= 0.7 {
+            Some((payload & 0xFFFF) as u16)
+        } else {
+            None
+        };
+        Ok(crate::audioseal::DetectionResult {
+            mean_presence: r.confidence,
+            // SilentCipher doesn't emit a per-sample presence map (no
+            // sample-level discriminator like AudioSeal's). Leave as
+            // an empty vec to satisfy the type without claiming false
+            // information.
+            presence_per_sample: Vec::new(),
+            message,
+        })
+    }
+}
+
+impl crate::watermark::Watermarker for SilentCipherWatermark {
+    fn embed(&self, audio: &[f32]) -> crate::error::Result<Vec<f32>> {
+        let codes = self.inner.encode_bits(self.default_payload);
+        self.inner
+            .embed(audio, &codes)
+            .map_err(|e| crate::CsmError::Config(format!("silentcipher embed: {e}")))
+    }
+
+    fn embed_with_message(&self, audio: &[f32], message: u16) -> crate::error::Result<Vec<f32>> {
+        let codes = self.inner.encode_bits(message as u32);
+        self.inner
+            .embed(audio, &codes)
+            .map_err(|e| crate::CsmError::Config(format!("silentcipher embed: {e}")))
+    }
 }
 
 #[cfg(test)]