rtx-csm: Phase 10.5 — SilentCipher in converse_server (Sesame parity shipped)

Adds `--watermark-silentcipher` flag to converse_server. Sesame's actual
production watermarker is now a drop-in option in the conversation
pipeline alongside the existing AudioSeal flags.

Usage:

  --watermark-silentcipher hf                # download from sony/silentcipher
  --watermark-silentcipher /path/to/dir      # local checkpoint dir
  --watermark-message 0xCAFE                 # 16-bit message (also drives AudioSeal)

Mutex with `--watermark-generator` / `--watermark-detector` (AudioSeal):
the Generator only carries one watermarker. Both are wrapped with
`ResampledWatermarker(24 kHz <-> 16 kHz)` for the CSM TTS path.

End-to-end production test (Q8 + Kyutai + mock LLM + SilentCipher 0xCAFE):

  client TTFA:     7888 ms
  total wall:     21860 ms
  assistant audio: 12.16 s @ 24 kHz, written to /tmp/converse_silent_response.wav
  re-detect:       confidence 0.7614, payload 0x0000CAFE  (PASS)

The 0.76 confidence (vs 1.00 in the standalone CLI test) is expected —
the assistant audio went through 24->16->24 resample plus stream-
encode-decode, all of which add noise. Still well above the 0.7
threshold we use for `Option<u16> -> Some/None` mapping in the
Watermarker trait impl.

A/B vs AudioSeal on the same /tmp/asr_test.flac (10.43 s @ 24 kHz):

                  AudioSeal           SilentCipher
  Embed timing    not in CLI          988 ms (0.10x rt)
  Detect timing   not in CLI         1493 ms (0.14x rt)
  Bit accuracy   16/16 bits          15/15 codes
  Confidence     1.0000              1.0000
  Message        0xCAFE              0xCAFE  (decimal 51966)

Both bit-perfect. AudioSeal carries 16 bits, SilentCipher carries up
to ~24 bits per patch (15 base-3 codes). For our use (16-bit job_id
or message hash), either fits.

Production recommendation: ship SilentCipher for literal Sesame
parity AND the structural advantages (smaller model, identical bit
accuracy, confidence-based threshold). AudioSeal stays available for
callers who want the per-sample presence map (which SilentCipher
doesn't provide).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-04-27 13:26:57 -07:00
co-authored by Claude Opus 4.7
parent df89372ad7
commit 385858a3ba
@@ -138,9 +138,26 @@ struct Cli {
#[arg(long)] #[arg(long)]
watermark_detector: Option<PathBuf>, watermark_detector: Option<PathBuf>,
/// 16-bit watermark message (decimal or 0xHEX). Defaults to 0. /// 16-bit watermark message (decimal or 0xHEX). Defaults to 0.
/// Used by both AudioSeal (`--watermark-{generator,detector}`)
/// and SilentCipher (`--watermark-silentcipher`).
#[arg(long, default_value = "0")] #[arg(long, default_value = "0")]
watermark_message: String, watermark_message: String,
/// Use Sesame's actual production watermarker — SilentCipher —
/// instead of AudioSeal. Pass a directory containing the three
/// released checkpoint files (`enc_c.ckpt`, `dec_c.ckpt`,
/// `dec_m_0.ckpt`). The 16 kHz model is the one we wire here;
/// the 44.1 kHz model is unsupported.
///
/// If you don't have a local copy, leave this empty and pass
/// the magic value `hf` to download from `sony/silentcipher`
/// at boot.
///
/// Mutex with `--watermark-generator` / `--watermark-detector`
/// (AudioSeal). Pick one watermarker.
#[arg(long)]
watermark_silentcipher: Option<String>,
/// Use the VAD-enabled STT variant (`kyutai/stt-1b-en_fr-candle`). /// Use the VAD-enabled STT variant (`kyutai/stt-1b-en_fr-candle`).
/// Adds 4 extra prediction heads emitting per-frame probabilities; /// Adds 4 extra prediction heads emitting per-frame probabilities;
/// /v1/converse uses head-2 probability > `--vad-threshold` for K /// /v1/converse uses head-2 probability > `--vad-threshold` for K
@@ -571,6 +588,68 @@ async fn main() -> Result<()> {
(AudioSealWatermarker requires both)." (AudioSealWatermarker requires both)."
); );
} }
// SilentCipher (Sesame's actual production watermarker). Mutex
// with --watermark-generator/-detector (AudioSeal): the Generator
// can only carry one watermarker.
if cli.watermark_silentcipher.is_some()
&& (cli.watermark_generator.is_some() || cli.watermark_detector.is_some())
{
anyhow::bail!(
"--watermark-silentcipher and --watermark-generator/-detector are mutually \
exclusive — pick one watermarker (Generator only carries one)."
);
}
if let Some(silent_path) = cli.watermark_silentcipher.as_ref() {
let msg_str = cli.watermark_message.trim();
let message: u16 = if let Some(rest) = msg_str
.strip_prefix("0x")
.or_else(|| msg_str.strip_prefix("0X"))
{
u16::from_str_radix(rest, 16)?
} else {
msg_str.parse::<u16>()?
};
// Resolve checkpoint paths: either an `hf` magic value (download
// from sony/silentcipher), or a local directory containing the
// three .ckpt files.
let (enc_c, dec_c, dec_m_0) = if silent_path == "hf" {
tracing::info!("downloading SilentCipher 16 kHz from sony/silentcipher...");
let api = hf_hub::api::sync::Api::new()?;
let repo = api.model("sony/silentcipher".to_string());
let dir = "16_khz/97561_iteration";
let enc = repo.get(&format!("{dir}/enc_c.ckpt"))?;
let dec = repo.get(&format!("{dir}/dec_c.ckpt"))?;
let dem = repo.get(&format!("{dir}/dec_m_0.ckpt"))?;
(enc, dec, dem)
} else {
let dir = std::path::PathBuf::from(silent_path);
(
dir.join("enc_c.ckpt"),
dir.join("dec_c.ckpt"),
dir.join("dec_m_0.ckpt"),
)
};
let cfg_sc = rtx_csm::silentcipher::SilentCipherConfig::sixteen_khz();
let inner = rtx_csm::silentcipher::SilentCipherWatermarker::from_ckpts(
cfg_sc, &enc_c, &dec_c, &dec_m_0, &device,
)?;
let wm = rtx_csm::silentcipher::SilentCipherWatermark::new(inner, message as u32);
// CSM is 24 kHz; SilentCipher 16 kHz. Wrap with the existing
// ResampledWatermarker so callers feed 24 kHz directly.
let wrapped = rtx_csm::ResampledWatermarker::new(
wm,
generator.config.sample_rate,
16_000,
);
generator.set_watermarker(Box::new(wrapped));
tracing::info!(
"SilentCipher watermarker installed (message=0x{:04X}, model 16 kHz, output {} Hz)",
message,
generator.config.sample_rate,
);
}
if cli.stream_tts && generator.watermarker.is_some() { if cli.stream_tts && generator.watermarker.is_some() {
anyhow::bail!( anyhow::bail!(
"--stream-tts cannot combine with --watermark-* (watermark needs \ "--stream-tts cannot combine with --watermark-* (watermark needs \