Researched what Sesame has actually disclosed publicly (vs. marketed)
and compared systematically against rtx-csm's shipped surface.
Key findings:
1. **rtx-csm has shipped a SUPERSET of Sesame's open release.** CSM-1B
inference + voice cloning + Q8 + production WebSocket server +
three-backend STT pipeline + per-phase observability — Sesame ships
inference code only.
2. **Sesame's deployed Maya is a cascaded STT->LLM->TTS pipeline**,
same architecture pattern as rtx-csm. Their research blog explicitly
states future work is "fully duplex models" — Maya today isn't
duplex either. We're structurally equivalent at the pipeline level
to Kyutai Unmute, Sesame's closest peer.
3. **Marketing latency claims are unverified.** "Sub-200 ms TTFA"
appears in third-party blogs, not in any Sesame paper. Production
benchmarks of similar cascaded stacks show 250-300 ms TTFT
(gpt-realtime, Unmute) — our 280-380 ms TTS-side is competitive.
Z.AI provider TTFT (~1.3 s) is the dominant cost in our 1.96 s
end-to-end.
4. **Critical correction**: Sesame ships SilentCipher (their fork of
Sony's), NOT AudioSeal (which is Meta's). Our Phase 4 AudioSeal
work is functionally equivalent but isn't the *literal* Sesame
watermarker. SilentCipher port is ~1-2 days.
5. **What's gated on Sesame**: CSM-3B / CSM-8B variants (trained but
never released), Maya personality fine-tune dataset, distilled
wearable variant. The Oct 2025 Series B + smart-glasses pivot
suggests they're unlikely to release any of these.
Punch-list of remaining gaps captured in the doc with status (Closed/
Partial/Open/N/A) per capability.
Recommended next chunks (prioritized):
1. SilentCipher port (~1-2 days) — literal Sesame watermarker parity
2. clawsample-csm integration (~5-10 days) — separate plan exists
3. Tier 2.1 VoXtream look-ahead (~5-7 days) — diminishing returns
after Phase 9.2's chunk_frames tuning
4. Tier 3 Frame-Stacked / VADUSA (training-required, ~3-6 weeks)
5. Distilled CSM (speculative, wait for product target)
Defensible framing: "rtx-csm is Sesame's open release + voice cloning
+ production HTTP/WS server + Kyutai-style cascaded duplex. Remaining
gap to internal Maya is (a) SilentCipher watermarker, (b) Sesame's
proprietary fine-tune dataset, (c) larger CSM variants Sesame chose
not to release."
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
