# VoXtream-style look-ahead port notes (Tier 2.1)

Working notes for the planned look-ahead-window optimization on the TTS
side. Captures architecture, ecosystem facts established by Phase 9.1
spike, and an ordered porting plan so a future session can pick up cold.

## Why this exists

Production stack post-Phase 8.10:
- recv_phase (Moonshine):     0 ms
- stt_post:                332 ms
- LLM TTFT (Z.AI):       ~1300 ms
- **TTS first chunk:      ~600 ms**   ← target of this work
- total client TTFA:    ~1959 ms

VoXtream (arXiv 2509.15969) claims **102 ms first packet** by adding a
phoneme-level look-ahead window before TTS gen starts. The look-ahead
runs ahead of the audio stream so it doesn't delay onset.

Ports cleanly to CSM (Llama backbone + depth decoder) — does not require
retraining the model. Just a new pre-conditioning pipeline.

## Phonemizer ecosystem (Phase 9.1 finding)

| Crate | Approach | Latency | Conflicts | Verdict |
|-------|----------|---------|-----------|---------|
| **espeak-ng 0.1.1** | pure Rust port of eSpeak NG | 1-5 ms/word | none | **PICK THIS** |
| voirs-g2p 0.1.0-rc.1 | neural, candle 0.9.2 | 10-50 ms/word | libc dep | only if neural needed |
| grapheme_to_phoneme 0.1.0 | seq2seq RNN, ARPAbet | 2-8 ms/word | none | abandoned 2020; viable fallback |
| phonetisaurus-g2p 0.1.1 | FST | sub-ms | none | needs pre-trained FST |

`espeak-ng` is the right pick:
- Zero C linkage (the `c-oracle` feature is opt-in for testing only)
- 100+ languages, IPA output
- Mature, used in real TTS pipelines
- No dep conflicts with candle 0.9, sentencepiece-sys, hound, symphonia, ebur128

For our latency budget: 5-10 word look-ahead × 1-5 ms = **5-50 ms total
phonemizer cost.** Well within the 50 ms allowance for a 102 ms first-
packet target.

## Architecture (planned)

Three integration points — all in `src/converse.rs` and `src/generator.rs`:

### 1. Phonemizer service

A small wrapper around `espeak-ng` exposed as
`rtx_csm::phonemizer::Phonemizer` (new module). Methods:

```rust
pub struct Phonemizer { /* espeak_ng::Voice */ }
impl Phonemizer {
    pub fn english() -> Result<Self>;
    /// Phonemize a sentence-fragment into IPA phonemes.
    pub fn phonemize(&self, text: &str) -> Result<Vec<Phoneme>>;
}
```

Use IPA for portability; an ARPAbet-tagged fork is doable later if CSM
is found to phonologize better with ARPAbet.

### 2. Look-ahead window in Converse

Modify `Converse::run_streaming` so that as the LLM emits tokens, we:

1. Phonemize each new word (call espeak-ng on the latest token-completed
   word — usually triggered on whitespace boundary).
2. Maintain a sliding "look-ahead phoneme buffer" of the next N words
   (N=5-10).
3. Pass the look-ahead phonemes to the TTS as a hint *before* the first
   audio frame is generated.

The hint is a free-form input; CSM's prompt format already accepts text
with formatting. We can experiment with formats:
- Inline phonemes in the text: `[hə hoʊpt] he hoped...`
- A separate "phoneme prompt" segment before the assistant turn
- A `Segment::PhonemeHint(...)` variant added to the prompt builder

### 3. Generator look-ahead-aware forward

Optionally modify `Generator::generate_streaming` to accept a "phoneme
prefix" that conditions the early frames on look-ahead phonemes,
helping the model start with prosody appropriate to the upcoming word.

This is the most speculative piece — the VoXtream paper's exact
mechanism (does the look-ahead enter the model as text? Embeddings?
Cross-attention?) needs careful reading. The arxiv abstract says
"phoneme transformer + monotonic alignment + dynamic look-ahead." We
may need to read the full paper before implementing this part.

## Open questions (resolve before implementing)

1. **Phoneme integration mechanism.** VoXtream's paper says the
   look-ahead doesn't delay onset, which implies phonemes are NOT just
   prepended to the text input. They're presumably consumed by an
   auxiliary path. Read full paper for the exact mechanism.

2. **CSM pre-training compatibility.** CSM-1B was trained on text +
   audio tokens. It has never seen IPA or ARPAbet during training. The
   look-ahead might need to be encoded as natural-language hints or
   phonetic-alphabet transliteration (ARPAbet → spelled-out form).

3. **Empirical win on this hardware.** VoXtream's 102 ms claim is on
   their training-aware model. CSM-1B was NOT trained for this. The
   port might give us a smaller win — say 200-400 ms TTS first chunk
   vs current 600 ms. Need a concrete bench to commit.

## Porting tasks (~5-7 days, multi-session)

In order of dependency:

1. **`src/phonemizer.rs` skeleton (~0.5 day)**
   - Add `espeak-ng = "0.1"` to Cargo.toml as optional dep behind
     `phonemizer` feature
   - Wrap `espeak_ng::Voice` with our Phonemizer API
   - Standalone smoke test: phonemize "He hoped there would be stew" →
     verify IPA output

2. **`docs/voxtream_paper_summary.md` (~0.5 day)**
   - Read arxiv 2509.15969 carefully
   - Document the exact mechanism: phoneme-transformer, monotonic
     alignment, dynamic look-ahead
   - Resolve open question (1) above

3. **Phoneme hint integration in Generator (~2 days)**
   - Decide format (inline / segment / new prompt slot)
   - Modify `prompt::build_prompt` to accept phoneme hints
   - Test with a known phrase: does CSM produce more natural prosody
     with phoneme hints?

4. **Look-ahead window in Converse::run_streaming (~1 day)**
   - Maintain a sliding phoneme buffer
   - Trigger phonemize on word boundary in the LLM stream
   - Pass updated buffer to Generator on each TTS call

5. **A/B bench (~1 day)**
   - Same `converse_server_bench` setup, with vs without look-ahead
   - Measure: sentence[0] tts_gen, llm_to_first_audio, client TTFA
   - If win is < 100 ms on M-series, deprioritize and document

6. **Wire into converse_server (~1 day, if perf justified)**
   - New `--lookahead-words N` flag on converse_server
   - Default 0 (off); recommend 5 for production English
   - Update `docs/perf_history.md`

## Alternative if VoXtream mechanism is too speculative

If the paper's exact mechanism turns out to require model retraining
(e.g., a phoneme cross-attention head), we have a simpler win
available: **streaming-Mimi reduction in chunk_frames** from 4 to 2.
Empirically measured this in Phase 6c.3a; current default 4 frames =
320 ms latency. Drop to 2 = 160 ms but adds decode overhead (Mimi has
per-call setup cost).

That's a 1-line change, no phonemizer needed. Worth measuring before
committing to the full VoXtream port.

## Recommended order of attack for the next session

1. Drop `chunk_frames=4` to `chunk_frames=2` and `chunk_frames=1` —
   measure first-chunk latency. If win ≥ 200 ms, ship it as Phase 9.0
   and skip the phonemizer port entirely.
2. If chunk_frames tuning isn't enough, then implement the espeak-ng
   wrapper (task 1 above) — bounded 0.5-day spike.
3. Read the VoXtream paper carefully (task 2). DO NOT IMPLEMENT until
   the mechanism is clear from the paper, not the abstract.
4. Iterate per the task list above.

## Cited sources

- Paper: arXiv 2509.15969 — VoXtream: Full-Stream TTS with Extremely Low Latency
- Phonemizer crates verified on crates.io (Phase 9.1 spike, agent ID `<not_returned_this_session>`)
- Production baseline numbers: `docs/perf_history.md` Phase 8.11 row