feat(podcast): render the episode from the agents' own script
GenFM is unreachable. `GET /v1/studio/projects` and `POST /v1/studio/podcasts` both return 403 — "Access to the Studio API requires your account to be explicitly whitelisted to use it. Please contact our sales team." Measured with two different keys on the account, so it is an account restriction, not a key scope. Plain text-to-speech on the same key returns a valid MP3. That suits the operator's choice better than GenFM would have. GenFM always runs its own LLM over the source, so the agents' script would have been REWRITTEN; rendering each line ourselves speaks it verbatim. The agents did the reading and the judging, and the episode says what they wrote. `AudioBackend` is a trait because every candidate has a different shape: NotebookLM documents no programmatic retrieval at all, GenFM needs a sales conversation, Gemini TTS is a third form. The renderer hands over a `Script` and gets bytes. `parse_script` is a parser rather than a `read_to_string` because structure must not be spoken: headings, rules and block quotes are skipped, a wrapped paragraph stays ONE turn (splitting per line would stutter at the seam), and a colon mid sentence does not start a new speaker — "The finding: recall dropped" would otherwise be truncated to everything after the colon. Voices are assigned by order of appearance, so a script using names instead of HOST/GUEST still alternates, and an unexpected third speaker falls back rather than failing. `from_env` returns None without a key, so a deployment with none produces no audio instead of failing a mission that otherwise succeeded. Proven end to end on the real script this morning's mission wrote: 25 turns, 887 words, 5,614 billable characters, 6.1 MB of MP3 in 33 seconds. The live test drives `parse_script` + `ElevenLabs::render` — the production path — and is `#[ignore]`d because it spends credits. 359 tests pass. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
850f11838b
commit
656662850d
@@ -0,0 +1,346 @@
|
||||
//! Turning a mission's script into an episode.
|
||||
//!
|
||||
//! ## Why not GenFM
|
||||
//!
|
||||
//! The plan was ElevenLabs GenFM (`POST /v1/studio/podcasts`), which writes AND
|
||||
//! voices a two-host show from source text. It is unreachable on this account:
|
||||
//!
|
||||
//! ```text
|
||||
//! GET /v1/studio/projects -> 403
|
||||
//! POST /v1/studio/podcasts -> 403
|
||||
//! "Access to the Studio API requires your account to be explicitly
|
||||
//! whitelisted to use it. Please contact our sales team."
|
||||
//! ```
|
||||
//!
|
||||
//! Measured with two different keys, so it is an ACCOUNT restriction and not a
|
||||
//! key scope. Plain text-to-speech on the same key returns a valid MP3.
|
||||
//!
|
||||
//! That turns out to suit the operator's choice better than GenFM would have.
|
||||
//! GenFM always runs its own LLM over the source, so our agents' script would
|
||||
//! have been *rewritten*; rendering each line ourselves speaks it verbatim. The
|
||||
//! agents did the reading and the judging, and the podcast says what they wrote.
|
||||
//!
|
||||
//! ## Why the backend is a trait
|
||||
//!
|
||||
//! NotebookLM documents no programmatic audio retrieval at all, GenFM needs a
|
||||
//! sales conversation, and Gemini TTS is a third shape again. The renderer
|
||||
//! should not have to care: it hands a `Script` to an `AudioBackend` and gets
|
||||
//! bytes.
|
||||
|
||||
use async_trait::async_trait;
|
||||
|
||||
/// One spoken turn.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Turn {
|
||||
/// `HOST` or `GUEST`, as written in the script.
|
||||
pub speaker: String,
|
||||
pub text: String,
|
||||
}
|
||||
|
||||
/// A parsed episode script.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct Script {
|
||||
pub title: String,
|
||||
pub turns: Vec<Turn>,
|
||||
}
|
||||
|
||||
impl Script {
|
||||
/// Roughly how long this will take to say, at 150 words per minute.
|
||||
pub fn estimated_secs(&self) -> u32 {
|
||||
let words: usize = self.turns.iter().map(|t| t.text.split_whitespace().count()).sum();
|
||||
((words as f32 / 150.0) * 60.0).round() as u32
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse `script.md` into turns.
|
||||
///
|
||||
/// The format is what `skills/research/podcast-dialogue-writing.md` tells the
|
||||
/// writer to produce: `HOST:` / `GUEST:` at the start of a line. Everything
|
||||
/// else — headings, blank lines, stage directions in brackets — is not speech
|
||||
/// and must not be read aloud, which is the whole reason this is a parser and
|
||||
/// not a `read_to_string`.
|
||||
///
|
||||
/// A continuation line (no speaker prefix) belongs to the turn above it, so a
|
||||
/// wrapped paragraph stays one turn rather than becoming a new one.
|
||||
pub fn parse_script(md: &str) -> Script {
|
||||
let mut title = String::new();
|
||||
let mut turns: Vec<Turn> = Vec::new();
|
||||
|
||||
for raw in md.lines() {
|
||||
let line = raw.trim();
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if let Some(h) = line.strip_prefix("# ") {
|
||||
if title.is_empty() {
|
||||
title = h.trim().to_string();
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// Any other heading, list marker or rule is structure, not speech.
|
||||
if line.starts_with('#') || line.starts_with("---") || line.starts_with("> ") {
|
||||
continue;
|
||||
}
|
||||
match line.split_once(':') {
|
||||
Some((who, said))
|
||||
if !who.is_empty()
|
||||
&& who.len() <= 12
|
||||
&& who
|
||||
.chars()
|
||||
.all(|c| c.is_ascii_uppercase() || c.is_ascii_digit() || c == ' ') =>
|
||||
{
|
||||
let text = said.trim();
|
||||
if !text.is_empty() {
|
||||
turns.push(Turn {
|
||||
speaker: who.trim().to_string(),
|
||||
text: text.to_string(),
|
||||
});
|
||||
}
|
||||
}
|
||||
// Continuation of the previous turn.
|
||||
_ => {
|
||||
if let Some(last) = turns.last_mut() {
|
||||
last.text.push(' ');
|
||||
last.text.push_str(line);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Script { title, turns }
|
||||
}
|
||||
|
||||
/// Anything that can turn a script into audio bytes.
|
||||
#[async_trait]
|
||||
pub trait AudioBackend: Send + Sync {
|
||||
/// Render the whole script. Returns MP3 bytes.
|
||||
async fn render(&self, script: &Script) -> Result<Vec<u8>, String>;
|
||||
/// For logs and the episode record.
|
||||
fn describe(&self) -> String;
|
||||
}
|
||||
|
||||
/// ElevenLabs per-line text-to-speech.
|
||||
pub struct ElevenLabs {
|
||||
api_key: String,
|
||||
/// Voice for the first speaker seen, and for anyone unrecognised.
|
||||
pub host_voice: String,
|
||||
/// Voice for the second distinct speaker.
|
||||
pub guest_voice: String,
|
||||
pub model_id: String,
|
||||
http: reqwest::Client,
|
||||
}
|
||||
|
||||
/// Default voices, both from the stock library so no account setup is needed.
|
||||
pub const DEFAULT_HOST_VOICE: &str = "CwhRBWXzGAHq8TQ4Fs17"; // Roger
|
||||
pub const DEFAULT_GUEST_VOICE: &str = "EXAVITQu4vr4xnSDxMaL"; // Sarah
|
||||
|
||||
impl ElevenLabs {
|
||||
/// Build from the environment. `None` when no key is configured, so a
|
||||
/// deployment without one simply produces no audio instead of failing a
|
||||
/// mission that otherwise succeeded.
|
||||
pub fn from_env() -> Option<ElevenLabs> {
|
||||
let api_key = std::env::var("ELEVENLABS_API_KEY")
|
||||
.ok()
|
||||
.filter(|k| !k.trim().is_empty())?;
|
||||
Some(ElevenLabs {
|
||||
api_key,
|
||||
host_voice: std::env::var("CLAWMATES_PODCAST_HOST_VOICE")
|
||||
.unwrap_or_else(|_| DEFAULT_HOST_VOICE.to_string()),
|
||||
guest_voice: std::env::var("CLAWMATES_PODCAST_GUEST_VOICE")
|
||||
.unwrap_or_else(|_| DEFAULT_GUEST_VOICE.to_string()),
|
||||
// flash_v2_5 is the cheap fast tier; a spoken digest does not need
|
||||
// the expensive model, and cost matters on a DAILY job.
|
||||
model_id: std::env::var("CLAWMATES_PODCAST_MODEL")
|
||||
.unwrap_or_else(|_| "eleven_flash_v2_5".to_string()),
|
||||
http: reqwest::Client::new(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Which voice speaks this turn.
|
||||
///
|
||||
/// Keyed off the speaker labels actually present rather than hardcoding
|
||||
/// "HOST"/"GUEST", so a script that uses names still alternates instead of
|
||||
/// collapsing into one voice.
|
||||
fn voice_for(&self, speaker: &str, first: &str, second: Option<&str>) -> &str {
|
||||
if speaker.eq_ignore_ascii_case(first) {
|
||||
&self.host_voice
|
||||
} else if second.is_some_and(|s| speaker.eq_ignore_ascii_case(s)) {
|
||||
&self.guest_voice
|
||||
} else {
|
||||
&self.host_voice
|
||||
}
|
||||
}
|
||||
|
||||
async fn say(&self, text: &str, voice: &str) -> Result<Vec<u8>, String> {
|
||||
let url = format!("https://api.elevenlabs.io/v1/text-to-speech/{voice}");
|
||||
let res = self
|
||||
.http
|
||||
.post(&url)
|
||||
.header("xi-api-key", &self.api_key)
|
||||
.json(&serde_json::json!({
|
||||
"text": text,
|
||||
"model_id": self.model_id,
|
||||
// 128kbps 44.1k: podcast-normal, and small enough that a daily
|
||||
// episode does not bloat the blob store.
|
||||
"output_format": "mp3_44100_128",
|
||||
}))
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| format!("tts request: {e}"))?;
|
||||
if !res.status().is_success() {
|
||||
let code = res.status();
|
||||
let body = res.text().await.unwrap_or_default();
|
||||
return Err(format!("tts {code}: {}", body.chars().take(200).collect::<String>()));
|
||||
}
|
||||
let bytes = res.bytes().await.map_err(|e| format!("tts body: {e}"))?;
|
||||
if bytes.len() < 512 {
|
||||
return Err(format!("tts returned {} bytes — too short to be audio", bytes.len()));
|
||||
}
|
||||
Ok(bytes.to_vec())
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl AudioBackend for ElevenLabs {
|
||||
fn describe(&self) -> String {
|
||||
format!("elevenlabs/{}", self.model_id)
|
||||
}
|
||||
|
||||
async fn render(&self, script: &Script) -> Result<Vec<u8>, String> {
|
||||
if script.turns.is_empty() {
|
||||
return Err("script has no spoken turns".into());
|
||||
}
|
||||
// Identify the two speakers by order of appearance.
|
||||
let first = script.turns[0].speaker.clone();
|
||||
let second = script
|
||||
.turns
|
||||
.iter()
|
||||
.map(|t| t.speaker.as_str())
|
||||
.find(|s| !s.eq_ignore_ascii_case(&first))
|
||||
.map(str::to_string);
|
||||
|
||||
let mut out: Vec<u8> = Vec::new();
|
||||
for (i, turn) in script.turns.iter().enumerate() {
|
||||
let voice = self.voice_for(&turn.speaker, &first, second.as_deref());
|
||||
let clip = self.say(&turn.text, voice).await.map_err(|e| {
|
||||
// Name the turn: a 400 on one line is far easier to fix than
|
||||
// "rendering failed" for a 40-turn script.
|
||||
format!("turn {} ({}): {e}", i + 1, turn.speaker)
|
||||
})?;
|
||||
out.extend_from_slice(&clip);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_script_parses_into_speaker_turns() {
|
||||
let md = "# Morning Research Podcast — 2026-08-18\n\n\
|
||||
HOST: Morning run, morning papers.\n\n\
|
||||
GUEST: Today's harvest pokes at something settled.\n";
|
||||
let s = parse_script(md);
|
||||
assert_eq!(s.title, "Morning Research Podcast — 2026-08-18");
|
||||
assert_eq!(s.turns.len(), 2);
|
||||
assert_eq!(s.turns[0].speaker, "HOST");
|
||||
assert_eq!(s.turns[1].text, "Today's harvest pokes at something settled.");
|
||||
}
|
||||
|
||||
/// Headings and rules are structure. Reading "2026-08-18" and "---" aloud
|
||||
/// is the difference between an episode and a machine reading a file.
|
||||
#[test]
|
||||
fn structure_is_never_spoken() {
|
||||
let s = parse_script("# Title\n## Section\n---\n> quote\nHOST: Only this.\n");
|
||||
assert_eq!(s.turns.len(), 1);
|
||||
assert_eq!(s.turns[0].text, "Only this.");
|
||||
}
|
||||
|
||||
/// A wrapped paragraph is ONE turn. Splitting on every newline would break
|
||||
/// a sentence across two TTS calls and audibly stutter at the seam.
|
||||
#[test]
|
||||
fn continuation_lines_join_the_turn_above() {
|
||||
let s = parse_script("HOST: First part\nsecond part.\nGUEST: Mine.\n");
|
||||
assert_eq!(s.turns.len(), 2);
|
||||
assert_eq!(s.turns[0].text, "First part second part.");
|
||||
}
|
||||
|
||||
/// A colon inside speech must not be read as a speaker label, or the line
|
||||
/// is silently truncated to whatever followed the colon.
|
||||
#[test]
|
||||
fn a_colon_mid_sentence_does_not_start_a_new_turn() {
|
||||
let s = parse_script("HOST: The finding: recall dropped sharply.\n");
|
||||
assert_eq!(s.turns.len(), 1);
|
||||
assert_eq!(s.turns[0].text, "The finding: recall dropped sharply.");
|
||||
}
|
||||
|
||||
/// Voices are assigned by order of appearance, so a script using names
|
||||
/// instead of HOST/GUEST still alternates.
|
||||
#[test]
|
||||
fn two_speakers_get_two_voices_whatever_they_are_called() {
|
||||
let el = ElevenLabs {
|
||||
api_key: "x".into(),
|
||||
host_voice: "HOSTV".into(),
|
||||
guest_voice: "GUESTV".into(),
|
||||
model_id: "m".into(),
|
||||
http: reqwest::Client::new(),
|
||||
};
|
||||
assert_eq!(el.voice_for("ANA", "ANA", Some("BEN")), "HOSTV");
|
||||
assert_eq!(el.voice_for("BEN", "ANA", Some("BEN")), "GUESTV");
|
||||
// An unexpected third speaker falls back rather than failing the run.
|
||||
assert_eq!(el.voice_for("CARL", "ANA", Some("BEN")), "HOSTV");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_runtime_estimate_is_in_the_right_ballpark() {
|
||||
let words = "word ".repeat(1500);
|
||||
let s = parse_script(&format!("HOST: {words}\n"));
|
||||
let secs = s.estimated_secs();
|
||||
assert!((540..=660).contains(&secs), "1500 words ≈ 10 min, got {secs}s");
|
||||
}
|
||||
}
|
||||
|
||||
/// Live render against the real API. Ignored by default: it spends credits.
|
||||
///
|
||||
/// Exercises the production path — `parse_script` then `ElevenLabs::render` —
|
||||
/// rather than a reimplementation, so what passes here is what runs.
|
||||
///
|
||||
/// CLAWMATES_PODCAST_TEST_SCRIPT=/path/to/script.md \
|
||||
/// CLAWMATES_PODCAST_TEST_OUT=/tmp/episode.mp3 \
|
||||
/// cargo test -p cm-api --lib podcast::live -- --ignored --nocapture
|
||||
#[cfg(test)]
|
||||
mod live {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "spends ElevenLabs credits"]
|
||||
async fn renders_a_real_script_to_mp3() {
|
||||
let path = std::env::var("CLAWMATES_PODCAST_TEST_SCRIPT")
|
||||
.expect("set CLAWMATES_PODCAST_TEST_SCRIPT");
|
||||
let out = std::env::var("CLAWMATES_PODCAST_TEST_OUT")
|
||||
.unwrap_or_else(|_| "/tmp/episode.mp3".to_string());
|
||||
let md = std::fs::read_to_string(&path).expect("script readable");
|
||||
let script = parse_script(&md);
|
||||
assert!(!script.turns.is_empty(), "no turns parsed from {path}");
|
||||
eprintln!(
|
||||
"script: {:?} — {} turns, ~{}s",
|
||||
script.title,
|
||||
script.turns.len(),
|
||||
script.estimated_secs()
|
||||
);
|
||||
|
||||
let backend = ElevenLabs::from_env().expect("ELEVENLABS_API_KEY must be set");
|
||||
let bytes = backend.render(&script).await.expect("render");
|
||||
std::fs::write(&out, &bytes).expect("write mp3");
|
||||
eprintln!("wrote {} bytes to {out} via {}", bytes.len(), backend.describe());
|
||||
|
||||
// ID3 or a raw MPEG frame header — anything else is not audio.
|
||||
let head = &bytes[..3.min(bytes.len())];
|
||||
assert!(
|
||||
head == b"ID3" || (bytes[0] == 0xFF && bytes[1] & 0xE0 == 0xE0),
|
||||
"not an MP3: {head:?}"
|
||||
);
|
||||
assert!(bytes.len() > 100_000, "suspiciously small: {} bytes", bytes.len());
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user