Omar SobhandClaude Opus 5
f4adc8d0f9
Two things the operator found by listening to a real episode.
**1. Identifiers were spoken as digit soup.** The script genuinely said
"arxiv 2608.12888", which the voice reads as "two six zero eight point one two
eight eight eight". Same for three-decimal values: "0.506" and "0.004" became
long strings of spoken digits. A listener on a treadmill cannot write an
identifier down and does not need a third decimal place.
`speakable()` strips arXiv references and bare identifier-shaped numbers, and
rounds decimals to two places — with a carve-out that matters: 0.004 rounds to
0.00, which would claim the value was ZERO when the whole point was that it
collapsed to nearly nothing, so it says "under 0.01" instead.
Deliberately narrow: it removes identifiers and shortens over-precise decimals,
and does not paraphrase, reorder or summarise. The agents' words are still the
episode. It also preserves the sentence's full stop — swallowing it turned
"…financial retrieval, arxiv 2608.00183. This one's a catch." into one run-on
sentence, and the pause is how a listener knows a thought ended.
Note that `podcast-dialogue-writing.md` ALREADY said "no arXiv ids" and the
writer included them anyway. That is this project's recurring lesson restated:
an instruction is a request, and a listener deserves a guarantee. The prose asks
and the code enforces.
**2. It was written for someone who already knew the field.** The skill and the
script phase's task now target a bright sixteen-year-old: define an acronym in
the sentence that first uses it, describe the mechanism rather than naming it
("a road map with motorways and side streets" instead of "a hierarchical
navigable small world graph"), one idea per sentence. The test offered is
whether the listener could explain the finding to a friend afterwards.
That is not dumbing down — it is the constraint that forces a writer to say what
a thing actually does rather than what it is called.
Tested against the exact lines from the episode that was listened to.
366 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
887 lines
34 KiB
Rust
887 lines
34 KiB
Rust
//! Turning a mission's script into an episode.
|
|
//!
|
|
//! ## Why not GenFM
|
|
//!
|
|
//! The plan was ElevenLabs GenFM (`POST /v1/studio/podcasts`), which writes AND
|
|
//! voices a two-host show from source text. It is unreachable on this account:
|
|
//!
|
|
//! ```text
|
|
//! GET /v1/studio/projects -> 403
|
|
//! POST /v1/studio/podcasts -> 403
|
|
//! "Access to the Studio API requires your account to be explicitly
|
|
//! whitelisted to use it. Please contact our sales team."
|
|
//! ```
|
|
//!
|
|
//! Measured with two different keys, so it is an ACCOUNT restriction and not a
|
|
//! key scope. Plain text-to-speech on the same key returns a valid MP3.
|
|
//!
|
|
//! That turns out to suit the operator's choice better than GenFM would have.
|
|
//! GenFM always runs its own LLM over the source, so our agents' script would
|
|
//! have been *rewritten*; rendering each line ourselves speaks it verbatim. The
|
|
//! agents did the reading and the judging, and the podcast says what they wrote.
|
|
//!
|
|
//! ## Why the backend is a trait
|
|
//!
|
|
//! NotebookLM documents no programmatic audio retrieval at all, GenFM needs a
|
|
//! sales conversation, and Gemini TTS is a third shape again. The renderer
|
|
//! should not have to care: it hands a `Script` to an `AudioBackend` and gets
|
|
//! bytes.
|
|
|
|
use async_trait::async_trait;
|
|
|
|
/// One spoken turn.
|
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
pub struct Turn {
|
|
/// `HOST` or `GUEST`, as written in the script.
|
|
pub speaker: String,
|
|
pub text: String,
|
|
}
|
|
|
|
/// A parsed episode script.
|
|
#[derive(Debug, Clone, Default)]
|
|
pub struct Script {
|
|
pub title: String,
|
|
pub turns: Vec<Turn>,
|
|
}
|
|
|
|
impl Script {
|
|
/// Roughly how long this will take to say, at 150 words per minute.
|
|
pub fn estimated_secs(&self) -> u32 {
|
|
let words: usize = self.turns.iter().map(|t| t.text.split_whitespace().count()).sum();
|
|
((words as f32 / 150.0) * 60.0).round() as u32
|
|
}
|
|
}
|
|
|
|
/// Parse `script.md` into turns.
|
|
///
|
|
/// The format is what `skills/research/podcast-dialogue-writing.md` tells the
|
|
/// writer to produce: `HOST:` / `GUEST:` at the start of a line. Everything
|
|
/// else — headings, blank lines, stage directions in brackets — is not speech
|
|
/// and must not be read aloud, which is the whole reason this is a parser and
|
|
/// not a `read_to_string`.
|
|
///
|
|
/// A continuation line (no speaker prefix) belongs to the turn above it, so a
|
|
/// wrapped paragraph stays one turn rather than becoming a new one.
|
|
pub fn parse_script(md: &str) -> Script {
|
|
let mut title = String::new();
|
|
let mut turns: Vec<Turn> = Vec::new();
|
|
|
|
for raw in md.lines() {
|
|
let line = raw.trim();
|
|
if line.is_empty() {
|
|
continue;
|
|
}
|
|
if let Some(h) = line.strip_prefix("# ") {
|
|
if title.is_empty() {
|
|
title = h.trim().to_string();
|
|
}
|
|
continue;
|
|
}
|
|
// Any other heading, list marker or rule is structure, not speech.
|
|
if line.starts_with('#') || line.starts_with("---") || line.starts_with("> ") {
|
|
continue;
|
|
}
|
|
match line.split_once(':') {
|
|
Some((who, said))
|
|
if !who.is_empty()
|
|
&& who.len() <= 12
|
|
&& who
|
|
.chars()
|
|
.all(|c| c.is_ascii_uppercase() || c.is_ascii_digit() || c == ' ') =>
|
|
{
|
|
let text = said.trim();
|
|
if !text.is_empty() {
|
|
turns.push(Turn {
|
|
speaker: who.trim().to_string(),
|
|
text: text.to_string(),
|
|
});
|
|
}
|
|
}
|
|
// Continuation of the previous turn.
|
|
_ => {
|
|
if let Some(last) = turns.last_mut() {
|
|
last.text.push(' ');
|
|
last.text.push_str(line);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
Script { title, turns }
|
|
}
|
|
|
|
|
|
/// MPEG1 Layer III bitrates (kbps) and sample rates, indexed as the frame
|
|
/// header encodes them.
|
|
const MP3_BITRATES: [u32; 16] = [
|
|
0, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 0,
|
|
];
|
|
const MP3_RATES: [u32; 4] = [44100, 48000, 32000, 0];
|
|
|
|
/// Strip a clip's container metadata so clips can be joined into ONE stream.
|
|
///
|
|
/// This is the difference between an episode and a six-second file. Each TTS
|
|
/// clip arrives as a standalone MP3: a small ID3v2 tag, then a first frame
|
|
/// carrying an `Info`/`Xing` VBR header that declares THAT CLIP's frame count.
|
|
/// Concatenated raw, a player reads the first clip's header, believes the whole
|
|
/// file is that long, and stops. Measured on two real clips of 4.86s and 4.68s:
|
|
///
|
|
/// ```text
|
|
/// raw concat -> 4.86s (only clip one plays)
|
|
/// strip second clip's ID3 -> 4.86s
|
|
/// strip both clips' ID3 -> 4.86s (the tag was never the issue)
|
|
/// strip ID3 *and* the Info frame -> 9.53s correct
|
|
/// ```
|
|
///
|
|
/// The ID3 tag is ~45 bytes and harmless; the header FRAME is what lies. Both
|
|
/// go, leaving pure audio frames that a player times from the stream itself.
|
|
fn strip_container(clip: &[u8]) -> &[u8] {
|
|
let mut i = 0usize;
|
|
// ID3v2: 10-byte header, then a syncsafe 28-bit size.
|
|
if clip.len() > 10 && &clip[..3] == b"ID3" {
|
|
let size = ((clip[6] as usize) << 21)
|
|
| ((clip[7] as usize) << 14)
|
|
| ((clip[8] as usize) << 7)
|
|
| (clip[9] as usize);
|
|
i = (10 + size).min(clip.len());
|
|
}
|
|
// A leading Xing/Info frame is metadata, not sound.
|
|
if i + 4 < clip.len() && clip[i] == 0xFF && clip[i + 1] & 0xE0 == 0xE0 {
|
|
let br = MP3_BITRATES[((clip[i + 2] >> 4) & 0x0F) as usize];
|
|
let sr = MP3_RATES[((clip[i + 2] >> 2) & 0x03) as usize];
|
|
if br > 0 && sr > 0 {
|
|
let pad = ((clip[i + 2] >> 1) & 1) as usize;
|
|
let len = (144 * br as usize * 1000 / sr as usize) + pad;
|
|
let end = (i + len).min(clip.len());
|
|
let frame = &clip[i..end];
|
|
if find(frame, b"Xing").is_some() || find(frame, b"Info").is_some() {
|
|
i = end;
|
|
}
|
|
}
|
|
}
|
|
&clip[i..]
|
|
}
|
|
|
|
fn find(hay: &[u8], needle: &[u8]) -> Option<usize> {
|
|
hay.windows(needle.len()).position(|w| w == needle)
|
|
}
|
|
|
|
|
|
/// Rewrite a line so it is worth HEARING.
|
|
///
|
|
/// Written from a real episode the operator listened to. Two things ruined it,
|
|
/// and neither is a TTS defect — the text genuinely said them:
|
|
///
|
|
/// ```text
|
|
/// "This is the ReFind paper, arxiv 2608.12888."
|
|
/// -> "two six zero eight point one two eight eight eight"
|
|
/// "BM25 recall dropped from 0.506 native to 0.004 cross-lingual"
|
|
/// -> "zero point five zero six ... zero point zero zero four"
|
|
/// ```
|
|
///
|
|
/// A listener on a treadmill cannot write an identifier down and does not need
|
|
/// three decimal places. `skills/research/podcast-dialogue-writing.md` already
|
|
/// told the writer not to include arXiv ids and it included them anyway — which
|
|
/// is the lesson of this whole project restated: an instruction is a request,
|
|
/// and a listener deserves a guarantee. So the prose asks and this enforces.
|
|
///
|
|
/// Deliberately narrow. It removes identifiers and shortens over-precise
|
|
/// decimals; it does not paraphrase, reorder or summarise. The agents' words
|
|
/// are still the episode.
|
|
pub fn speakable(line: &str) -> String {
|
|
let mut out = String::with_capacity(line.len());
|
|
let b: Vec<char> = line.chars().collect();
|
|
let mut i = 0usize;
|
|
|
|
while i < b.len() {
|
|
// "arXiv:2608.12888", "arxiv 2608.12888", "arXiv 2608.12888v2"
|
|
if starts_with_ci(&b, i, "arxiv") {
|
|
let mut j = i + 5;
|
|
while j < b.len() && (b[j] == ':' || b[j] == ' ' || b[j] == '.') {
|
|
j += 1;
|
|
}
|
|
let digits_start = j;
|
|
while j < b.len() && (b[j].is_ascii_digit() || b[j] == '.' || b[j] == 'v') {
|
|
j += 1;
|
|
}
|
|
// Do not swallow the sentence's full stop. "…retrieval, arxiv
|
|
// 2608.00183. This one's a catch." must not become one run-on
|
|
// sentence — the pause is how a listener knows a thought ended.
|
|
while j > digits_start && !b[j - 1].is_ascii_digit() {
|
|
j -= 1;
|
|
}
|
|
if j > digits_start + 4 {
|
|
// Drop the whole reference, and any comma or space it left
|
|
// dangling: "the ReFind paper, arxiv 2608.12888." must not
|
|
// become "the ReFind paper, ."
|
|
trim_trailing_separator(&mut out);
|
|
i = j;
|
|
skip_leading_separator(&b, &mut i);
|
|
continue;
|
|
}
|
|
}
|
|
|
|
// A bare arXiv-shaped number: 4 digits, dot, 4-5 digits.
|
|
if b[i].is_ascii_digit() {
|
|
let start = i;
|
|
let mut j = i;
|
|
while j < b.len() && b[j].is_ascii_digit() {
|
|
j += 1;
|
|
}
|
|
let int_len = j - start;
|
|
if j < b.len() && b[j] == '.' {
|
|
let frac_start = j + 1;
|
|
let mut k = frac_start;
|
|
while k < b.len() && b[k].is_ascii_digit() {
|
|
k += 1;
|
|
}
|
|
let frac_len = k - frac_start;
|
|
if int_len == 4 && (4..=5).contains(&frac_len) {
|
|
// An identifier, not a quantity.
|
|
trim_trailing_separator(&mut out);
|
|
i = k;
|
|
skip_leading_separator(&b, &mut i);
|
|
continue;
|
|
}
|
|
if frac_len >= 3 {
|
|
// Over-precise. Nobody hears the third decimal place.
|
|
let text: String = b[start..k].iter().collect();
|
|
out.push_str(&round_decimal(&text));
|
|
i = k;
|
|
continue;
|
|
}
|
|
}
|
|
}
|
|
out.push(b[i]);
|
|
i += 1;
|
|
}
|
|
// Collapse any double spaces a removal left behind.
|
|
let collapsed = out.split_whitespace().collect::<Vec<_>>().join(" ");
|
|
collapsed
|
|
.replace(" ,", ",")
|
|
.replace(" .", ".")
|
|
.replace("( )", "")
|
|
.replace("()", "")
|
|
}
|
|
|
|
fn starts_with_ci(b: &[char], i: usize, word: &str) -> bool {
|
|
let w: Vec<char> = word.chars().collect();
|
|
if i + w.len() > b.len() {
|
|
return false;
|
|
}
|
|
b[i..i + w.len()]
|
|
.iter()
|
|
.zip(&w)
|
|
.all(|(a, c)| a.to_ascii_lowercase() == *c)
|
|
}
|
|
|
|
fn trim_trailing_separator(out: &mut String) {
|
|
while out.ends_with(' ') || out.ends_with(',') || out.ends_with('(') {
|
|
out.pop();
|
|
}
|
|
}
|
|
|
|
fn skip_leading_separator(b: &[char], i: &mut usize) {
|
|
while *i < b.len() && (b[*i] == ')' || b[*i] == ',') {
|
|
*i += 1;
|
|
}
|
|
}
|
|
|
|
/// Two decimal places, or "under 0.01" when rounding would say "0.00".
|
|
///
|
|
/// `0.004` rounded to two places is `0.00`, which is worse than the original:
|
|
/// it says the value is zero when the point was that it collapsed to nearly
|
|
/// nothing.
|
|
fn round_decimal(text: &str) -> String {
|
|
let Ok(v) = text.parse::<f64>() else {
|
|
return text.to_string();
|
|
};
|
|
let r = (v * 100.0).round() / 100.0;
|
|
if r == 0.0 && v != 0.0 {
|
|
return "under 0.01".to_string();
|
|
}
|
|
let s = format!("{r:.2}");
|
|
s.trim_end_matches('0').trim_end_matches('.').to_string()
|
|
}
|
|
|
|
/// Anything that can turn a script into audio bytes.
|
|
#[async_trait]
|
|
pub trait AudioBackend: Send + Sync {
|
|
/// Render the whole script. Returns MP3 bytes.
|
|
async fn render(&self, script: &Script) -> Result<Vec<u8>, String>;
|
|
/// For logs and the episode record.
|
|
fn describe(&self) -> String;
|
|
}
|
|
|
|
/// ElevenLabs per-line text-to-speech.
|
|
pub struct ElevenLabs {
|
|
api_key: String,
|
|
/// Voice for the first speaker seen, and for anyone unrecognised.
|
|
pub host_voice: String,
|
|
/// Voice for the second distinct speaker.
|
|
pub guest_voice: String,
|
|
pub model_id: String,
|
|
http: reqwest::Client,
|
|
}
|
|
|
|
/// Default voices, both from the stock library so no account setup is needed.
|
|
pub const DEFAULT_HOST_VOICE: &str = "CwhRBWXzGAHq8TQ4Fs17"; // Roger
|
|
pub const DEFAULT_GUEST_VOICE: &str = "EXAVITQu4vr4xnSDxMaL"; // Sarah
|
|
|
|
impl ElevenLabs {
|
|
/// Build from the environment. `None` when no key is configured, so a
|
|
/// deployment without one simply produces no audio instead of failing a
|
|
/// mission that otherwise succeeded.
|
|
pub fn from_env() -> Option<ElevenLabs> {
|
|
let api_key = std::env::var("ELEVENLABS_API_KEY")
|
|
.ok()
|
|
.filter(|k| !k.trim().is_empty())?;
|
|
Some(ElevenLabs {
|
|
api_key,
|
|
host_voice: std::env::var("CLAWMATES_PODCAST_HOST_VOICE")
|
|
.unwrap_or_else(|_| DEFAULT_HOST_VOICE.to_string()),
|
|
guest_voice: std::env::var("CLAWMATES_PODCAST_GUEST_VOICE")
|
|
.unwrap_or_else(|_| DEFAULT_GUEST_VOICE.to_string()),
|
|
// flash_v2_5 is the cheap fast tier; a spoken digest does not need
|
|
// the expensive model, and cost matters on a DAILY job.
|
|
model_id: std::env::var("CLAWMATES_PODCAST_MODEL")
|
|
.unwrap_or_else(|_| "eleven_flash_v2_5".to_string()),
|
|
http: reqwest::Client::new(),
|
|
})
|
|
}
|
|
|
|
/// Which voice speaks this turn.
|
|
///
|
|
/// Keyed off the speaker labels actually present rather than hardcoding
|
|
/// "HOST"/"GUEST", so a script that uses names still alternates instead of
|
|
/// collapsing into one voice.
|
|
fn voice_for(&self, speaker: &str, first: &str, second: Option<&str>) -> &str {
|
|
if speaker.eq_ignore_ascii_case(first) {
|
|
&self.host_voice
|
|
} else if second.is_some_and(|s| speaker.eq_ignore_ascii_case(s)) {
|
|
&self.guest_voice
|
|
} else {
|
|
&self.host_voice
|
|
}
|
|
}
|
|
|
|
async fn say(&self, text: &str, voice: &str) -> Result<Vec<u8>, String> {
|
|
let url = format!("https://api.elevenlabs.io/v1/text-to-speech/{voice}");
|
|
let res = self
|
|
.http
|
|
.post(&url)
|
|
.header("xi-api-key", &self.api_key)
|
|
.json(&serde_json::json!({
|
|
"text": text,
|
|
"model_id": self.model_id,
|
|
// 128kbps 44.1k: podcast-normal, and small enough that a daily
|
|
// episode does not bloat the blob store.
|
|
"output_format": "mp3_44100_128",
|
|
}))
|
|
.send()
|
|
.await
|
|
.map_err(|e| format!("tts request: {e}"))?;
|
|
if !res.status().is_success() {
|
|
let code = res.status();
|
|
let body = res.text().await.unwrap_or_default();
|
|
return Err(format!("tts {code}: {}", body.chars().take(200).collect::<String>()));
|
|
}
|
|
let bytes = res.bytes().await.map_err(|e| format!("tts body: {e}"))?;
|
|
if bytes.len() < 512 {
|
|
return Err(format!("tts returned {} bytes — too short to be audio", bytes.len()));
|
|
}
|
|
Ok(bytes.to_vec())
|
|
}
|
|
}
|
|
|
|
#[async_trait]
|
|
impl AudioBackend for ElevenLabs {
|
|
fn describe(&self) -> String {
|
|
format!("elevenlabs/{}", self.model_id)
|
|
}
|
|
|
|
async fn render(&self, script: &Script) -> Result<Vec<u8>, String> {
|
|
if script.turns.is_empty() {
|
|
return Err("script has no spoken turns".into());
|
|
}
|
|
// Identify the two speakers by order of appearance.
|
|
let first = script.turns[0].speaker.clone();
|
|
let second = script
|
|
.turns
|
|
.iter()
|
|
.map(|t| t.speaker.as_str())
|
|
.find(|s| !s.eq_ignore_ascii_case(&first))
|
|
.map(str::to_string);
|
|
|
|
let mut out: Vec<u8> = Vec::new();
|
|
for (i, turn) in script.turns.iter().enumerate() {
|
|
let voice = self.voice_for(&turn.speaker, &first, second.as_deref());
|
|
let clip = self.say(&speakable(&turn.text), voice).await.map_err(|e| {
|
|
// Name the turn: a 400 on one line is far easier to fix than
|
|
// "rendering failed" for a 40-turn script.
|
|
format!("turn {} ({}): {e}", i + 1, turn.speaker)
|
|
})?;
|
|
// Join as ONE stream: see `strip_container`. Concatenating whole
|
|
// MP3 files yields a file that plays only its first clip.
|
|
out.extend_from_slice(strip_container(&clip));
|
|
}
|
|
Ok(out)
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn a_script_parses_into_speaker_turns() {
|
|
let md = "# Morning Research Podcast — 2026-08-18\n\n\
|
|
HOST: Morning run, morning papers.\n\n\
|
|
GUEST: Today's harvest pokes at something settled.\n";
|
|
let s = parse_script(md);
|
|
assert_eq!(s.title, "Morning Research Podcast — 2026-08-18");
|
|
assert_eq!(s.turns.len(), 2);
|
|
assert_eq!(s.turns[0].speaker, "HOST");
|
|
assert_eq!(s.turns[1].text, "Today's harvest pokes at something settled.");
|
|
}
|
|
|
|
/// Headings and rules are structure. Reading "2026-08-18" and "---" aloud
|
|
/// is the difference between an episode and a machine reading a file.
|
|
#[test]
|
|
fn structure_is_never_spoken() {
|
|
let s = parse_script("# Title\n## Section\n---\n> quote\nHOST: Only this.\n");
|
|
assert_eq!(s.turns.len(), 1);
|
|
assert_eq!(s.turns[0].text, "Only this.");
|
|
}
|
|
|
|
/// A wrapped paragraph is ONE turn. Splitting on every newline would break
|
|
/// a sentence across two TTS calls and audibly stutter at the seam.
|
|
#[test]
|
|
fn continuation_lines_join_the_turn_above() {
|
|
let s = parse_script("HOST: First part\nsecond part.\nGUEST: Mine.\n");
|
|
assert_eq!(s.turns.len(), 2);
|
|
assert_eq!(s.turns[0].text, "First part second part.");
|
|
}
|
|
|
|
/// A colon inside speech must not be read as a speaker label, or the line
|
|
/// is silently truncated to whatever followed the colon.
|
|
#[test]
|
|
fn a_colon_mid_sentence_does_not_start_a_new_turn() {
|
|
let s = parse_script("HOST: The finding: recall dropped sharply.\n");
|
|
assert_eq!(s.turns.len(), 1);
|
|
assert_eq!(s.turns[0].text, "The finding: recall dropped sharply.");
|
|
}
|
|
|
|
/// Voices are assigned by order of appearance, so a script using names
|
|
/// instead of HOST/GUEST still alternates.
|
|
#[test]
|
|
fn two_speakers_get_two_voices_whatever_they_are_called() {
|
|
let el = ElevenLabs {
|
|
api_key: "x".into(),
|
|
host_voice: "HOSTV".into(),
|
|
guest_voice: "GUESTV".into(),
|
|
model_id: "m".into(),
|
|
http: reqwest::Client::new(),
|
|
};
|
|
assert_eq!(el.voice_for("ANA", "ANA", Some("BEN")), "HOSTV");
|
|
assert_eq!(el.voice_for("BEN", "ANA", Some("BEN")), "GUESTV");
|
|
// An unexpected third speaker falls back rather than failing the run.
|
|
assert_eq!(el.voice_for("CARL", "ANA", Some("BEN")), "HOSTV");
|
|
}
|
|
|
|
/// The bug that produced a six-second "episode".
|
|
///
|
|
/// Each clip is a standalone MP3 whose first frame carries an Info/Xing
|
|
/// header declaring that clip's length. Joined raw, a player reads clip
|
|
/// one's header and stops there. Fixtures are REAL ElevenLabs clips, so
|
|
/// this pins the actual wire format rather than a hand-built approximation.
|
|
#[test]
|
|
fn joining_strips_the_header_that_declares_one_clips_length() {
|
|
let clip = std::fs::read(concat!(env!("CARGO_MANIFEST_DIR"), "/tests/fixtures/tts-clip.mp3"));
|
|
let Ok(clip) = clip else {
|
|
eprintln!("fixture absent; skipping");
|
|
return;
|
|
};
|
|
assert_eq!(&clip[..3], b"ID3", "fixture should be a raw TTS clip");
|
|
let body = strip_container(&clip);
|
|
assert!(body.len() < clip.len(), "something must be stripped");
|
|
assert_eq!(body[0], 0xFF, "must start on a frame sync, got {:#04x}", body[0]);
|
|
assert!(body[1] & 0xE0 == 0xE0, "frame sync incomplete");
|
|
// The lying header must be gone from the head of the stream.
|
|
let head = &body[..body.len().min(1024)];
|
|
assert!(
|
|
find(head, b"Info").is_none() && find(head, b"Xing").is_none(),
|
|
"the VBR header frame survived — the join will report one clip's length"
|
|
);
|
|
}
|
|
|
|
/// Not an MP3, or a truncated one, must pass through rather than panic:
|
|
/// a bad clip should fail the render with a message, not crash the server.
|
|
#[test]
|
|
fn stripping_is_safe_on_junk() {
|
|
for junk in [&b""[..], &b"ID3"[..], &[0xFFu8][..], &b"not audio at all"[..]] {
|
|
let out = strip_container(junk);
|
|
assert!(out.len() <= junk.len());
|
|
}
|
|
}
|
|
|
|
/// Real lines from the episode the operator listened to. These are the
|
|
/// exact strings the TTS read aloud as digit soup.
|
|
/// Print what the operator's own episode WOULD have said. Not an
|
|
/// assertion — a way to read the diff on real input.
|
|
#[test]
|
|
fn show_real_script_lines() {
|
|
let Ok(md) = std::env::var("CLAWMATES_SPEAKABLE_DEMO") else { return };
|
|
let Ok(text) = std::fs::read_to_string(&md) else { return };
|
|
for line in text.lines() {
|
|
let out = speakable(line);
|
|
if out != line && !line.trim().is_empty() {
|
|
eprintln!(" BEFORE {}", line.trim());
|
|
eprintln!(" AFTER {}\n", out.trim());
|
|
}
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn identifiers_are_never_spoken() {
|
|
for (input, must_not) in [
|
|
("This is the ReFind paper, arxiv 2608.12888.", "2608"),
|
|
("There was an agricultural paper too, arXiv:2608.14886 — does it matter?", "14886"),
|
|
("evidence-unit fairness in financial retrieval, arxiv 2608.00183", "00183"),
|
|
] {
|
|
let out = speakable(input);
|
|
assert!(!out.contains(must_not), "{must_not} survived in {out:?}");
|
|
assert!(!out.to_lowercase().contains("arxiv"), "dangling label: {out:?}");
|
|
// The sentence must still read cleanly.
|
|
assert!(!out.contains(" ,"), "orphan comma: {out:?}");
|
|
assert!(!out.contains(",."), "orphan comma: {out:?}");
|
|
}
|
|
}
|
|
|
|
/// Three decimal places is data, not speech.
|
|
#[test]
|
|
fn over_precise_decimals_are_shortened() {
|
|
let out = speakable("BM25 recall dropped from 0.506 native to 0.004 cross-lingual.");
|
|
assert!(out.contains("0.51"), "0.506 should round: {out:?}");
|
|
assert!(!out.contains("0.506"), "{out:?}");
|
|
// 0.004 rounds to 0.00, which would claim the value was zero — the
|
|
// opposite of the point being made.
|
|
assert!(out.contains("under 0.01"), "{out:?}");
|
|
assert!(!out.contains("0.00 "), "must never say zero: {out:?}");
|
|
}
|
|
|
|
/// Two decimals, years, percentages and small integers are all fine spoken
|
|
/// and must survive untouched — over-processing would mangle the meaning.
|
|
#[test]
|
|
fn ordinary_numbers_are_left_alone() {
|
|
for s in [
|
|
"58.2 versus 53.2 mean accuracy",
|
|
"roughly 2,800 questions",
|
|
"21.8% of theoretical headroom",
|
|
"NDCG at 10 of 0.15",
|
|
"about 2026 papers",
|
|
] {
|
|
assert_eq!(speakable(s), s, "should be unchanged");
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn the_runtime_estimate_is_in_the_right_ballpark() {
|
|
let words = "word ".repeat(1500);
|
|
let s = parse_script(&format!("HOST: {words}\n"));
|
|
let secs = s.estimated_secs();
|
|
assert!((540..=660).contains(&secs), "1500 words ≈ 10 min, got {secs}s");
|
|
}
|
|
}
|
|
|
|
/// Live render against the real API. Ignored by default: it spends credits.
|
|
///
|
|
/// Exercises the production path — `parse_script` then `ElevenLabs::render` —
|
|
/// rather than a reimplementation, so what passes here is what runs.
|
|
///
|
|
/// CLAWMATES_PODCAST_TEST_SCRIPT=/path/to/script.md \
|
|
/// CLAWMATES_PODCAST_TEST_OUT=/tmp/episode.mp3 \
|
|
/// cargo test -p cm-api --lib podcast::live -- --ignored --nocapture
|
|
#[cfg(test)]
|
|
mod live {
|
|
use super::*;
|
|
|
|
#[tokio::test]
|
|
#[ignore = "spends ElevenLabs credits"]
|
|
async fn renders_a_real_script_to_mp3() {
|
|
let path = std::env::var("CLAWMATES_PODCAST_TEST_SCRIPT")
|
|
.expect("set CLAWMATES_PODCAST_TEST_SCRIPT");
|
|
let out = std::env::var("CLAWMATES_PODCAST_TEST_OUT")
|
|
.unwrap_or_else(|_| "/tmp/episode.mp3".to_string());
|
|
let md = std::fs::read_to_string(&path).expect("script readable");
|
|
let script = parse_script(&md);
|
|
assert!(!script.turns.is_empty(), "no turns parsed from {path}");
|
|
eprintln!(
|
|
"script: {:?} — {} turns, ~{}s",
|
|
script.title,
|
|
script.turns.len(),
|
|
script.estimated_secs()
|
|
);
|
|
|
|
let backend = ElevenLabs::from_env().expect("ELEVENLABS_API_KEY must be set");
|
|
let bytes = backend.render(&script).await.expect("render");
|
|
std::fs::write(&out, &bytes).expect("write mp3");
|
|
eprintln!("wrote {} bytes to {out} via {}", bytes.len(), backend.describe());
|
|
|
|
// ID3 or a raw MPEG frame header — anything else is not audio.
|
|
let head = &bytes[..3.min(bytes.len())];
|
|
assert!(
|
|
head == b"ID3" || (bytes[0] == 0xFF && bytes[1] & 0xE0 == 0xE0),
|
|
"not an MP3: {head:?}"
|
|
);
|
|
assert!(bytes.len() > 100_000, "suspiciously small: {} bytes", bytes.len());
|
|
}
|
|
}
|
|
|
|
// ── Rendering a finished mission into an episode ──────────────────────
|
|
|
|
/// Where a mission's script lives inside its checkout.
|
|
pub fn script_path(date: &str) -> String {
|
|
format!("ContinuousResearch/{date}/script.md")
|
|
}
|
|
|
|
/// Blob key for an episode's audio.
|
|
pub fn blob_key(mission_id: uuid::Uuid, date: &str) -> String {
|
|
format!("podcast/{date}/{mission_id}.mp3")
|
|
}
|
|
|
|
/// Duration of a joined CBR stream, from its frame headers.
|
|
///
|
|
/// Read from the audio rather than estimated from the script, because the
|
|
/// estimate is what a listener is NOT owed: the feed advertises a length and
|
|
/// that length should be the real one. Also the check that caught a six-second
|
|
/// "episode" — a file can be 6 MB and still play for seconds.
|
|
pub fn duration_secs(mp3: &[u8]) -> u32 {
|
|
let mut i = 0usize;
|
|
let mut seconds = 0f64;
|
|
while i + 4 <= mp3.len() {
|
|
if mp3[i] == 0xFF && mp3[i + 1] & 0xE0 == 0xE0 {
|
|
let br = MP3_BITRATES[((mp3[i + 2] >> 4) & 0x0F) as usize];
|
|
let sr = MP3_RATES[((mp3[i + 2] >> 2) & 0x03) as usize];
|
|
if br > 0 && sr > 0 {
|
|
let pad = ((mp3[i + 2] >> 1) & 1) as usize;
|
|
let len = (144 * br as usize * 1000 / sr as usize) + pad;
|
|
seconds += 1152.0 / sr as f64;
|
|
i += len.max(4);
|
|
continue;
|
|
}
|
|
}
|
|
i += 1;
|
|
}
|
|
seconds.round() as u32
|
|
}
|
|
|
|
fn sha_hex(bytes: &[u8]) -> String {
|
|
use sha2::{Digest, Sha256};
|
|
format!("{:x}", Sha256::digest(bytes))
|
|
}
|
|
|
|
/// Render every finished Continuous Research mission that has a script and no
|
|
/// episode yet.
|
|
///
|
|
/// Driven from a sweep rather than the phase itself: rendering is not the
|
|
/// agents' work and must not be able to fail a phase that succeeded. It is also
|
|
/// retryable by construction — a run that fails on a transient API error is
|
|
/// simply picked up next tick, and one that succeeded is skipped because the
|
|
/// episode row exists.
|
|
pub async fn render_pending(
|
|
pool: &sqlx::PgPool,
|
|
blobs: &std::sync::Arc<dyn cm_files::BlobStore>,
|
|
backend: &dyn AudioBackend,
|
|
) -> Result<usize, String> {
|
|
use sqlx::Row;
|
|
let rows = sqlx::query(
|
|
"SELECT m.id, m.workspace_id, m.title
|
|
FROM missions m
|
|
WHERE m.template_kind = $1
|
|
AND m.status IN ('completed', 'failed')
|
|
AND NOT EXISTS (SELECT 1 FROM podcast_episodes e WHERE e.mission_id = m.id)
|
|
ORDER BY m.completed_at DESC NULLS LAST
|
|
LIMIT 3",
|
|
)
|
|
.bind(crate::continuous_research::TEMPLATE_KIND)
|
|
.fetch_all(pool)
|
|
.await
|
|
.map_err(|e| format!("select missions to render: {e}"))?;
|
|
|
|
let mut made = 0usize;
|
|
for row in rows {
|
|
let mission_id: uuid::Uuid = row.get("id");
|
|
let workspace_id: uuid::Uuid = row.get("workspace_id");
|
|
let mission_title: String = row.get("title");
|
|
|
|
// A `failed` mission is included on purpose: the script phase may have
|
|
// written a perfectly good script and failed its judge. The audio is
|
|
// worth having either way, and the mission record still says it failed.
|
|
let date = crate::continuous_research::today();
|
|
let checkout = crate::mission_workspace::checkout_path(mission_id);
|
|
let mut path = checkout.join(script_path(&date));
|
|
if !path.is_file() {
|
|
// The mission may have run yesterday; take the newest script it has
|
|
// rather than assuming the render happens on the same UTC day.
|
|
match newest_script(&checkout) {
|
|
Some(p) => path = p,
|
|
None => {
|
|
// NEVER silent. The checkout is deleted 30 minutes after a
|
|
// mission reaches a terminal state (`mission_runtime`'s
|
|
// sweeper tears down the container and the tree with it), so
|
|
// a script that is not here is not late — it is gone, and
|
|
// this mission will never produce an episode. Saying so is
|
|
// the difference between a known gap and a feed that is
|
|
// quietly missing a day.
|
|
//
|
|
// The audio is recoverable by hand: the script was pushed to
|
|
// the phase's own vault branch by `mission_delivery`.
|
|
record_unrenderable(pool, mission_id, &checkout).await;
|
|
continue;
|
|
}
|
|
}
|
|
}
|
|
let md = match std::fs::read_to_string(&path) {
|
|
Ok(s) => s,
|
|
Err(e) => {
|
|
eprintln!("podcast: cannot read {}: {e}", path.display());
|
|
continue;
|
|
}
|
|
};
|
|
let script = parse_script(&md);
|
|
if script.turns.is_empty() {
|
|
eprintln!("podcast: {} has no spoken turns — skipping", path.display());
|
|
continue;
|
|
}
|
|
|
|
let audio = match backend.render(&script).await {
|
|
Ok(a) => a,
|
|
Err(e) => {
|
|
// Loud, and NOT fatal to the sweep: one mission's transient API
|
|
// failure must not stop the others being rendered.
|
|
eprintln!("podcast: render failed for mission {mission_id}: {e}");
|
|
continue;
|
|
}
|
|
};
|
|
let secs = duration_secs(&audio);
|
|
let key = blob_key(mission_id, &date);
|
|
if let Err(e) = blobs.put(&key, &audio).await {
|
|
eprintln!("podcast: shelving {key} failed: {e}");
|
|
continue;
|
|
}
|
|
|
|
let title = if script.title.trim().is_empty() {
|
|
mission_title
|
|
} else {
|
|
script.title.clone()
|
|
};
|
|
if let Err(e) = sqlx::query(
|
|
"INSERT INTO podcast_episodes
|
|
(id, workspace_id, mission_id, episode_date, title, blob_key,
|
|
bytes, duration_secs, rendered_by, script_sha)
|
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)
|
|
ON CONFLICT (mission_id) DO UPDATE
|
|
SET title = EXCLUDED.title, blob_key = EXCLUDED.blob_key,
|
|
bytes = EXCLUDED.bytes, duration_secs = EXCLUDED.duration_secs,
|
|
rendered_by = EXCLUDED.rendered_by, script_sha = EXCLUDED.script_sha",
|
|
)
|
|
.bind(uuid::Uuid::now_v7())
|
|
.bind(workspace_id)
|
|
.bind(mission_id)
|
|
.bind(&date)
|
|
.bind(&title)
|
|
.bind(&key)
|
|
.bind(audio.len() as i64)
|
|
.bind(secs as i32)
|
|
.bind(backend.describe())
|
|
.bind(sha_hex(md.as_bytes()))
|
|
.execute(pool)
|
|
.await
|
|
{
|
|
eprintln!("podcast: recording episode for {mission_id} failed: {e}");
|
|
continue;
|
|
}
|
|
eprintln!(
|
|
"podcast: episode for mission {mission_id} — {} turns, {}s, {} bytes at {key}",
|
|
script.turns.len(),
|
|
secs,
|
|
audio.len()
|
|
);
|
|
made += 1;
|
|
}
|
|
Ok(made)
|
|
}
|
|
|
|
/// Say — once — that a mission can never be rendered.
|
|
///
|
|
/// Once, not every tick: the sweep revisits the same missions forever, and a
|
|
/// line per mission per five minutes would bury everything else in the log. The
|
|
/// episode row is the marker, with a zero-length blob key that the feed skips.
|
|
async fn record_unrenderable(pool: &sqlx::PgPool, mission_id: uuid::Uuid, checkout: &std::path::Path) {
|
|
eprintln!(
|
|
"podcast: mission {mission_id} has no script at {} — the checkout was reaped before the \
|
|
render sweep reached it, so this day has no episode. The script is still on the phase's \
|
|
vault branch if it is wanted.",
|
|
checkout.display()
|
|
);
|
|
let _ = sqlx::query(
|
|
"INSERT INTO podcast_episodes
|
|
(id, workspace_id, mission_id, episode_date, title, blob_key, bytes,
|
|
duration_secs, rendered_by, script_sha)
|
|
SELECT $1, m.workspace_id, m.id, '', m.title, '', 0, 0, 'unrenderable', ''
|
|
FROM missions m WHERE m.id = $2
|
|
ON CONFLICT (mission_id) DO NOTHING",
|
|
)
|
|
.bind(uuid::Uuid::now_v7())
|
|
.bind(mission_id)
|
|
.execute(pool)
|
|
.await;
|
|
}
|
|
|
|
/// The most recent `ContinuousResearch/<date>/script.md` in a checkout.
|
|
fn newest_script(checkout: &std::path::Path) -> Option<std::path::PathBuf> {
|
|
let root = checkout.join("ContinuousResearch");
|
|
let mut dates: Vec<String> = std::fs::read_dir(root)
|
|
.ok()?
|
|
.filter_map(Result::ok)
|
|
.filter(|e| e.path().is_dir())
|
|
.map(|e| e.file_name().to_string_lossy().to_string())
|
|
.collect();
|
|
// ISO dates sort lexicographically, which is the whole reason for the format.
|
|
dates.sort();
|
|
dates.iter().rev().find_map(|d| {
|
|
let p = checkout.join(script_path(d));
|
|
p.is_file().then_some(p)
|
|
})
|
|
}
|
|
|
|
/// Spawn the render sweep.
|
|
pub fn spawn(
|
|
pool: sqlx::PgPool,
|
|
blobs: Option<std::sync::Arc<dyn cm_files::BlobStore>>,
|
|
interval: std::time::Duration,
|
|
) {
|
|
let Some(blobs) = blobs else {
|
|
eprintln!("podcast: no blob storage configured — episodes will not be rendered");
|
|
return;
|
|
};
|
|
let Some(backend) = ElevenLabs::from_env() else {
|
|
// Not an error. A deployment without a key simply produces no audio,
|
|
// and every other part of the mission still works.
|
|
eprintln!("podcast: ELEVENLABS_API_KEY not set — episodes will not be rendered");
|
|
return;
|
|
};
|
|
tokio::spawn(async move {
|
|
let mut tick = tokio::time::interval(interval);
|
|
tick.tick().await;
|
|
loop {
|
|
tick.tick().await;
|
|
match render_pending(&pool, &blobs, &backend).await {
|
|
Ok(n) if n > 0 => eprintln!("podcast: rendered {n} episode(s)"),
|
|
Ok(_) => {}
|
|
Err(e) => eprintln!("podcast: sweep failed: {e}"),
|
|
}
|
|
}
|
|
});
|
|
}
|