fix(podcast): stop reading identifiers aloud, and pitch the episode at a teenager
Two things the operator found by listening to a real episode.
**1. Identifiers were spoken as digit soup.** The script genuinely said
"arxiv 2608.12888", which the voice reads as "two six zero eight point one two
eight eight eight". Same for three-decimal values: "0.506" and "0.004" became
long strings of spoken digits. A listener on a treadmill cannot write an
identifier down and does not need a third decimal place.
`speakable()` strips arXiv references and bare identifier-shaped numbers, and
rounds decimals to two places — with a carve-out that matters: 0.004 rounds to
0.00, which would claim the value was ZERO when the whole point was that it
collapsed to nearly nothing, so it says "under 0.01" instead.
Deliberately narrow: it removes identifiers and shortens over-precise decimals,
and does not paraphrase, reorder or summarise. The agents' words are still the
episode. It also preserves the sentence's full stop — swallowing it turned
"…financial retrieval, arxiv 2608.00183. This one's a catch." into one run-on
sentence, and the pause is how a listener knows a thought ended.
Note that `podcast-dialogue-writing.md` ALREADY said "no arXiv ids" and the
writer included them anyway. That is this project's recurring lesson restated:
an instruction is a request, and a listener deserves a guarantee. The prose asks
and the code enforces.
**2. It was written for someone who already knew the field.** The skill and the
script phase's task now target a bright sixteen-year-old: define an acronym in
the sentence that first uses it, describe the mechanism rather than naming it
("a road map with motorways and side streets" instead of "a hierarchical
navigable small world graph"), one idea per sentence. The test offered is
whether the listener could explain the finding to a friend afterwards.
That is not dumbing down — it is the constraint that forces a writer to say what
a thing actually does rather than what it is called.
Tested against the exact lines from the episode that was listened to.
366 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
1f39f642a3
commit
f4adc8d0f9
@@ -165,6 +165,144 @@ fn find(hay: &[u8], needle: &[u8]) -> Option<usize> {
|
|||||||
hay.windows(needle.len()).position(|w| w == needle)
|
hay.windows(needle.len()).position(|w| w == needle)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/// Rewrite a line so it is worth HEARING.
|
||||||
|
///
|
||||||
|
/// Written from a real episode the operator listened to. Two things ruined it,
|
||||||
|
/// and neither is a TTS defect — the text genuinely said them:
|
||||||
|
///
|
||||||
|
/// ```text
|
||||||
|
/// "This is the ReFind paper, arxiv 2608.12888."
|
||||||
|
/// -> "two six zero eight point one two eight eight eight"
|
||||||
|
/// "BM25 recall dropped from 0.506 native to 0.004 cross-lingual"
|
||||||
|
/// -> "zero point five zero six ... zero point zero zero four"
|
||||||
|
/// ```
|
||||||
|
///
|
||||||
|
/// A listener on a treadmill cannot write an identifier down and does not need
|
||||||
|
/// three decimal places. `skills/research/podcast-dialogue-writing.md` already
|
||||||
|
/// told the writer not to include arXiv ids and it included them anyway — which
|
||||||
|
/// is the lesson of this whole project restated: an instruction is a request,
|
||||||
|
/// and a listener deserves a guarantee. So the prose asks and this enforces.
|
||||||
|
///
|
||||||
|
/// Deliberately narrow. It removes identifiers and shortens over-precise
|
||||||
|
/// decimals; it does not paraphrase, reorder or summarise. The agents' words
|
||||||
|
/// are still the episode.
|
||||||
|
pub fn speakable(line: &str) -> String {
|
||||||
|
let mut out = String::with_capacity(line.len());
|
||||||
|
let b: Vec<char> = line.chars().collect();
|
||||||
|
let mut i = 0usize;
|
||||||
|
|
||||||
|
while i < b.len() {
|
||||||
|
// "arXiv:2608.12888", "arxiv 2608.12888", "arXiv 2608.12888v2"
|
||||||
|
if starts_with_ci(&b, i, "arxiv") {
|
||||||
|
let mut j = i + 5;
|
||||||
|
while j < b.len() && (b[j] == ':' || b[j] == ' ' || b[j] == '.') {
|
||||||
|
j += 1;
|
||||||
|
}
|
||||||
|
let digits_start = j;
|
||||||
|
while j < b.len() && (b[j].is_ascii_digit() || b[j] == '.' || b[j] == 'v') {
|
||||||
|
j += 1;
|
||||||
|
}
|
||||||
|
// Do not swallow the sentence's full stop. "…retrieval, arxiv
|
||||||
|
// 2608.00183. This one's a catch." must not become one run-on
|
||||||
|
// sentence — the pause is how a listener knows a thought ended.
|
||||||
|
while j > digits_start && !b[j - 1].is_ascii_digit() {
|
||||||
|
j -= 1;
|
||||||
|
}
|
||||||
|
if j > digits_start + 4 {
|
||||||
|
// Drop the whole reference, and any comma or space it left
|
||||||
|
// dangling: "the ReFind paper, arxiv 2608.12888." must not
|
||||||
|
// become "the ReFind paper, ."
|
||||||
|
trim_trailing_separator(&mut out);
|
||||||
|
i = j;
|
||||||
|
skip_leading_separator(&b, &mut i);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A bare arXiv-shaped number: 4 digits, dot, 4-5 digits.
|
||||||
|
if b[i].is_ascii_digit() {
|
||||||
|
let start = i;
|
||||||
|
let mut j = i;
|
||||||
|
while j < b.len() && b[j].is_ascii_digit() {
|
||||||
|
j += 1;
|
||||||
|
}
|
||||||
|
let int_len = j - start;
|
||||||
|
if j < b.len() && b[j] == '.' {
|
||||||
|
let frac_start = j + 1;
|
||||||
|
let mut k = frac_start;
|
||||||
|
while k < b.len() && b[k].is_ascii_digit() {
|
||||||
|
k += 1;
|
||||||
|
}
|
||||||
|
let frac_len = k - frac_start;
|
||||||
|
if int_len == 4 && (4..=5).contains(&frac_len) {
|
||||||
|
// An identifier, not a quantity.
|
||||||
|
trim_trailing_separator(&mut out);
|
||||||
|
i = k;
|
||||||
|
skip_leading_separator(&b, &mut i);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if frac_len >= 3 {
|
||||||
|
// Over-precise. Nobody hears the third decimal place.
|
||||||
|
let text: String = b[start..k].iter().collect();
|
||||||
|
out.push_str(&round_decimal(&text));
|
||||||
|
i = k;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out.push(b[i]);
|
||||||
|
i += 1;
|
||||||
|
}
|
||||||
|
// Collapse any double spaces a removal left behind.
|
||||||
|
let collapsed = out.split_whitespace().collect::<Vec<_>>().join(" ");
|
||||||
|
collapsed
|
||||||
|
.replace(" ,", ",")
|
||||||
|
.replace(" .", ".")
|
||||||
|
.replace("( )", "")
|
||||||
|
.replace("()", "")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn starts_with_ci(b: &[char], i: usize, word: &str) -> bool {
|
||||||
|
let w: Vec<char> = word.chars().collect();
|
||||||
|
if i + w.len() > b.len() {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
b[i..i + w.len()]
|
||||||
|
.iter()
|
||||||
|
.zip(&w)
|
||||||
|
.all(|(a, c)| a.to_ascii_lowercase() == *c)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn trim_trailing_separator(out: &mut String) {
|
||||||
|
while out.ends_with(' ') || out.ends_with(',') || out.ends_with('(') {
|
||||||
|
out.pop();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn skip_leading_separator(b: &[char], i: &mut usize) {
|
||||||
|
while *i < b.len() && (b[*i] == ')' || b[*i] == ',') {
|
||||||
|
*i += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Two decimal places, or "under 0.01" when rounding would say "0.00".
|
||||||
|
///
|
||||||
|
/// `0.004` rounded to two places is `0.00`, which is worse than the original:
|
||||||
|
/// it says the value is zero when the point was that it collapsed to nearly
|
||||||
|
/// nothing.
|
||||||
|
fn round_decimal(text: &str) -> String {
|
||||||
|
let Ok(v) = text.parse::<f64>() else {
|
||||||
|
return text.to_string();
|
||||||
|
};
|
||||||
|
let r = (v * 100.0).round() / 100.0;
|
||||||
|
if r == 0.0 && v != 0.0 {
|
||||||
|
return "under 0.01".to_string();
|
||||||
|
}
|
||||||
|
let s = format!("{r:.2}");
|
||||||
|
s.trim_end_matches('0').trim_end_matches('.').to_string()
|
||||||
|
}
|
||||||
|
|
||||||
/// Anything that can turn a script into audio bytes.
|
/// Anything that can turn a script into audio bytes.
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
pub trait AudioBackend: Send + Sync {
|
pub trait AudioBackend: Send + Sync {
|
||||||
@@ -277,7 +415,7 @@ impl AudioBackend for ElevenLabs {
|
|||||||
let mut out: Vec<u8> = Vec::new();
|
let mut out: Vec<u8> = Vec::new();
|
||||||
for (i, turn) in script.turns.iter().enumerate() {
|
for (i, turn) in script.turns.iter().enumerate() {
|
||||||
let voice = self.voice_for(&turn.speaker, &first, second.as_deref());
|
let voice = self.voice_for(&turn.speaker, &first, second.as_deref());
|
||||||
let clip = self.say(&turn.text, voice).await.map_err(|e| {
|
let clip = self.say(&speakable(&turn.text), voice).await.map_err(|e| {
|
||||||
// Name the turn: a 400 on one line is far easier to fix than
|
// Name the turn: a 400 on one line is far easier to fix than
|
||||||
// "rendering failed" for a 40-turn script.
|
// "rendering failed" for a 40-turn script.
|
||||||
format!("turn {} ({}): {e}", i + 1, turn.speaker)
|
format!("turn {} ({}): {e}", i + 1, turn.speaker)
|
||||||
@@ -386,6 +524,66 @@ mod tests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Real lines from the episode the operator listened to. These are the
|
||||||
|
/// exact strings the TTS read aloud as digit soup.
|
||||||
|
/// Print what the operator's own episode WOULD have said. Not an
|
||||||
|
/// assertion — a way to read the diff on real input.
|
||||||
|
#[test]
|
||||||
|
fn show_real_script_lines() {
|
||||||
|
let Ok(md) = std::env::var("CLAWMATES_SPEAKABLE_DEMO") else { return };
|
||||||
|
let Ok(text) = std::fs::read_to_string(&md) else { return };
|
||||||
|
for line in text.lines() {
|
||||||
|
let out = speakable(line);
|
||||||
|
if out != line && !line.trim().is_empty() {
|
||||||
|
eprintln!(" BEFORE {}", line.trim());
|
||||||
|
eprintln!(" AFTER {}\n", out.trim());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn identifiers_are_never_spoken() {
|
||||||
|
for (input, must_not) in [
|
||||||
|
("This is the ReFind paper, arxiv 2608.12888.", "2608"),
|
||||||
|
("There was an agricultural paper too, arXiv:2608.14886 — does it matter?", "14886"),
|
||||||
|
("evidence-unit fairness in financial retrieval, arxiv 2608.00183", "00183"),
|
||||||
|
] {
|
||||||
|
let out = speakable(input);
|
||||||
|
assert!(!out.contains(must_not), "{must_not} survived in {out:?}");
|
||||||
|
assert!(!out.to_lowercase().contains("arxiv"), "dangling label: {out:?}");
|
||||||
|
// The sentence must still read cleanly.
|
||||||
|
assert!(!out.contains(" ,"), "orphan comma: {out:?}");
|
||||||
|
assert!(!out.contains(",."), "orphan comma: {out:?}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Three decimal places is data, not speech.
|
||||||
|
#[test]
|
||||||
|
fn over_precise_decimals_are_shortened() {
|
||||||
|
let out = speakable("BM25 recall dropped from 0.506 native to 0.004 cross-lingual.");
|
||||||
|
assert!(out.contains("0.51"), "0.506 should round: {out:?}");
|
||||||
|
assert!(!out.contains("0.506"), "{out:?}");
|
||||||
|
// 0.004 rounds to 0.00, which would claim the value was zero — the
|
||||||
|
// opposite of the point being made.
|
||||||
|
assert!(out.contains("under 0.01"), "{out:?}");
|
||||||
|
assert!(!out.contains("0.00 "), "must never say zero: {out:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Two decimals, years, percentages and small integers are all fine spoken
|
||||||
|
/// and must survive untouched — over-processing would mangle the meaning.
|
||||||
|
#[test]
|
||||||
|
fn ordinary_numbers_are_left_alone() {
|
||||||
|
for s in [
|
||||||
|
"58.2 versus 53.2 mean accuracy",
|
||||||
|
"roughly 2,800 questions",
|
||||||
|
"21.8% of theoretical headroom",
|
||||||
|
"NDCG at 10 of 0.15",
|
||||||
|
"about 2026 papers",
|
||||||
|
] {
|
||||||
|
assert_eq!(speakable(s), s, "should be unchanged");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_runtime_estimate_is_in_the_right_ballpark() {
|
fn the_runtime_estimate_is_in_the_right_ballpark() {
|
||||||
let words = "word ".repeat(1500);
|
let words = "word ".repeat(1500);
|
||||||
|
|||||||
@@ -1,20 +1,60 @@
|
|||||||
---
|
---
|
||||||
name: podcast-dialogue-writing
|
name: podcast-dialogue-writing
|
||||||
description: How to turn a paper analysis into a two-host script someone can follow while running, and the hard limits the audio API imposes.
|
description: How to turn a paper analysis into a two-host script a bright sixteen-year-old can follow while running, and the hard limits the audio pipeline imposes.
|
||||||
when_to_use: You are writing the script phase of a Continuous Research mission, turning analysis.md into script.md and episode.json.
|
when_to_use: You are writing the script phase of a Continuous Research mission, turning analysis.md into script.md and episode.json.
|
||||||
tags: [research, writing]
|
tags: [research, writing]
|
||||||
---
|
---
|
||||||
|
|
||||||
# Writing for someone on a treadmill
|
# Writing for someone on a treadmill
|
||||||
|
|
||||||
The listener is not at a desk. They cannot scroll back, they cannot see a table,
|
The listener is moving. They cannot scroll back, cannot see a table, and are
|
||||||
and they are half-listening. Everything follows from that.
|
half-listening. Everything below follows from that.
|
||||||
|
|
||||||
|
## Pitch it at a bright sixteen-year-old
|
||||||
|
|
||||||
|
Assume someone curious and clever who has **not** read the paper, does not know
|
||||||
|
the jargon, and cannot pause to look things up. That is not dumbing down — it is
|
||||||
|
the discipline that forces you to say what a thing actually does.
|
||||||
|
|
||||||
|
**Define an acronym the first time, in the sentence, then use it.**
|
||||||
|
|
||||||
|
> Bad: "NDCG at 10 of 0.15 on SciDocs."
|
||||||
|
> Good: "Their accuracy score was about 0.15 — where 1.0 is perfect. That is
|
||||||
|
> low."
|
||||||
|
|
||||||
|
**Say the mechanism, not the label.**
|
||||||
|
|
||||||
|
> Bad: "They use HNSW with hierarchical navigable small world graphs."
|
||||||
|
> Good: "The index works like a road map with motorways and side streets: you
|
||||||
|
> take the fast road most of the way, then drop down to local roads to
|
||||||
|
> find the exact house."
|
||||||
|
|
||||||
|
**One idea per sentence.** If a sentence needs a comma-spliced clause to
|
||||||
|
survive, it needs to be two sentences.
|
||||||
|
|
||||||
|
A good test: could the listener explain the finding to a friend afterwards? If
|
||||||
|
the only honest answer is "they would repeat a phrase they did not understand",
|
||||||
|
rewrite it.
|
||||||
|
|
||||||
|
## Numbers are spoken, not read
|
||||||
|
|
||||||
|
The audio pipeline strips identifiers and rounds over-precise decimals before
|
||||||
|
speech, but write it right in the first place — the safety net should never fire.
|
||||||
|
|
||||||
|
- **Never write an arXiv id.** "arxiv 2608.12888" is heard as "two six zero
|
||||||
|
eight point one two eight eight eight". Name the paper instead: "the ReFind
|
||||||
|
paper". The id is in the vault note if anyone wants it.
|
||||||
|
- **Two decimal places, maximum.** "0.506" becomes "about 0.51", and better
|
||||||
|
still "about half".
|
||||||
|
- **Round in speech.** "roughly a third faster" beats "34.7% faster". "About
|
||||||
|
three thousand questions" beats "2,847".
|
||||||
|
- **No URLs, no figure or table references, no version suffixes.** If it cannot
|
||||||
|
be held in the ear, it does not belong in the audio.
|
||||||
|
|
||||||
## Structure
|
## Structure
|
||||||
|
|
||||||
Lead with the conclusion, then support it. A paper-by-paper walk is the wrong
|
Lead with the conclusion, then support it. A paper-by-paper walk is the wrong
|
||||||
shape: the listener does not know which papers matter until you tell them, and
|
shape: the listener does not know which papers matter until you tell them.
|
||||||
by then the first one is gone.
|
|
||||||
|
|
||||||
```
|
```
|
||||||
COLD OPEN one sentence: the single thing that changed today
|
COLD OPEN one sentence: the single thing that changed today
|
||||||
@@ -28,50 +68,32 @@ matter is what makes the three that did credible.
|
|||||||
|
|
||||||
## Two hosts, with a reason to be two
|
## Two hosts, with a reason to be two
|
||||||
|
|
||||||
HOST carries the thread and asks what the listener would ask. GUEST has read the
|
HOST carries the thread and asks what the listener would ask — including "wait,
|
||||||
paper and answers. The reason for a second voice is that disagreement and
|
what does that mean?". GUEST has read the paper and answers in plain words. The
|
||||||
"wait, why?" are easier to follow than monologue — not that alternating names
|
second voice exists so that confusion can be voiced and cleared, not so that
|
||||||
looks like a script.
|
alternating names can decorate a monologue.
|
||||||
|
|
||||||
Bad, because these are one voice cut in half:
|
> HOST: Twelve percent better recall — is that on something we care about?
|
||||||
|
> GUEST: That is the catch. They tested on a million short web snippets. Ours
|
||||||
> HOST: The paper introduces a new pruning method.
|
> are longer and there are far fewer of them, so their win might not
|
||||||
> GUEST: Yes, and it improves recall by 12%.
|
> survive the switch.
|
||||||
|
|
||||||
Good, because the second voice is doing work:
|
|
||||||
|
|
||||||
> HOST: Twelve percent recall — is that on a benchmark we'd care about?
|
|
||||||
> GUEST: That's the catch. It's on SIFT1M, which is a million 128-dim vectors.
|
|
||||||
> Our embeddings are 1536-dim and we're at about 40,000. So the shape of
|
|
||||||
> their win might not survive at our dimensionality.
|
|
||||||
|
|
||||||
## Say numbers the ear can hold
|
|
||||||
|
|
||||||
"Roughly a third faster" beats "34.7% faster". "About a million vectors" beats
|
|
||||||
"1,000,000". Read every line aloud in your head; if you stumble, the TTS will
|
|
||||||
too.
|
|
||||||
|
|
||||||
Never write anything the listener cannot hold: no URLs, no arXiv ids, no table
|
|
||||||
references, no "as shown in Figure 4". If they need the citation it is in the
|
|
||||||
vault note.
|
|
||||||
|
|
||||||
## Hard limits, not style preferences
|
## Hard limits, not style preferences
|
||||||
|
|
||||||
`episode.json` is consumed by the audio API, and these bounds are enforced by it:
|
`episode.json` is consumed by the audio pipeline and these bounds are enforced:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "title": "<one line, under 80 characters>",
|
{ "title": "<one line, under 80 characters>",
|
||||||
"highlights": ["<each 10-70 characters>", "..."] }
|
"highlights": ["<each 10-70 characters>", "..."] }
|
||||||
```
|
```
|
||||||
|
|
||||||
- **At most 5 highlights**, each **10–70 characters**. Outside that range the
|
At most **5** highlights, each **10-70 characters**. Outside that range the
|
||||||
request is rejected, not truncated.
|
entry is rejected, not trimmed.
|
||||||
- Target **seven minutes of speech, roughly 1,000 words**. Audio quality
|
|
||||||
degrades on long single generations, so a sprawling script produces a worse
|
Target **seven minutes of speech, roughly 1,000 words**.
|
||||||
episode, not just a longer one.
|
|
||||||
|
|
||||||
## Cut rather than pad
|
## Cut rather than pad
|
||||||
|
|
||||||
If only one paper mattered, write a four-minute episode about one paper. An
|
If one paper mattered, write four minutes about one paper. An episode padded to
|
||||||
episode padded to length with work that did not matter trains the listener to
|
length teaches the listener to skip, and once they skip, the pipeline is worth
|
||||||
skip, and once they skip, the whole pipeline is worthless.
|
nothing.
|
||||||
|
|||||||
@@ -87,8 +87,20 @@ Also write ContinuousResearch/<today>/episode.json:
|
|||||||
Those bounds are the podcast API's, not a style preference — a highlight \
|
Those bounds are the podcast API's, not a style preference — a highlight \
|
||||||
outside them is rejected.
|
outside them is rejected.
|
||||||
|
|
||||||
|
Pitch it at a bright sixteen-year-old: someone curious who has NOT read the \
|
||||||
|
paper, does not know the jargon, and cannot pause to look anything up. Define \
|
||||||
|
an acronym the first time in the sentence, then use it. Describe the mechanism \
|
||||||
|
rather than naming it — "a road map with motorways and side streets" beats \
|
||||||
|
"a hierarchical navigable small world graph". That is not dumbing down; it is \
|
||||||
|
what forces you to say what the thing actually does.
|
||||||
|
|
||||||
|
Numbers are HEARD. Never write an arXiv id — "arxiv 2608.12888" is heard as \
|
||||||
|
"two six zero eight point one two eight eight eight"; name the paper instead. \
|
||||||
|
Two decimal places at most, and round in speech: "roughly a third faster" \
|
||||||
|
beats "34.7% faster". No URLs, no figure references.
|
||||||
|
|
||||||
Target seven minutes of speech, roughly 1,000 words. Say the specific thing: \
|
Target seven minutes of speech, roughly 1,000 words. Say the specific thing: \
|
||||||
"this changes how we prune the HNSW graph in clawhdf5" beats "researchers \
|
"this changes how we prune the search index in clawhdf5" beats "researchers \
|
||||||
propose a novel method". Skip a paper entirely rather than pad the episode \
|
propose a novel method". Skip a paper entirely rather than pad the episode \
|
||||||
with one that does not matter.
|
with one that does not matter.
|
||||||
"""
|
"""
|
||||||
|
|||||||
Reference in New Issue
Block a user