fix(research): the manifest belongs to the mission, and a brittle phrase must not mean silence

Two defects from the first on-topic run.

**1. The manifest could never be updated twice in a day.** It was written into
the VAULT at a per-DATE path, but it is per-RUN data. A second mission the same
day rewrites a file that already exists, and `auto_merge` correctly refused the
whole branch:

    diff is not additive (1 non-add change(s), first:
    M ContinuousResearch/2026-08-18/harvest.jsonl); left for a human

So `main` kept the FIRST run's manifest, the next mission cloned it, and the
agents analysed yesterday's papers while every log line reported a successful
harvest. The merge policy was right; the placement was wrong. The manifest now
goes into the mission's own checkout after `ensure_checkout`, which keeps the
vault additive and gives each mission exactly its own papers. The agents commit
it alongside their analysis through the normal delivery path.

**2. A quoted phrase that matches nothing looked like a quiet day.** Phrase
search is precise and brittle: "hybrid retrieval BM25 dense" is a reasonable
topic and appears verbatim in no paper on arXiv — measured, 0 hits — while the
same four terms unquoted return exactly the hybrid-retrieval evaluations the
topic asked for. Harvesting zero because of adjacency is indistinguishable from
a genuinely quiet field, which is the distinction `Harvest::healthy()` vs
`added_anything()` exists to preserve. `search` now retries unquoted when the
phrase finds nothing, and says so in the log.

Proven in one run, all three behaviours at once:

    "approximate nearest neighbor search" -> 5 candidates, 5 already held, 0 shelved
    "hybrid retrieval BM25 dense"         -> no exact phrase match, retrying broad
                                          -> 5 candidates, 0 already held, 5 shelved
    "LLM as a judge evaluation"           -> 5 candidates, 5 already held, 0 shelved
    wrote 5 paper(s) to .../ContinuousResearch/2026-08-18/harvest.jsonl

The seen-set suppressing 10 of 15 is the whole point of a recurring mission, and
the 5 that landed are on topic for the first time: RAG architecture evaluation,
agent-controlled search over chat logs, compute-aware retrieval and reranking,
hybrid retrieval in hyperbolic space, sparse-dense fusion limits.

353 tests pass.

Co-Authored-By: Claude Opus 5 <[email protected]>

crates/cm-api/src/continuous_research.rs

@@ -76,7 +76,7 @@ pub async fn harvest_for_mission(
     mission_id: Uuid,
     topics: &[String],
     per_topic: usize,
-) -> Result<usize, String> {
+) -> Result<Vec<crate::papers::Paper>, String> {
     let work_root = std::env::temp_dir().join("clawmates-library");
     let run = crate::library::run_to_vault(
         pool,
@@ -106,7 +106,37 @@ pub async fn harvest_for_mission(
     for (source_id, why) in &run.harvest.failed {
         eprintln!("continuous_research: {source_id} not shelved: {why}");
     }
-    Ok(shelved)
+    Ok(run.harvest.papers)
+}
+
+/// Write the run manifest into the MISSION's checkout.
+///
+/// Not into the vault. The manifest is per-RUN input for one mission, and the
+/// vault path is per-DATE and shared, so a second run on the same day rewrites
+/// a file that already exists — which `auto_merge` correctly refuses, because
+/// it only merges provably additive diffs:
+///
+///   "diff is not additive (1 non-add change(s), first:
+///    M ContinuousResearch/2026-08-18/harvest.jsonl); left for a human"
+///
+/// The branch was then left unmerged, `main` kept the previous run's manifest,
+/// and the next mission cloned STALE papers while every log line said the
+/// harvest succeeded. Writing into the checkout keeps the vault additive and
+/// gives each mission exactly its own papers. The agents commit it alongside
+/// their analysis through the normal delivery path.
+pub fn write_manifest(
+    checkout: &std::path::Path,
+    papers: &[crate::papers::Paper],
+    date: &str,
+) -> Result<std::path::PathBuf, String> {
+    let rel = manifest_path(date);
+    let abs = checkout.join(&rel);
+    if let Some(parent) = abs.parent() {
+        std::fs::create_dir_all(parent).map_err(|e| format!("create {}: {e}", parent.display()))?;
+    }
+    let body = manifest_lines(papers, date);
+    std::fs::write(&abs, format!("{body}\n")).map_err(|e| format!("write {}: {e}", abs.display()))?;
+    Ok(abs)
 }
 
 /// The manifest lines for a set of freshly shelved papers.

crates/cm-api/src/library.rs

@@ -177,37 +177,7 @@ pub async fn run_to_vault(
 
     git(&vault, &["checkout", "-B", &branch]).await?;
 
-    // The run manifest, beside the notes.
+    git(&vault, &["add", "--", "60 Papers"]).await?;
-    //
-    // `templates/teams/continuous_research.toml` has told its reader role to
-    // start from `ContinuousResearch/<date>/harvest.jsonl` since it was
-    // authored, and nothing wrote it — the agents were pointed at a file that
-    // did not exist. This is the code producing what the prompt already
-    // promises, so "nothing new today" is a fact the agent reads rather than a
-    // conclusion it guesses from an empty folder.
-    //
-    // Written only when the run is attributed to a mission: a plain library run
-    // has no agent waiting on it and does not need the extra file in the vault.
-    let mut staged: Vec<&str> = vec!["60 Papers"];
-    let manifest_rel = crate::continuous_research::manifest_path(&crate::continuous_research::today());
-    if mission_id.is_some() {
-        let manifest_abs = vault.join(&manifest_rel);
-        if let Some(parent) = manifest_abs.parent() {
-            std::fs::create_dir_all(parent)
-                .map_err(|e| format!("create {}: {e}", parent.display()))?;
-        }
-        let body = crate::continuous_research::manifest_lines(
-            &total.papers,
-            &crate::continuous_research::today(),
-        );
-        std::fs::write(&manifest_abs, format!("{body}\n"))
-            .map_err(|e| format!("write {}: {e}", manifest_abs.display()))?;
-        staged.push("ContinuousResearch");
-    }
-
-    let mut add: Vec<&str> = vec!["add", "--"];
-    add.extend(staged);
-    git(&vault, &add).await?;
     let message = format!(
         "library: {} new paper(s)\n\n{}\n\nShelved in the blob store; this commit is the catalogue.",
         total.shelved.len(),

crates/cm-api/src/mission_orchestrator.rs

@@ -75,6 +75,7 @@ pub async fn on_launch(
     // because the phase is what reports whether today was quiet or broken, and
     // those must stay distinguishable. What is never acceptable is silence, so
     // both outcomes are logged with their counts.
+    let mut harvested: Vec<crate::papers::Paper> = Vec::new();
     if mission.template_kind == crate::continuous_research::TEMPLATE_KIND {
         match blobs.as_ref() {
             Some(b) => {
@@ -89,9 +90,13 @@ pub async fn on_launch(
                 )
                 .await
                 {
-                    Ok(n) => eprintln!(
+                    Ok(papers) => {
-                        "mission_orchestrator: continuous research harvest shelved {n} paper(s)                          for mission {mission_id}"
+                        eprintln!(
-                    ),
+                            "mission_orchestrator: continuous research harvest shelved {} paper(s) for mission {mission_id}",
+                            papers.len()
+                        );
+                        harvested = papers;
+                    }
                     Err(e) => eprintln!(
                         "mission_orchestrator: continuous research harvest FAILED for                          {mission_id} (phases still start, and will report an empty day): {e}"
                     ),
@@ -113,10 +118,33 @@ pub async fn on_launch(
     // repo checkout even though its team was minted on the first
     // launch. Non-fatal — logs and continues on failure.
     match crate::mission_workspace::ensure_checkout(pool, workspace_id, mission_id).await {
-        Ok(Some(path)) => eprintln!(
+        Ok(Some(path)) => {
+            eprintln!(
                 "mission_orchestrator: repo checked out at {} for mission {mission_id}",
                 path.display()
+            );
+            // The manifest goes in the CHECKOUT, not the vault: it is this run's
+            // input, and the vault path is per-date and shared, so a second run
+            // the same day rewrites a file that already exists and auto_merge
+            // rightly refuses the branch. See `write_manifest`.
+            if mission.template_kind == crate::continuous_research::TEMPLATE_KIND {
+                let date = crate::continuous_research::today();
+                match crate::continuous_research::write_manifest(&path, &harvested, &date) {
+                    Ok(at) => eprintln!(
+                        "mission_orchestrator: wrote {} paper(s) to {}",
+                        harvested.len(),
+                        at.display()
                     ),
+                    // Loud: the reader phase would find no manifest and, being
+                    // resourceful, go and search arXiv itself — which corrupts
+                    // the seen-set. Better to see why here.
+                    Err(e) => eprintln!(
+                        "mission_orchestrator: could NOT write the harvest manifest for \
+                         {mission_id} — the reader phase will see no papers: {e}"
+                    ),
+                }
+            }
+        }
         Ok(None) => eprintln!(
             "mission_orchestrator: mission {mission_id} has no repo bound, skipping checkout"
         ),

crates/cm-api/src/papers.rs

@@ -181,13 +181,51 @@ pub fn arxiv_query(topic: &str) -> String {
     format!("all:\"{escaped}\" AND cat:cs.*")
 }
 
+/// The looser form of a topic: every term required, but not adjacent.
+///
+/// A quoted phrase is precise and brittle. "hybrid retrieval BM25 dense" is a
+/// perfectly good topic and appears verbatim in no paper on arXiv — measured, 0
+/// hits — while requiring the same four terms anywhere returns exactly the
+/// hybrid-retrieval evaluations the topic was asking for. Used only when the
+/// phrase finds nothing, so an exact match still wins when one exists.
+pub fn arxiv_query_broad(topic: &str) -> String {
+    let terms: Vec<String> = topic
+        .split_whitespace()
+        .map(|w| w.trim_matches(|c: char| !c.is_alphanumeric() && c != '-'))
+        .filter(|w| !w.is_empty())
+        .map(|w| format!("all:{w}"))
+        .collect();
+    if terms.is_empty() {
+        return arxiv_query(topic);
+    }
+    format!("{} AND cat:cs.*", terms.join(" AND "))
+}
+
 /// Search arXiv. `max_results` is capped to keep one run bounded.
 pub async fn search(query: &str, max_results: usize) -> Result<Vec<Paper>, String> {
+    let found = search_with(&arxiv_query(query), max_results).await?;
+    if !found.is_empty() {
+        return Ok(found);
+    }
+    // The phrase matched nothing. Before reporting a quiet day — which the whole
+    // pipeline treats as a real and legitimate outcome — try the same terms
+    // unquoted. A topic the operator writes as prose often is not a literal
+    // phrase in any title, and silently harvesting zero because of punctuation
+    // would be indistinguishable from a genuinely quiet field.
+    let broad = arxiv_query_broad(query);
+    if broad == arxiv_query(query) {
+        return Ok(found);
+    }
+    eprintln!("papers: no exact phrase match for {query:?} — retrying as {broad}");
+    search_with(&broad, max_results).await
+}
+
+async fn search_with(search_query: &str, max_results: usize) -> Result<Vec<Paper>, String> {
     let max = max_results.clamp(1, 50);
     let url = format!(
         "https://export.arxiv.org/api/query?search_query={}&start=0&max_results={max}\
          &sortBy=submittedDate&sortOrder=descending",
-        urlencoding(&arxiv_query(query))
+        urlencoding(search_query)
     );
     let body = reqwest::Client::new()
         .get(&url)
@@ -314,6 +352,36 @@ mod tests {
         }
     }
 
+    /// The broad form requires every term but not adjacency. Measured: the
+    /// phrase "hybrid retrieval BM25 dense" has 0 hits on arXiv; the same four
+    /// terms unquoted return the hybrid-retrieval evaluations that were asked
+    /// for. Without the fallback that topic silently harvests nothing, which is
+    /// indistinguishable from a genuinely quiet day.
+    #[test]
+    fn the_broad_form_requires_every_term_without_adjacency() {
+        let q = arxiv_query_broad("hybrid retrieval BM25 dense");
+        assert_eq!(
+            q,
+            "all:hybrid AND all:retrieval AND all:BM25 AND all:dense AND cat:cs.*"
+        );
+        assert!(!q.contains('"'), "the broad form must not be a phrase: {q}");
+        assert!(q.contains("cat:cs.*"), "still category-bound: {q}");
+    }
+
+    /// Punctuation must not leak into a term and must not empty the query.
+    #[test]
+    fn the_broad_form_strips_punctuation_and_never_empties() {
+        assert_eq!(
+            arxiv_query_broad("retrieval-augmented, generation!"),
+            "all:retrieval-augmented AND all:generation AND cat:cs.*",
+            "hyphens are part of a term; trailing punctuation is not"
+        );
+        // Nothing usable left: fall back to the phrase form rather than
+        // emitting a bare `cat:cs.*`, which would match all of computer science.
+        let q = arxiv_query_broad("!!!");
+        assert!(q.contains("all:"), "must never degrade to a bare category: {q}");
+    }
+
     /// Quotes in a topic would terminate the phrase early and corrupt the query.
     #[test]
     fn quotes_in_a_topic_cannot_break_out_of_the_phrase() {