fix(research): the manifest belongs to the mission, and a brittle phrase must not mean silence
deploy / test (push) Successful in 4m28s
deploy / build (push) Successful in 5m21s

Two defects from the first on-topic run.

**1. The manifest could never be updated twice in a day.** It was written into
the VAULT at a per-DATE path, but it is per-RUN data. A second mission the same
day rewrites a file that already exists, and `auto_merge` correctly refused the
whole branch:

    diff is not additive (1 non-add change(s), first:
    M ContinuousResearch/2026-08-18/harvest.jsonl); left for a human

So `main` kept the FIRST run's manifest, the next mission cloned it, and the
agents analysed yesterday's papers while every log line reported a successful
harvest. The merge policy was right; the placement was wrong. The manifest now
goes into the mission's own checkout after `ensure_checkout`, which keeps the
vault additive and gives each mission exactly its own papers. The agents commit
it alongside their analysis through the normal delivery path.

**2. A quoted phrase that matches nothing looked like a quiet day.** Phrase
search is precise and brittle: "hybrid retrieval BM25 dense" is a reasonable
topic and appears verbatim in no paper on arXiv — measured, 0 hits — while the
same four terms unquoted return exactly the hybrid-retrieval evaluations the
topic asked for. Harvesting zero because of adjacency is indistinguishable from
a genuinely quiet field, which is the distinction `Harvest::healthy()` vs
`added_anything()` exists to preserve. `search` now retries unquoted when the
phrase finds nothing, and says so in the log.

Proven in one run, all three behaviours at once:

    "approximate nearest neighbor search" -> 5 candidates, 5 already held, 0 shelved
    "hybrid retrieval BM25 dense"         -> no exact phrase match, retrying broad
                                          -> 5 candidates, 0 already held, 5 shelved
    "LLM as a judge evaluation"           -> 5 candidates, 5 already held, 0 shelved
    wrote 5 paper(s) to .../ContinuousResearch/2026-08-18/harvest.jsonl

The seen-set suppressing 10 of 15 is the whole point of a recurring mission, and
the 5 that landed are on topic for the first time: RAG architecture evaluation,
agent-controlled search over chat logs, compute-aware retrieval and reranking,
hybrid retrieval in hyperbolic space, sparse-dense fusion limits.

353 tests pass.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-18 07:18:11 -07:00
co-authored by Claude Opus 5
parent 2cd0872e50
commit 850f11838b
4 changed files with 137 additions and 41 deletions
+69 -1
View File
@@ -181,13 +181,51 @@ pub fn arxiv_query(topic: &str) -> String {
format!("all:\"{escaped}\" AND cat:cs.*")
}
/// The looser form of a topic: every term required, but not adjacent.
///
/// A quoted phrase is precise and brittle. "hybrid retrieval BM25 dense" is a
/// perfectly good topic and appears verbatim in no paper on arXiv — measured, 0
/// hits — while requiring the same four terms anywhere returns exactly the
/// hybrid-retrieval evaluations the topic was asking for. Used only when the
/// phrase finds nothing, so an exact match still wins when one exists.
pub fn arxiv_query_broad(topic: &str) -> String {
let terms: Vec<String> = topic
.split_whitespace()
.map(|w| w.trim_matches(|c: char| !c.is_alphanumeric() && c != '-'))
.filter(|w| !w.is_empty())
.map(|w| format!("all:{w}"))
.collect();
if terms.is_empty() {
return arxiv_query(topic);
}
format!("{} AND cat:cs.*", terms.join(" AND "))
}
/// Search arXiv. `max_results` is capped to keep one run bounded.
pub async fn search(query: &str, max_results: usize) -> Result<Vec<Paper>, String> {
let found = search_with(&arxiv_query(query), max_results).await?;
if !found.is_empty() {
return Ok(found);
}
// The phrase matched nothing. Before reporting a quiet day — which the whole
// pipeline treats as a real and legitimate outcome — try the same terms
// unquoted. A topic the operator writes as prose often is not a literal
// phrase in any title, and silently harvesting zero because of punctuation
// would be indistinguishable from a genuinely quiet field.
let broad = arxiv_query_broad(query);
if broad == arxiv_query(query) {
return Ok(found);
}
eprintln!("papers: no exact phrase match for {query:?} — retrying as {broad}");
search_with(&broad, max_results).await
}
async fn search_with(search_query: &str, max_results: usize) -> Result<Vec<Paper>, String> {
let max = max_results.clamp(1, 50);
let url = format!(
"https://export.arxiv.org/api/query?search_query={}&start=0&max_results={max}\
&sortBy=submittedDate&sortOrder=descending",
urlencoding(&arxiv_query(query))
urlencoding(search_query)
);
let body = reqwest::Client::new()
.get(&url)
@@ -314,6 +352,36 @@ mod tests {
}
}
/// The broad form requires every term but not adjacency. Measured: the
/// phrase "hybrid retrieval BM25 dense" has 0 hits on arXiv; the same four
/// terms unquoted return the hybrid-retrieval evaluations that were asked
/// for. Without the fallback that topic silently harvests nothing, which is
/// indistinguishable from a genuinely quiet day.
#[test]
fn the_broad_form_requires_every_term_without_adjacency() {
let q = arxiv_query_broad("hybrid retrieval BM25 dense");
assert_eq!(
q,
"all:hybrid AND all:retrieval AND all:BM25 AND all:dense AND cat:cs.*"
);
assert!(!q.contains('"'), "the broad form must not be a phrase: {q}");
assert!(q.contains("cat:cs.*"), "still category-bound: {q}");
}
/// Punctuation must not leak into a term and must not empty the query.
#[test]
fn the_broad_form_strips_punctuation_and_never_empties() {
assert_eq!(
arxiv_query_broad("retrieval-augmented, generation!"),
"all:retrieval-augmented AND all:generation AND cat:cs.*",
"hyphens are part of a term; trailing punctuation is not"
);
// Nothing usable left: fall back to the phrase form rather than
// emitting a bare `cat:cs.*`, which would match all of computer science.
let q = arxiv_query_broad("!!!");
assert!(q.contains("all:"), "must never degrade to a bare category: {q}");
}
/// Quotes in a topic would terminate the phrase early and corrupt the query.
#[test]
fn quotes_in_a_topic_cannot_break_out_of_the_phrase() {