Omar SobhandClaude Opus 5
48606f2fe3
Mission 01a0c940 ran the paper triage live for the first time: 10 papers, 10 tagged, 10 scored — and the relevance scores were 2.93, 2.94, 2.97, 2.97, 2.98, 2.99, 2.99, 2.99, 3.00, 3.00. A spread of 0.07 across a 4-level scale, every answer confident, no ranking information at all. Of course: the harvest runs the operator's own arXiv topic queries, so every paper in the file is about agents by construction. Asked on the same ten abstracts, 'how actionable is it' saturated the same way (spread 0.20). What separated them was the strength of the evidence behind the claims: 1.36 (a benchmark paper) to 3.00 (measured on real systems with ablations), spread 1.64 — and what KIND of paper it is (method / benchmark / measurement / survey / position), with the confidence of that call beside it so an unplaceable paper reads as unplaceable. The manifest now carries those two and no relevance number, and arxiv-daily.md tells the reading agents what each means and why there is no relevance. The general rule, since this class of mistake is invisible — a saturated score looks exactly like a working feature: patterns::spread() with SATURATED_BELOW, and triage_papers warns when a live harvest's scores span less than that. A question that returns the same number for everything is a defect in the question, not a fact about the population. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
3.0 KiB
name, description, when_to_use, tags
| name | description | when_to_use | tags | ||
|---|---|---|---|---|---|
| arxiv-daily | What the arXiv harvest already guarantees, and what an agent must therefore not redo. | You are working with a harvest manifest in a Continuous Research mission. |
|
The harvest is already done when you start
Searching arXiv is not your job. Before your phase launches, the server has already run the harvest: searched each topic, checked the seen-set, downloaded each new PDF, verified it, shelved it in blob storage and written a vault note. It takes seconds and it is deterministic.
Your input is the result:
ContinuousResearch/<date>/harvest.jsonl
{ source, url, title, snippet, first_seen, topic_tags, kind, evidence }
topic_tags, kind and evidence are filled by a decision model before you
see the file. kind.is is one of benchmark / method / measurement / survey /
position, with the confidence of that call beside it — a low confidence means
the paper does not sit cleanly in one, so read before you trust the label.
evidence.score runs 0 (a position piece with no experiments) to 3 (measured
on real systems with ablations); it is the one dimension that separates papers
in a harvest, because every paper in the file already matched your topics.
There is deliberately no "relevance" number: measured on a real harvest it
came back 2.93–3.00 for every paper, which ranks nothing. All three keys are
absent or empty when the triage could not run; that is not a signal about the
paper.
One line per paper that is new since the last run. Papers already covered are not in it — by design, not omission.
What that guarantees
- No repeats across runs.
corpus_itemsis keyedarxiv:<id>with the version suffix stripped, so2401.12345v3cannot reappear as new becausev1was seen. This is the entire reason a recurring mission is worth having. - Every shelved paper has a real PDF. Fetched bytes are rejected unless they
start with
%PDF— arXiv serves an HTML holding page while a paper renders, and shelving that would leave an unreadable file behind a checkmark. - A failed download stays unseen. The checkmark is written last, after the PDF and the note are both on disk, so a paper that failed halfway is retried next run rather than being lost behind a mark that says it was handled.
What that means for you
- Do not search arXiv yourself. Anything you find outside the manifest is either already covered or was deliberately not selected, and shelving it outside the pipeline leaves the seen-set wrong.
- Do not treat an empty manifest as a failure. A quiet day is a real outcome. Say "nothing new today" and stop — do not go looking for something to talk about.
- Do read the paper. The manifest's
snippetis a trimmed abstract, and the abstract is the authors' pitch.curlthe arXiv abstract page for the rest. What a paper claims and what its evaluation shows differ often enough that catching the gap is most of the value you add.