---
name: arxiv-daily
description: What the arXiv harvest already guarantees, and what an agent must therefore not redo.
when_to_use: You are working with a harvest manifest in a Continuous Research mission.
tags: [research, sources]
---

# The harvest is already done when you start

Searching arXiv is not your job. Before your phase launches, the server has
already run the harvest: searched each topic, checked the seen-set, downloaded
each new PDF, verified it, shelved it in blob storage and written a vault note.
It takes seconds and it is deterministic.

Your input is the result:

```
ContinuousResearch/<date>/harvest.jsonl
  { source, url, title, snippet, first_seen, topic_tags, kind, evidence }
```

`topic_tags`, `kind` and `evidence` are filled by a decision model before you
see the file. `kind.is` is one of benchmark / method / measurement / survey /
position, with the confidence of that call beside it — a low confidence means
the paper does not sit cleanly in one, so read before you trust the label.
`evidence.score` runs 0 (a position piece with no experiments) to 3 (measured
on real systems with ablations); it is the one dimension that separates papers
in a harvest, because every paper in the file already matched your topics.
There is deliberately no "relevance" number: measured on a real harvest it
came back 2.93–3.00 for every paper, which ranks nothing. All three keys are
absent or empty when the triage could not run; that is not a signal about the
paper.

One line per paper that is **new since the last run**. Papers already covered
are not in it — by design, not omission.

## What that guarantees

- **No repeats across runs.** `corpus_items` is keyed `arxiv:<id>` with the
  version suffix stripped, so `2401.12345v3` cannot reappear as new because
  `v1` was seen. This is the entire reason a recurring mission is worth having.
- **Every shelved paper has a real PDF.** Fetched bytes are rejected unless they
  start with `%PDF` — arXiv serves an HTML holding page while a paper renders,
  and shelving that would leave an unreadable file behind a checkmark.
- **A failed download stays unseen.** The checkmark is written last, after the
  PDF and the note are both on disk, so a paper that failed halfway is retried
  next run rather than being lost behind a mark that says it was handled.

## What that means for you

- **Do not search arXiv yourself.** Anything you find outside the manifest is
  either already covered or was deliberately not selected, and shelving it
  outside the pipeline leaves the seen-set wrong.
- **Do not treat an empty manifest as a failure.** A quiet day is a real
  outcome. Say "nothing new today" and stop — do not go looking for something to
  talk about.
- **Do read the paper.** The manifest's `snippet` is a trimmed abstract, and the
  abstract is the authors' pitch. `curl` the arXiv abstract page for the rest.
  What a paper claims and what its evaluation shows differ often enough that
  catching the gap is most of the value you add.