---
name: arxiv-daily
description: What the arXiv harvest already guarantees, and what an agent must therefore not redo.
when_to_use: You are working with a harvest manifest in a Continuous Research mission.
tags: [research, sources]
---

# The harvest is already done when you start

Searching arXiv is not your job. Before your phase launches, the server has
already run the harvest: searched each topic, checked the seen-set, downloaded
each new PDF, verified it, shelved it in blob storage and written a vault note.
It takes seconds and it is deterministic.

Your input is the result:

```
ContinuousResearch/<date>/harvest.jsonl
  { source, url, title, snippet, first_seen, topic_tags }
```

One line per paper that is **new since the last run**. Papers already covered
are not in it — by design, not omission.

## What that guarantees

- **No repeats across runs.** `corpus_items` is keyed `arxiv:<id>` with the
  version suffix stripped, so `2401.12345v3` cannot reappear as new because
  `v1` was seen. This is the entire reason a recurring mission is worth having.
- **Every shelved paper has a real PDF.** Fetched bytes are rejected unless they
  start with `%PDF` — arXiv serves an HTML holding page while a paper renders,
  and shelving that would leave an unreadable file behind a checkmark.
- **A failed download stays unseen.** The checkmark is written last, after the
  PDF and the note are both on disk, so a paper that failed halfway is retried
  next run rather than being lost behind a mark that says it was handled.

## What that means for you

- **Do not search arXiv yourself.** Anything you find outside the manifest is
  either already covered or was deliberately not selected, and shelving it
  outside the pipeline leaves the seen-set wrong.
- **Do not treat an empty manifest as a failure.** A quiet day is a real
  outcome. Say "nothing new today" and stop — do not go looking for something to
  talk about.
- **Do read the paper.** The manifest's `snippet` is a trimmed abstract, and the
  abstract is the authors' pitch. `curl` the arXiv abstract page for the rest.
  What a paper claims and what its evaluation shows differ often enough that
  catching the gap is most of the value you add.