Commit Graph
1 Commits
Author SHA1 Message Date
Omar SobhandClaude Opus 5 30eaa50c50 feat(harvest): one run — find, skip what we hold, shelve the rest
Turns the parts into a job. Order is the point: the checkmark list is
consulted BEFORE anything downloads. Checking afterwards would still
dedupe the catalogue while re-downloading every paper we already have,
every week, forever.

Two properties the tests pin down, both learned the hard way this week:

- A quiet week is not a failure. `shelved == 0` with no errors is a
  healthy run against a mature library; `shelved == 0` with errors is
  broken. Harvest::healthy() and ::added_anything() keep those apart
  rather than collapsing them into one ambiguous "did nothing".
- A failed download leaves the paper UNSEEN. Checking it off before the
  PDF is safely shelved would mean one transient network error retires
  that paper permanently. The checkmark is written last, after the bytes
  and the note are both on disk.

The skip test gives every candidate a pdf_url pointing at a closed port,
so if the skip ever regresses the test fails loudly instead of quietly
re-fetching.

Live end-to-end against arXiv, run twice:
  RUN1  3 candidates, 0 already held, 3 shelved, 0 failed
  RUN2  3 candidates, 3 already held, 0 shelved, 0 failed

Library<'_> groups the five values that always describe one library;
passing them loose is how a run shelves into one place and catalogues
into another (also silences clippy::too_many_arguments honestly rather
than by allow).

391 tests, clippy clean.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-03 07:53:56 -07:00