1678452a93aa9a306da23638c22b98b41d52139f
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
850f11838b |
fix(research): the manifest belongs to the mission, and a brittle phrase must not mean silence
Two defects from the first on-topic run.
**1. The manifest could never be updated twice in a day.** It was written into
the VAULT at a per-DATE path, but it is per-RUN data. A second mission the same
day rewrites a file that already exists, and `auto_merge` correctly refused the
whole branch:
diff is not additive (1 non-add change(s), first:
M ContinuousResearch/2026-08-18/harvest.jsonl); left for a human
So `main` kept the FIRST run's manifest, the next mission cloned it, and the
agents analysed yesterday's papers while every log line reported a successful
harvest. The merge policy was right; the placement was wrong. The manifest now
goes into the mission's own checkout after `ensure_checkout`, which keeps the
vault additive and gives each mission exactly its own papers. The agents commit
it alongside their analysis through the normal delivery path.
**2. A quoted phrase that matches nothing looked like a quiet day.** Phrase
search is precise and brittle: "hybrid retrieval BM25 dense" is a reasonable
topic and appears verbatim in no paper on arXiv — measured, 0 hits — while the
same four terms unquoted return exactly the hybrid-retrieval evaluations the
topic asked for. Harvesting zero because of adjacency is indistinguishable from
a genuinely quiet field, which is the distinction `Harvest::healthy()` vs
`added_anything()` exists to preserve. `search` now retries unquoted when the
phrase finds nothing, and says so in the log.
Proven in one run, all three behaviours at once:
"approximate nearest neighbor search" -> 5 candidates, 5 already held, 0 shelved
"hybrid retrieval BM25 dense" -> no exact phrase match, retrying broad
-> 5 candidates, 0 already held, 5 shelved
"LLM as a judge evaluation" -> 5 candidates, 5 already held, 0 shelved
wrote 5 paper(s) to .../ContinuousResearch/2026-08-18/harvest.jsonl
The seen-set suppressing 10 of 15 is the whole point of a recurring mission, and
the 5 that landed are on topic for the first time: RAG architecture evaluation,
agent-controlled search over chat logs, compute-aware retrieval and reranking,
hybrid retrieval in hyperbolic space, sparse-dense fusion limits.
353 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
2cd0872e50 |
fix(papers): the arXiv topic was never actually searched for
The operator's topic went RAW into `search_query=`, unfielded. arXiv matched
essentially nothing, and `sortBy=submittedDate` then returned the newest
submissions across the entire archive — so the library shelved whatever had been
posted in the last few minutes and called it research.
A real run for "agentic topology", "retrieval augmented generation" and "vector
index pruning" shelved, among 13 papers: Galois extensions of geometric fixed
point spectra, a bulk path integral for a quantum black hole microstate, blazar
boosted dark matter in IceCube, and colloidal packing. Nothing was broken —
every layer reported success, the notes were written, the seen-set was updated,
the branch auto-merged. The papers were simply unrelated to anything asked for.
Measured against the live API:
speculative decoding -> pixel-space diffusion, simplicial actions
all:"speculative decoding" -> S2-MoE self-speculative decoding, DARTree
So a bare topic is quoted into `all:` and bound to `cat:cs.*`. The quotes make
it a phrase (unquoted, "vector index pruning" matches any paper containing all
three words anywhere, which is most of cs), and the category bound is needed
because the archive's physics and maths volume dominates any recency sort.
A topic that already starts with a field prefix passes through untouched, so an
operator who knows arXiv syntax keeps control.
The passthrough originally also accepted anything containing " AND "/" OR ", and
the injection test caught it on the first run: `agent" OR cat:hep-th` escaped the
phrase and rewrote the category bound. Only a LEADING field prefix counts now.
348 tests pass.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
e4a395b72e |
feat(papers): find papers on arXiv, shelve the PDF, catalogue the note
Corrects a misread of the design. I had built this as "read the vault to find papers"; the vault is the CARD CATALOGUE, not the source. Papers are found on arXiv, the PDF is pulled down and shelved in our own library, and a note recording it goes in the vault. Three parts, and which is which matters: arXiv — where papers are found blob store — the shelf; the PDF lives there (cm-files, local + S3) the vault — the catalogue; one note per paper, pointing at the shelf The checkmark list (corpus, 0064) is what makes this continuous rather than a job that redoes itself every week — the failure that killed the previous attempt (0030-0044, dropped in 0053). The load-bearing detail: every catalogue note carries `source_id: arxiv:NNNN.NNNNN` in frontmatter, which is exactly the key corpus::parse_note reads. So the checkmark list is rebuildable FROM the vault. If the database were lost, re-indexing restores what we have — the catalogue is authoritative, the index is derived. A test asserts that round trip rather than trusting the two halves to agree. Version suffixes are stripped (2401.12345v3 -> 2401.12345) or a weekly job re-downloads a paper every time authors post a revision. Fetches are rejected unless the bytes start with %PDF: arXiv serves an HTML holding page while a PDF renders, and shelving that leaves a file that looks present and is unreadable. Verified against live arXiv, not fixtures: arxiv:2607.29678 TokTier: Exact Stateful Tokenization for Agentic LLM… arxiv:2607.29677 ExtractBench: A Benchmark for Schema-Guided Enterpri… arxiv:2607.29658 Reusing Past Repairs Through Hierarchical Trajectory… pdf: 1,361,770 bytes, %PDF verified 388 tests, clippy clean. Co-Authored-By: Claude Opus 5 <[email protected]> |