Commit Graph
2 Commits
Author SHA1 Message Date
Omar SobhandClaude Opus 5 48606f2fe3 fix(triage): a relevance score on a pre-filtered corpus ranks nothing — measure evidence instead
deploy / test (push) Successful in 5m30s
deploy / build (push) Successful in 5m57s
Mission 01a0c940 ran the paper triage live for the first time: 10 papers,
10 tagged, 10 scored — and the relevance scores were 2.93, 2.94, 2.97,
2.97, 2.98, 2.99, 2.99, 2.99, 3.00, 3.00. A spread of 0.07 across a
4-level scale, every answer confident, no ranking information at all. Of
course: the harvest runs the operator's own arXiv topic queries, so every
paper in the file is about agents by construction. Asked on the same ten
abstracts, 'how actionable is it' saturated the same way (spread 0.20).

What separated them was the strength of the evidence behind the claims:
1.36 (a benchmark paper) to 3.00 (measured on real systems with
ablations), spread 1.64 — and what KIND of paper it is (method /
benchmark / measurement / survey / position), with the confidence of that
call beside it so an unplaceable paper reads as unplaceable. The manifest
now carries those two and no relevance number, and arxiv-daily.md tells
the reading agents what each means and why there is no relevance.

The general rule, since this class of mistake is invisible — a saturated
score looks exactly like a working feature: patterns::spread() with
SATURATED_BELOW, and triage_papers warns when a live harvest's scores
span less than that. A question that returns the same number for
everything is a defect in the question, not a fact about the population.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
2026-09-22 08:19:41 -05:00
Omar SobhandClaude Opus 5 a2d7e3ea92 feat(skills): author the Continuous Research skills, and make its roles honest
deploy / test (push) Successful in 4m4s
deploy / build (push) Successful in 1m45s
The boot log has said `continuous_research — 4 role skills bound, 8 unresolved`
on every start. Those eight roles ran without the instructions their template
promised them, silently: `skills_loader` reports the miss and carries on.

Worse than the missing files was what the prompts described. v1 told the
harvester to sweep "RSS feeds, GitHub trending, HN front page, YouTube /
podcast RSS" — none of which exist. `harvest.rs` searches arXiv and nothing
else. A role prompt describing a machine that was never built is the failure
this codebase keeps paying for, so v2 describes the machine that exists.

Roles now match the pipeline: paper_reader (the harvest already ran; read the
manifest and the papers), signal_ranker (unchanged in spirit), script_writer
(the podcast half, which had no role at all).

Seven skills authored under skills/research/, kebab-case to match the loader —
team_template_loader.rs:177-181 documents the snake_case/kebab-case trap that
already unbinds skills elsewhere:

  arxiv-daily                  what the harvest guarantees, so an agent does
                               NOT re-search arXiv and corrupt the seen-set
  paper-to-project-relevance   name a file or roadmap item, or say "no bearing"
  duplicate-detection          the seen-set catches identity; this catches the
                               same work under a different id
  signal-to-noise-ranking      novelty/relevance/depth, and the two biases to
                               resist (recency up, inconvenience down)
  executive-summary-writing    what it is / why it matters / what to do — decide,
                               do not hedge
  obsidian-vault-conventions   the vault is a human's live workspace; never
                               main, never reorganise, hash the body not the file
  podcast-dialogue-writing     write for someone on a treadmill; the 10-70 char
                               highlight bound is the API's, not a style rule

`web_fetch` dropped from mcp_bundles: runtime_provision.rs binds every mission
claw to `["clawmates_door"]` and never reads that field, so declaring it
instructed roles to use a tool that never arrived. The prompts say `curl` via
Bash, which is what they actually have.

Boot now reports `continuous_research — 11 role skills bound`, with no
unresolved clause. 344 tests pass.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-17 14:44:03 -07:00