Omar SobhandClaude Opus 5
ceec0423ad
`research_only` is repo-less, one research phase, "produce a markdown
artifact" — and it defaulted to `rust_sdlc`. So it was staffed with a
planner, a coder, a tester, a reviewer and a committer, four of whom had
nothing to do, each carrying the code-and-commit skills its role is bound
to. Measured 2026-08-21: 9 distinct skills across 5 role prompts, ~50KB,
one applicable. That is what "most skills score not_applicable" in the
Skill-Use baseline has been measuring all along — the skills were
correctly bound to their roles; the roles were wrong for the workflow.
None of the three existing research templates fit, so this adds
`topic_research`: frame the brief into answerable questions, gather
evidence with the URL and the quoted passage, check every claim against
its source, write the report. Three roles, four skills, each checked
against its own `when_to_use` before binding — and two obvious candidates
deliberately NOT bound, because `executive-summary-writing` tells the
writer to discard any item not tied to a named project and
`signal-to-noise-ranking` scores relevance the same way. On a standalone
topic report that discards the deliverable.
`default_phase_teams` lets a recipe staff each phase PURPOSE separately,
resolved into `config.phase_teams` at create. A multi-phase recipe does
not have one job: `research_and_code`'s research phase spends a paragraph
of `task` telling its team not to change source files, because
`rust_sdlc` gave that phase a coder and a committer and they did what
coders do — mission 01a00c57 shipped both INT items during RESEARCH and
the coding phase then delivered +0/-0. Prose was the only lever
available; staffing is the actual one.
Also fixed in the three existing research templates, all verified rather
than inferred:
- `papers_research` bound `arxiv-daily` to its DOMAIN SCOUT. That
skill's entire content is "Do not search arXiv yourself — the harvest
already ran", and its `when_to_use` names Continuous Research
missions, which are the only ones the platform writes a harvest
manifest for. The role whose job is searching was bound a skill
forbidding it.
- Its PAPER READER was told to "fetch the PDF, extract text". The
runtime image has no pdftotext, no mutool and no pypdf — checked in
the container. Every paper would have hit the `[read: abstract only]`
fallback, which reads identically to the fallback working as designed.
- `insight_research` cross-referenced "our repos'" history. A mission
binds ONE repo (`missions.repo_id`).
- `codebase_research` wrote to "the Obsidian vault"; no vault is
mounted, and both it and `papers_research` were committing in "PRs",
which the platform does not open.
And `research_only` itself had neither `task` nor `done_when` — the same
defect `benchmark`, `security_hardening` and `research_and_code` were each
fixed for, and it was left out. A phase with no `done_when` is never
judged. It also still asked for `pdf`, a format nothing generates.
Two new guards, both negative-controlled: every team a recipe names must
exist (a typo currently only logs, and the mission is staffed by the
fallback crew looking deliberate), and every `default_phase_teams` key
must be a purpose `purposes_for` actually emits.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
118 lines
5.9 KiB
TOML
118 lines
5.9 KiB
TOML
key = "research_and_code"
|
|
title = "Research + Coding Loop"
|
|
blurb = "Research a topic against a repo, then loop the coding team through the produced INT-XX items until done."
|
|
requires_repo = true
|
|
|
|
default_team_template = "rust_sdlc"
|
|
|
|
# Per-purpose staffing. The research phase's `task` below spends a paragraph
|
|
# telling the team NOT to change source files, because `rust_sdlc` gave that
|
|
# phase a coder, a tester and a committer and they did what coders do — mission
|
|
# 01a00c57 shipped both INT items during RESEARCH (+276/-57) and the coding
|
|
# phase then opened a clean tree and delivered +0/-0. Prose was the only lever
|
|
# available; staffing is the actual one. The `task` stays as the belt to this
|
|
# braces.
|
|
[default_phase_teams]
|
|
research = "topic_research"
|
|
coding = "rust_sdlc"
|
|
|
|
[[phases]]
|
|
kind = "research"
|
|
order_idx = 0
|
|
[phases.config]
|
|
# `pdf` names a format nothing generates — artifacts are served as Markdown.
|
|
produces = ["md"]
|
|
default_topology = "hub_spoke"
|
|
# Research PLANS; coding BUILDS. Without this the split is a fiction:
|
|
# `rust_sdlc` gives the research team coding roles and a writable
|
|
# /mission/repo, so it implements what it finds and the coding phase then
|
|
# opens a clean tree, produces an empty diff and fails. That is exactly how
|
|
# mission 01a00c57 ended — research shipped both INT items itself
|
|
# (+276/-57) and coding delivered +0/-0.
|
|
#
|
|
# It also matters WHERE the work lands: research runs under a gate that
|
|
# does not check tests, so anything it implements reaches a branch without
|
|
# `cargo test` ever running. Keeping implementation in the coding phase is
|
|
# what puts it behind `on_green_tests`.
|
|
task = """
|
|
Produce the implementation brief. Do NOT change source files — the coding \
|
|
phase implements what you specify, and it can only do that if you leave it \
|
|
something to build.
|
|
|
|
Write research/IMPLEMENTATION_BRIEF.md. Give every proposed change its own \
|
|
numbered INT item, and for each one state: the target file path, the problem \
|
|
with the code as it stands, and the change you propose. Read the repository's \
|
|
own ROADMAP.md, IMPROVEMENT_LOG.md and CLAUDE.md first so you do not re-propose \
|
|
work already merged.
|
|
|
|
Reporting that an area is already sound is a real result — say so and move on \
|
|
rather than manufacturing an item for it.
|
|
|
|
Writing or editing files under research/ is expected. Editing anything under \
|
|
crates/ or src/ is not your task this phase.
|
|
"""
|
|
# The goal is stated as what the tree must CONTAIN — never "and nothing
|
|
# else", which measurably makes a judge invent requirements. The
|
|
# no-source-edits constraint lives in `task` above for that reason.
|
|
done_when = "research/IMPLEMENTATION_BRIEF.md exists and describes each proposed change as a numbered INT item carrying its target file path, the problem, and the proposed fix"
|
|
# 3, not 2. On mission 01a00cfa research got no real attempts out of two: pass
|
|
# 1 was spent on a fabricated commit claim the judge correctly rejected, and
|
|
# pass 2 did the work but was judged against a tree the agent's output had not
|
|
# reached yet. A budget that small turns one bad pass into a failed phase.
|
|
max_iterations = 3
|
|
# Safety net, not the main control. If research edits source anyway, that
|
|
# code still has to pass the project's own suite before it reaches a
|
|
# cleanly-named branch — the gap that let two untested source changes ship
|
|
# on mission 01a00c57.
|
|
commit_policy = "on_green_tests"
|
|
|
|
[[phases]]
|
|
kind = "coding"
|
|
order_idx = 1
|
|
[phases.config]
|
|
# NOTE: `loop` is INERT. `phase_config.rs` lists it under
|
|
# DECLARED_BUT_UNREAD — "NOT IMPLEMENTED — phase iteration uses
|
|
# max_iterations + done_when". It is kept only so the intent stays
|
|
# visible next to the keys that actually drive the loop.
|
|
loop = "until_no_more_int_items"
|
|
# The real loop. Without `done_when` the phase never enters
|
|
# `evaluating`, is never judged, and reports `completed` whatever it
|
|
# did — which is how this workflow ran once, unvalidated, and still
|
|
# went green. With it, `evaluate_finished_phases` judges each pass and
|
|
# re-queues with the verdict's guidance until the condition is met or
|
|
# the passes run out (running out is a FAILURE, not a quiet success).
|
|
#
|
|
# Wording follows the measured rule: say what the tree must CONTAIN.
|
|
# Positional phrasing or "and nothing else" makes the judge invent
|
|
# requirements it was never given.
|
|
done_when = "the repository contains an implementation for each INT-XX item listed in the research phase's IMPLEMENTATION_BRIEF, and `cargo test` passes"
|
|
max_iterations = 3
|
|
# The counterpart to the research phase's `task`: research left a brief and
|
|
# no code, so this phase's input is that brief and its output is working
|
|
# source. Stated explicitly because a phase that finds a clean tree and no
|
|
# instructions has historically written a report about the work instead of
|
|
# doing it — four documentation commits and one implementation, on the run
|
|
# that produced the haiku baseline branch.
|
|
task = """
|
|
Implement the INT items in research/IMPLEMENTATION_BRIEF.md.
|
|
|
|
Work through them in order. For each item, change the real source under \
|
|
crates/, run the project's tests, and commit that item before starting the \
|
|
next one. A commit per item is what lets a reviewer see which change belongs \
|
|
to which finding.
|
|
|
|
`cargo test` must pass when you are done — the branch is only published \
|
|
untagged if it does, and a run that leaves the suite red is not finished.
|
|
|
|
Writing a document that describes the change does NOT implement it. If you \
|
|
believe an item should not be built, say so plainly and explain why rather \
|
|
than marking it complete.
|
|
"""
|
|
# Preamble injected at the head of each iteration's task text so
|
|
# the agents know where the repo lives + how to commit. Covered by
|
|
# Slice 3.5c's `workspace-repo-commit-protocol` skill.
|
|
# Only commit when tests pass. Enforced by the team's TEST_PASS
|
|
# marker before the committer runs. If a coding role wants to bypass
|
|
# (rare — pure docs commit), it emits COMMIT_POLICY_OVERRIDE: <reason>.
|
|
commit_policy = "on_green_tests"
|