Omar SobhandClaude Opus 5
b5032a732a
`Sandbox::for_mission` builds the judge's verification copy from the HOST
checkout. In copy mode the agents write inside the container, and their work
only reached the host when `sync_out` ran — in the capture sweep, AFTER the
phase closed. So every phase was judged against a tree that did not yet contain
the pass being judged, and the judge truthfully reported nothing there.
Mission 01a00cfa is the proof. Research pass 2 wrote a 434-line
IMPLEMENTATION_BRIEF.md, `cargo test` passed, and it was pushed to a clean
branch (clawmates/mission-01a00cfa-c69f39fd-i2 at 563cdd21). Its verdict:
failed after 2 pass(es) — met=false — research/IMPLEMENTATION_BRIEF.md
does not exist anywhere
logged one line BEFORE `captured (+434/-0 across 1 file(s))`. A phase that
succeeded was failed because the evidence had not been collected yet.
This hid because it only bites a phase judged on its OWN pass. The v2 coding
verdict cited real commits (339a5bd, 167671f) — research had already synced
that work to the host in an earlier phase.
`evaluate_finished_phases` now runs `sync_out` first, and on failure leaves the
phase `evaluating` for the next sweep rather than recording a verdict nobody
could stand behind — the same policy the capture sweep already applies, for the
same reason. microVM keeps its carve-out: `microvm_executor` collects out of
the guest over this same path before the VM is destroyed.
Research goes to 3 passes. On 01a00cfa it got no real attempts out of two: one
spent on a fabricated commit claim the judge correctly rejected, one on this
bug.
Co-Authored-By: Claude Opus 5 <[email protected]>
106 lines
5.3 KiB
TOML
106 lines
5.3 KiB
TOML
key = "research_and_code"
|
|
title = "Research + Coding Loop"
|
|
blurb = "Research a topic against a repo, then loop the coding team through the produced INT-XX items until done."
|
|
requires_repo = true
|
|
|
|
default_team_template = "rust_sdlc"
|
|
|
|
[[phases]]
|
|
kind = "research"
|
|
order_idx = 0
|
|
[phases.config]
|
|
produces = ["md", "pdf"]
|
|
default_topology = "hub_spoke"
|
|
# Research PLANS; coding BUILDS. Without this the split is a fiction:
|
|
# `rust_sdlc` gives the research team coding roles and a writable
|
|
# /mission/repo, so it implements what it finds and the coding phase then
|
|
# opens a clean tree, produces an empty diff and fails. That is exactly how
|
|
# mission 01a00c57 ended — research shipped both INT items itself
|
|
# (+276/-57) and coding delivered +0/-0.
|
|
#
|
|
# It also matters WHERE the work lands: research runs under a gate that
|
|
# does not check tests, so anything it implements reaches a branch without
|
|
# `cargo test` ever running. Keeping implementation in the coding phase is
|
|
# what puts it behind `on_green_tests`.
|
|
task = """
|
|
Produce the implementation brief. Do NOT change source files — the coding \
|
|
phase implements what you specify, and it can only do that if you leave it \
|
|
something to build.
|
|
|
|
Write research/IMPLEMENTATION_BRIEF.md. Give every proposed change its own \
|
|
numbered INT item, and for each one state: the target file path, the problem \
|
|
with the code as it stands, and the change you propose. Read the repository's \
|
|
own ROADMAP.md, IMPROVEMENT_LOG.md and CLAUDE.md first so you do not re-propose \
|
|
work already merged.
|
|
|
|
Reporting that an area is already sound is a real result — say so and move on \
|
|
rather than manufacturing an item for it.
|
|
|
|
Writing or editing files under research/ is expected. Editing anything under \
|
|
crates/ or src/ is not your task this phase.
|
|
"""
|
|
# The goal is stated as what the tree must CONTAIN — never "and nothing
|
|
# else", which measurably makes a judge invent requirements. The
|
|
# no-source-edits constraint lives in `task` above for that reason.
|
|
done_when = "research/IMPLEMENTATION_BRIEF.md exists and describes each proposed change as a numbered INT item carrying its target file path, the problem, and the proposed fix"
|
|
# 3, not 2. On mission 01a00cfa research got no real attempts out of two: pass
|
|
# 1 was spent on a fabricated commit claim the judge correctly rejected, and
|
|
# pass 2 did the work but was judged against a tree the agent's output had not
|
|
# reached yet. A budget that small turns one bad pass into a failed phase.
|
|
max_iterations = 3
|
|
# Safety net, not the main control. If research edits source anyway, that
|
|
# code still has to pass the project's own suite before it reaches a
|
|
# cleanly-named branch — the gap that let two untested source changes ship
|
|
# on mission 01a00c57.
|
|
commit_policy = "on_green_tests"
|
|
|
|
[[phases]]
|
|
kind = "coding"
|
|
order_idx = 1
|
|
[phases.config]
|
|
# NOTE: `loop` is INERT. `phase_config.rs` lists it under
|
|
# DECLARED_BUT_UNREAD — "NOT IMPLEMENTED — phase iteration uses
|
|
# max_iterations + done_when". It is kept only so the intent stays
|
|
# visible next to the keys that actually drive the loop.
|
|
loop = "until_no_more_int_items"
|
|
# The real loop. Without `done_when` the phase never enters
|
|
# `evaluating`, is never judged, and reports `completed` whatever it
|
|
# did — which is how this workflow ran once, unvalidated, and still
|
|
# went green. With it, `evaluate_finished_phases` judges each pass and
|
|
# re-queues with the verdict's guidance until the condition is met or
|
|
# the passes run out (running out is a FAILURE, not a quiet success).
|
|
#
|
|
# Wording follows the measured rule: say what the tree must CONTAIN.
|
|
# Positional phrasing or "and nothing else" makes the judge invent
|
|
# requirements it was never given.
|
|
done_when = "the repository contains an implementation for each INT-XX item listed in the research phase's IMPLEMENTATION_BRIEF, and `cargo test` passes"
|
|
max_iterations = 3
|
|
# The counterpart to the research phase's `task`: research left a brief and
|
|
# no code, so this phase's input is that brief and its output is working
|
|
# source. Stated explicitly because a phase that finds a clean tree and no
|
|
# instructions has historically written a report about the work instead of
|
|
# doing it — four documentation commits and one implementation, on the run
|
|
# that produced the haiku baseline branch.
|
|
task = """
|
|
Implement the INT items in research/IMPLEMENTATION_BRIEF.md.
|
|
|
|
Work through them in order. For each item, change the real source under \
|
|
crates/, run the project's tests, and commit that item before starting the \
|
|
next one. A commit per item is what lets a reviewer see which change belongs \
|
|
to which finding.
|
|
|
|
`cargo test` must pass when you are done — the branch is only published \
|
|
untagged if it does, and a run that leaves the suite red is not finished.
|
|
|
|
Writing a document that describes the change does NOT implement it. If you \
|
|
believe an item should not be built, say so plainly and explain why rather \
|
|
than marking it complete.
|
|
"""
|
|
# Preamble injected at the head of each iteration's task text so
|
|
# the agents know where the repo lives + how to commit. Covered by
|
|
# Slice 3.5c's `workspace-repo-commit-protocol` skill.
|
|
# Only commit when tests pass. Enforced by the team's TEST_PASS
|
|
# marker before the committer runs. If a coding role wants to bypass
|
|
# (rare — pure docs commit), it emits COMMIT_POLICY_OVERRIDE: <reason>.
|
|
commit_policy = "on_green_tests"
|