fix(missions): a caller's own task does not inherit the recipe's done_when
A recipe's completion condition is a condition on the recipe's own task. phases_for_create merged the recipe config under the caller's, so a phase that supplied a different task and no condition inherited a condition about work it was never given: research_and_code's coding phase carries "an implementation for each INT-XX item in IMPLEMENTATION_BRIEF", and a phase asked to write CHAIN.md failed on it, honestly, every time (01a0c20d, 01a0c493). Decided from the caller's config before the merge (afterwards a recipe task and a caller task look the same): caller task + no caller condition → the recipe's done_when/done_when_check are not inherited. A caller condition is kept; a phase with neither keeps the recipe's pair. Test fixture now carries a recipe task+condition. Harness: the triage agreement line dedupes per skill and excludes skills whose Trigger is not observable (always_inject is inlined). First live datapoint, 01a0c493: for 'create CHAIN.md and commit' Jev's top picks were workspace-repo-commit-protocol 0.63 / small-focused-commits 0.57; the agent read code-review-checklist (~0) and nothing else. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
co-authored by
Claude Opus 5
parent
37eb6bcad1
commit
0d46f892db
@@ -1246,12 +1246,21 @@ assert_skill_triage() { # <token> <mission> <label>
|
||||
# not expect. Only meaningful when the arm is a retrieval arm.
|
||||
agree=$(api "$token" GET "/api/missions/$mission/skill-use" | python3 -c '
|
||||
import json,sys
|
||||
d=json.load(sys.stdin); sk=d.get("skills") or []
|
||||
if not sk or all(s.get("triage_p") is None for s in sk): print("n/a"); sys.exit()
|
||||
def read(s): return s["trigger"].get("verdict") in ("pass",) if isinstance(s.get("trigger"),dict) else str(s.get("trigger")).startswith("pass")
|
||||
applies=[s for s in sk if (s.get("triage_p") or 0)>=0.5]
|
||||
hit=sum(1 for s in applies if read(s)); read_unexpected=sum(1 for s in sk if read(s) and (s.get("triage_p") or 0)<0.5)
|
||||
print(f"oracle says {len(applies)} of {len(sk)} delivered skills apply; agent read {hit} of those and {read_unexpected} it did not expect")' 2>/dev/null)
|
||||
d=json.load(sys.stdin); rows=d.get("skills") or []
|
||||
if not rows or all(s.get("triage_p") is None for s in rows): print("n/a"); sys.exit()
|
||||
# One row per skill per phase: collapse to the skill, keeping any pass. Skills
|
||||
# whose Trigger is not observable (always_inject, inlined) cannot be "not read".
|
||||
by={}
|
||||
for s in rows:
|
||||
v=s["trigger"].get("verdict") if isinstance(s.get("trigger"),dict) else str(s.get("trigger"))
|
||||
e=by.setdefault(s["skill"],{"p":s.get("triage_p") or 0,"read":False,"obs":False})
|
||||
e["p"]=max(e["p"],s.get("triage_p") or 0)
|
||||
if v=="pass": e["read"]=True
|
||||
if v in ("pass","fail","not_applicable"): e["obs"]=True
|
||||
obs={k:v for k,v in by.items() if v["obs"]}
|
||||
applies=[k for k,v in obs.items() if v["p"]>=0.5]
|
||||
hit=[k for k in applies if obs[k]["read"]]; unexpected=[k for k,v in obs.items() if v["read"] and v["p"]<0.5]
|
||||
print(f"of {len(obs)} observable skills the oracle says {len(applies)} apply ({", ".join(applies) or "-"}); agent read {len(hit)} of those and {len(unexpected)} it did not expect ({", ".join(unexpected) or "-"})")' 2>/dev/null)
|
||||
case "$agree" in
|
||||
""|n/a) pass "$label-triage: agreement n/a on this arm (no retrieval to compare against)" ;;
|
||||
*) pass "$label-triage: $agree" ;;
|
||||
|
||||
Reference in New Issue
Block a user