feat(skill-use): red-first is observable from what the RUNS reported

The open item said this needed the repository diff rather than tool
order. That was wrong, and the skill says why: "Commit the RED-to-GREEN
pair as one commit." The failing test and its fix land together by
instruction, so the diff and the commit history are as blind as the tool
ordering already was — in Rust one `Edit` adds the implementation and its
`#[cfg(test)] mod tests` in the same call.

The only remaining witness is what each test run itself printed, and the
tap was throwing it away. Claude Code's PostToolUse payload carries
`tool_response` — verified against the real binary, keys
stdout/stderr/interrupted, plus `duration_ms` and `tool_use_id`.

So `Observed.response` now keeps it, for COMMANDS only: a `Read`'s
response is the file it just read and a `Write`'s restates its own
argument — both already knowable, both large, and storing them would
double the biggest write path in the system for nothing.

`bounded_response` keeps the **end** of the output, which is the opposite
of `bounded_input` and deliberately so. An argument's meaning is its verb,
at the start. A command's meaning is its verdict, at the end: `cargo test`
prints hundreds of lines and then `test result: ok` or `FAILED`. A
head-biased truncation would keep the noise and discard the only thing
being stored for — negative-controlled with a 400-line fixture.

`red_before_green` now falls through to the run outcomes:

  failing run, then a passing one  → Pass, red then green observed
  every run failed                 → Fail, the loop ends on green
  every run passed                 → NotObservable, and the reason says
                                     why: a test that never failed is
                                     equally what a correct implementation
                                     written first looks like
  no outputs recorded              → NotObservable (pre-capture missions)

Read from the runner's verdict line, not an exit code — the payload
carries none.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
This commit is contained in:
Omar Sobh
2026-08-21 12:45:20 -07:00
co-authored by Claude Opus 5
parent b47ae7fa6b
commit 5220f3bfea
4 changed files with 276 additions and 7 deletions
+62
View File
@@ -79,6 +79,20 @@ pub struct Observed {
/// is per-CONTAINER and every role in a phase shares one, so without this
/// the whole phase arrives as an undifferentiated stream.
pub session: Option<String>,
/// What a **command** produced, bounded by [`bounded_response`].
///
/// Only for tools that run something. `Read`'s response is the file it just
/// read and `Write`'s is a restatement of what was written — both are
/// already knowable from the arguments and the delivered diff, and storing
/// them would double the largest write path in the system for nothing.
///
/// A command's OUTCOME is different: it is the only place a failing test
/// run is visible. Without it "did this phase go red before it went green"
/// cannot be answered from anything — not from tool order (in Rust the
/// unit test lives in the file under test, so one `Edit` adds both), and
/// not from the repository either, because `tdd-red-green-refactor` says
/// in so many words to "commit the RED-to-GREEN pair as one commit".
pub response: Value,
/// The tool's arguments, bounded by [`bounded_input`].
///
/// Kept because the tool NAME alone answers almost nothing. A phase that
@@ -106,6 +120,46 @@ const MAX_ARG_LEN: usize = 512;
/// authority on what was written anyway.
const BODY_KEYS: [&str; 4] = ["content", "new_string", "old_string", "edits"];
/// Tools whose response is an outcome rather than a restatement.
const RESPONSE_TOOLS: [&str; 1] = ["Bash"];
/// How much of a command's output to keep.
const MAX_OUTPUT_LEN: usize = 600;
/// Shrink a command's response, keeping the **end** of its output.
///
/// The opposite of [`bounded_input`], and deliberately so. An argument's
/// meaning is at the start — the verb of the command. A command's meaning is at
/// the END: `cargo test` prints hundreds of lines and then `test result: ok` or
/// `test result: FAILED`, and a head-biased truncation would keep the noise and
/// throw away the verdict, which is the one thing being stored for.
pub fn bounded_response(tool: &str, response: &Value) -> Value {
if !RESPONSE_TOOLS.contains(&tool) {
return Value::Null;
}
let Some(obj) = response.as_object() else {
return Value::Null;
};
let mut out = serde_json::Map::new();
for key in ["stdout", "stderr", "interrupted"] {
match obj.get(key) {
Some(Value::String(s)) if s.len() > MAX_OUTPUT_LEN => {
let start = s
.char_indices()
.map(|(i, _)| i)
.find(|i| *i >= s.len().saturating_sub(MAX_OUTPUT_LEN))
.unwrap_or(0);
out.insert(key.into(), Value::String(format!("[truncated]…{}", &s[start..])));
}
Some(v) => {
out.insert(key.into(), v.clone());
}
None => {}
}
}
Value::Object(out)
}
/// Shrink a tool's arguments to something safe to store on every call.
///
/// Bounded rather than whitelisted on purpose. A whitelist of "interesting"
@@ -250,9 +304,15 @@ pub fn parse(raw: &str) -> Vec<Observed> {
.or_else(|| v.get("toolInput"))
.cloned()
.unwrap_or(Value::Null);
let response = v
.get("tool_response")
.or_else(|| v.get("toolResponse"))
.cloned()
.unwrap_or(Value::Null);
Some(Observed {
path: crate::mission_events::tool_path(&input),
input: bounded_input(&input),
response: bounded_response(&tool, &response),
session: v
.get("session_id")
.or_else(|| v.get("sessionId"))
@@ -351,12 +411,14 @@ mod tests {
path: Some("/mission/repo/src/a.rs".into()),
input: json!({"file_path": "/mission/repo/src/a.rs"}),
session: None,
response: Value::Null,
},
Observed {
tool: "Bash".into(),
path: None,
input: json!({"command": "ls"}),
session: None,
response: Value::Null,
},
]
);