feat(missions): keep the tool's arguments, not just its name
The container tier's first measured mission recorded `Bash × 6` and not one of them said what it ran. Every behavioural question about the phase — did it run the tests, did it commit, did it call an API a skill forbids — was unanswerable from a record that looked complete. `vm_tool_tap::parse` already read `tool_input` to pull the path out of it, then dropped the rest on the floor. It now keeps it, bounded: file bodies (`content`, `new_string`, `old_string`, `edits`) become a byte count, and any other over-long string is truncated with a marker saying so. Bounded rather than whitelisted, because a whitelist silently loses the one argument that matters the first time a tool grows a field. `file.touch` keeps the absolute path in `detail.abs` alongside the repo-relative `target`. Normalising is what the map needs and exactly what destroys "did this write land outside the checkout". `tool.call` also gains `detail.path`, which the World's SSE has been reading and getting a null from on every container-tier call. `mission_events::tool_evidence_for_mission` is the reader — the counterpart to `narrative_for_mission`, and the reason it exists: the narrative is what an agent SAID it did. Host-side only. No image rebuild: the arguments were always in the tap file, the first parse threw them away. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_018i9Ten1LU4jUr5d7TAWda9
This commit is contained in:
co-authored by
Claude Opus 5
parent
0b4d91889a
commit
8cb38d1320
@@ -183,6 +183,68 @@ pub async fn narrative_for_mission(
|
||||
.collect())
|
||||
}
|
||||
|
||||
/// One action an agent took, as a reader gets it back.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct ToolEvidence {
|
||||
/// The tool's name, e.g. `Bash`, `Write`.
|
||||
pub tool: String,
|
||||
/// The absolute path inside the sandbox, when the tool named one.
|
||||
///
|
||||
/// Absolute, unlike the sibling `file.touch` row's `target`. See the note
|
||||
/// in `phase_runner::record_vm_tools`: normalising is what destroys the
|
||||
/// only question a path can settle.
|
||||
pub path: Option<String>,
|
||||
/// The tool's arguments, bounded by `vm_tool_tap::bounded_input`.
|
||||
pub input: Value,
|
||||
}
|
||||
|
||||
impl ToolEvidence {
|
||||
/// The shell command, for the tools that run one.
|
||||
pub fn command(&self) -> Option<&str> {
|
||||
self.input.get("command").and_then(Value::as_str)
|
||||
}
|
||||
}
|
||||
|
||||
/// Every tool call recorded for a mission, in order.
|
||||
///
|
||||
/// The counterpart to [`narrative_for_mission`], and the reason it exists: the
|
||||
/// narrative is what an agent *said* it did. These rows are what it did. A
|
||||
/// measurement built on the narrative alone scores prose, and prose is written
|
||||
/// by the thing being measured.
|
||||
///
|
||||
/// **Bounded by [`PER_PHASE_CAP`].** A phase that ran more tools than the cap
|
||||
/// returns the first `PER_PHASE_CAP` and no marker saying so, so a check that
|
||||
/// concludes "this never happened" from an empty result is only sound for
|
||||
/// phases under the cap. Every check in `skill_use` is one-sided in the safe
|
||||
/// direction for that reason: it reports a violation it can see, never
|
||||
/// compliance it inferred from silence.
|
||||
pub async fn tool_evidence_for_mission(
|
||||
pool: &PgPool,
|
||||
mission_id: Uuid,
|
||||
) -> Result<Vec<ToolEvidence>, sqlx::Error> {
|
||||
let rows: Vec<(Option<String>, Value)> = sqlx::query_as(
|
||||
"SELECT target, detail
|
||||
FROM mission_events
|
||||
WHERE mission_id = $1 AND kind = $2
|
||||
ORDER BY id",
|
||||
)
|
||||
.bind(mission_id)
|
||||
.bind(TOOL_CALL)
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
Ok(rows
|
||||
.into_iter()
|
||||
.map(|(target, detail)| ToolEvidence {
|
||||
tool: target.unwrap_or_default(),
|
||||
path: detail
|
||||
.get("path")
|
||||
.and_then(Value::as_str)
|
||||
.map(str::to_string),
|
||||
input: detail.get("input").cloned().unwrap_or(Value::Null),
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
pub async fn record_all(pool: &PgPool, events: Vec<MissionEvent>) {
|
||||
for e in events {
|
||||
record(pool, e).await;
|
||||
|
||||
Reference in New Issue
Block a user