fix(skills): a pinned skill contradicted the platform inside the same prompt
Extending the Skill-Use mechanical checks, per the baseline's own next step, found something bigger than a missing check. THE DEFECT `workspace-repo-commit-protocol` told agents that `/workspace/repo` was "the ONLY path where source-modifying edits belong". The platform mounts and advertises `/mission/repo` — 26 references in the code; `/workspace/repo` appears in none of them. The skill is bound on 29 role bindings and was delivered TWICE in the run already measured, so an agent received the real path in its tool preamble and a skill contradicting it a few hundred tokens later, in one prompt. An agent that obeyed the skill wrote source into a directory nothing collects — the phase then delivers nothing, and looks like an agent that did no work. The same skill instructed `file_read` / `file_write` / `shell`: ZeroClaw's names, the exact ones `phase_task_text` was fixed to stop advertising after five agents on a single mission spent 7.4k tokens describing the mismatch instead of working. The prompt was corrected and the skill kept saying it. Rewritten against what the code actually does, including the repo-less case (`/mission/repo` exists, is collected as artifacts, has nothing to push). THE CLASS, AND THE GUARD The skills were never checked against the platform they describe. Nothing compared them, so a skill could contradict the prompt it ships inside and stay that way indefinitely — the same shape as PLAN_COMPLETE being documented and never implemented. Two tests in `skills_loader::contradiction_tests` now hold it: no skill may name a repo path the platform does not mount, and none may instruct a tool the agent's subprocess does not expose. The second matches backticked instructions and skips corrective lines, so a skill may still WARN against the wrong names — as this one now does. Both negative-controlled by restoring the old wording. AND THE CHECK THAT STARTED IT `workspace-repo-commit-protocol` now has a Boundary check: writing outside `/mission/repo` fails, and the message names the consequence — a phase that delivers nothing — rather than just the wrong path. docs/SKILL-USE-BASELINE.md records this as the fourth defect the measurement found, and corrects the "next unit of work" note now that this one is done. Full workspace suite green: 106 binaries, zero build errors. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
113de610ec
commit
771092b165
@@ -113,6 +113,7 @@ fn check(skill: &str, output: &str) -> (Verdict, Verdict) {
|
|||||||
match skill {
|
match skill {
|
||||||
"int-xx-marker-protocol" => (marker_compliance(output), marker_boundary(output)),
|
"int-xx-marker-protocol" => (marker_compliance(output), marker_boundary(output)),
|
||||||
"arxiv-daily" => (Verdict::NotApplicable, arxiv_boundary(output)),
|
"arxiv-daily" => (Verdict::NotApplicable, arxiv_boundary(output)),
|
||||||
|
"workspace-repo-commit-protocol" => (Verdict::NotApplicable, workspace_boundary(output)),
|
||||||
_ => (Verdict::NotApplicable, Verdict::NotApplicable),
|
_ => (Verdict::NotApplicable, Verdict::NotApplicable),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -178,6 +179,35 @@ fn looks_like_marker_attempt(line: &str) -> bool {
|
|||||||
KINDS.iter().any(|k| t.starts_with(k))
|
KINDS.iter().any(|k| t.starts_with(k))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The mission checkout is `/mission/repo`. Working anywhere else is not
|
||||||
|
/// delivered.
|
||||||
|
///
|
||||||
|
/// Scored as a BOUNDARY rather than compliance: the skill's positive
|
||||||
|
/// instruction ("cd there at the start of every substantive turn") has no
|
||||||
|
/// reliable trace in the output, but writing source somewhere that is never
|
||||||
|
/// collected does, and it is the failure that costs a whole phase.
|
||||||
|
///
|
||||||
|
/// This check exists because the skill itself was wrong. It taught
|
||||||
|
/// `/workspace/repo` — a path the platform does not mount — while the same
|
||||||
|
/// prompt told the agent `/mission/repo`. An agent that obeyed the skill wrote
|
||||||
|
/// into a directory nothing collects. Corrected 2026-08-19, and
|
||||||
|
/// `skills_loader::contradiction_tests` now holds it.
|
||||||
|
fn workspace_boundary(output: &str) -> Verdict {
|
||||||
|
// Only paths that look like a REPO root the agent chose to work in. A
|
||||||
|
// mention of /tmp is normal; `cd /workspace/repo` is not.
|
||||||
|
const WRONG_ROOTS: &[&str] = &["/workspace/repo", "~/workspace/repo"];
|
||||||
|
for root in WRONG_ROOTS {
|
||||||
|
if output.contains(root) {
|
||||||
|
return Verdict::Fail(format!(
|
||||||
|
"worked in {root} — the mission checkout is /mission/repo, so \
|
||||||
|
anything written there is never collected and the phase \
|
||||||
|
delivers nothing"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Verdict::Pass
|
||||||
|
}
|
||||||
|
|
||||||
/// `arxiv-daily` forbids searching arXiv — the harvest already ran.
|
/// `arxiv-daily` forbids searching arXiv — the harvest already ran.
|
||||||
///
|
///
|
||||||
/// This is the one boundary we have watched an agent cross in production, so it
|
/// This is the one boundary we have watched an agent cross in production, so it
|
||||||
@@ -442,6 +472,32 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Agent-authored skills must stay visible as such in the report.
|
/// Agent-authored skills must stay visible as such in the report.
|
||||||
|
#[test]
|
||||||
|
fn writing_outside_the_mission_checkout_crosses_the_boundary() {
|
||||||
|
let prompt = rendered(&[("workspace-repo-commit-protocol", "Work in /mission/repo.")]);
|
||||||
|
|
||||||
|
let wrong = score(
|
||||||
|
&prompt,
|
||||||
|
"cd /workspace/repo && git add -A && git commit -m 'INT-01 done'",
|
||||||
|
&builtin,
|
||||||
|
);
|
||||||
|
match &wrong[0].boundary {
|
||||||
|
Verdict::Fail(why) => assert!(
|
||||||
|
why.contains("never collected"),
|
||||||
|
"the failure must name the consequence — a phase that delivers \
|
||||||
|
nothing — not just the wrong path: {why}"
|
||||||
|
),
|
||||||
|
other => panic!("the wrong checkout must be caught; got {other:?}"),
|
||||||
|
}
|
||||||
|
|
||||||
|
let right = score(
|
||||||
|
&prompt,
|
||||||
|
"cd /mission/repo && git add -A && git commit -m 'INT-01 done'",
|
||||||
|
&builtin,
|
||||||
|
);
|
||||||
|
assert_eq!(right[0].boundary, Verdict::Pass);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_source_of_a_skill_is_carried_into_its_score() {
|
fn the_source_of_a_skill_is_carried_into_its_score() {
|
||||||
let prompt = rendered(&[("self-made", "x")]);
|
let prompt = rendered(&[("self-made", "x")]);
|
||||||
|
|||||||
@@ -170,3 +170,107 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod contradiction_tests {
|
||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
|
fn skills_root() -> PathBuf {
|
||||||
|
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||||
|
.join("../../skills")
|
||||||
|
.canonicalize()
|
||||||
|
.expect("skills dir")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn all_skills() -> Vec<(String, String)> {
|
||||||
|
fn walk(dir: &std::path::Path, out: &mut Vec<(String, String)>) {
|
||||||
|
for e in std::fs::read_dir(dir).expect("read skills dir") {
|
||||||
|
let p = e.expect("entry").path();
|
||||||
|
if p.is_dir() {
|
||||||
|
walk(&p, out);
|
||||||
|
} else if p.extension().and_then(|x| x.to_str()) == Some("md") {
|
||||||
|
out.push((
|
||||||
|
p.file_name().unwrap().to_string_lossy().to_string(),
|
||||||
|
std::fs::read_to_string(&p).expect("read skill"),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let mut out = Vec::new();
|
||||||
|
walk(&skills_root(), &mut out);
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
/// No skill may teach a workspace path the platform does not use.
|
||||||
|
///
|
||||||
|
/// `workspace-repo-commit-protocol` told agents that `/workspace/repo` was
|
||||||
|
/// "the ONLY path where source-modifying edits belong". The platform mounts
|
||||||
|
/// and advertises `/mission/repo` — in 26 places — and `/workspace/repo`
|
||||||
|
/// appears nowhere in the code. The skill is pinned on 29 role bindings and
|
||||||
|
/// was delivered twice in a single measured run, so agents received the
|
||||||
|
/// platform's real path and a skill contradicting it in the SAME prompt.
|
||||||
|
#[test]
|
||||||
|
fn no_skill_teaches_a_repo_path_the_platform_does_not_mount() {
|
||||||
|
let mut offenders = Vec::new();
|
||||||
|
for (name, body) in all_skills() {
|
||||||
|
if body.contains("/workspace/repo") {
|
||||||
|
offenders.push(name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert!(
|
||||||
|
offenders.is_empty(),
|
||||||
|
"{} skill(s) name /workspace/repo; the mission checkout is \
|
||||||
|
/mission/repo, so an agent following them writes somewhere that is \
|
||||||
|
never delivered: {}",
|
||||||
|
offenders.len(),
|
||||||
|
offenders.join(", ")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// No skill may instruct an agent to call a tool it does not have.
|
||||||
|
///
|
||||||
|
/// Every mission turn ends in `claude -p`, so the tools are Claude Code's
|
||||||
|
/// (`Read`/`Edit`/`Write`/`Bash`/`Glob`/`Grep`). `phase_task_text` used to
|
||||||
|
/// advertise ZeroClaw's names and was fixed after five agents spent 7.4k
|
||||||
|
/// tokens on one mission describing the mismatch instead of working — and
|
||||||
|
/// the same wrong names survived inside a pinned skill.
|
||||||
|
///
|
||||||
|
/// Matched as a backticked instruction, not as bare words: a skill may
|
||||||
|
/// legitimately DISCUSS these names, as this one now does when warning
|
||||||
|
/// against them.
|
||||||
|
#[test]
|
||||||
|
fn no_skill_instructs_an_agent_to_call_a_zeroclaw_tool() {
|
||||||
|
const ZEROCLAW_TOOLS: &[&str] = &[
|
||||||
|
"`file_read`",
|
||||||
|
"`file_write`",
|
||||||
|
"`file_edit`",
|
||||||
|
"`content_search`",
|
||||||
|
"`glob_search`",
|
||||||
|
];
|
||||||
|
let mut offenders = Vec::new();
|
||||||
|
for (name, body) in all_skills() {
|
||||||
|
// The line has to READ as an instruction. "Do not reach for
|
||||||
|
// `file_read`" is the correction, not the defect.
|
||||||
|
for line in body.lines() {
|
||||||
|
let l = line.to_ascii_lowercase();
|
||||||
|
if l.contains("do not")
|
||||||
|
|| l.contains("never")
|
||||||
|
|| l.contains("instead of")
|
||||||
|
|| l.contains("not what")
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if ZEROCLAW_TOOLS.iter().any(|t| line.contains(t)) {
|
||||||
|
offenders.push(format!("{name}: {}", line.trim()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert!(
|
||||||
|
offenders.is_empty(),
|
||||||
|
"{} skill line(s) tell an agent to use a tool its subprocess does \
|
||||||
|
not expose:\n {}",
|
||||||
|
offenders.len(),
|
||||||
|
offenders.join("\n ")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -92,7 +92,7 @@ this one without first establishing that floor.
|
|||||||
|
|
||||||
## What the measurement found
|
## What the measurement found
|
||||||
|
|
||||||
Three defects, none of which any test or log would have surfaced.
|
Four defects, none of which any test or log would have surfaced.
|
||||||
|
|
||||||
### 1. The prompt format made its own record unparseable
|
### 1. The prompt format made its own record unparseable
|
||||||
|
|
||||||
@@ -115,7 +115,31 @@ A provenance record of something that did not happen is worse than no record: it
|
|||||||
is the wrong answer, delivered confidently. Recording now happens inside each
|
is the wrong answer, delivered confidently. Recording now happens inside each
|
||||||
tier, and a test asserts every launcher records the prompt it actually sends.
|
tier, and a test asserts every launcher records the prompt it actually sends.
|
||||||
|
|
||||||
### 3. The skill documents a marker the platform never implemented
|
### 3. A pinned skill contradicted the platform in the same prompt
|
||||||
|
|
||||||
|
`workspace-repo-commit-protocol` told agents that **`/workspace/repo`** was "the
|
||||||
|
ONLY path where source-modifying edits belong". The platform mounts and
|
||||||
|
advertises **`/mission/repo`** — 26 references in the code; `/workspace/repo`
|
||||||
|
appears in none. The skill is bound on **29 role bindings** and was delivered
|
||||||
|
twice in run 2, so agents received the real path in the tool preamble and a
|
||||||
|
skill contradicting it a few hundred tokens later.
|
||||||
|
|
||||||
|
It also instructed `file_read` / `file_write` / `shell` — ZeroClaw's tool names,
|
||||||
|
the exact ones `phase_task_text` was fixed to stop advertising after five agents
|
||||||
|
on one mission spent 7.4k tokens describing the mismatch instead of working.
|
||||||
|
|
||||||
|
An agent that obeyed this skill wrote source into a directory nothing collects,
|
||||||
|
and reached for tools its subprocess does not expose. Rewritten against what the
|
||||||
|
code actually does, with two guards in `skills_loader::contradiction_tests`: no
|
||||||
|
skill may name a repo path the platform does not mount, and none may instruct a
|
||||||
|
tool the agent does not have. Both negative-controlled.
|
||||||
|
|
||||||
|
This is the same shape as the finding below and it is worth stating as a class:
|
||||||
|
**the skills were never checked against the platform they describe.** Nothing
|
||||||
|
compared them, so a skill could contradict the prompt it ships inside and stay
|
||||||
|
that way indefinitely.
|
||||||
|
|
||||||
|
### 4. The skill documents a marker the platform never implemented
|
||||||
|
|
||||||
`int-xx-marker-protocol` lists `PLAN_COMPLETE: INT-NN` in its ladder.
|
`int-xx-marker-protocol` lists `PLAN_COMPLETE: INT-NN` in its ladder.
|
||||||
`task_card_parser` has **no such kind** and never has. An agent following the
|
`task_card_parser` has **no such kind** and never has. An agent following the
|
||||||
@@ -136,11 +160,11 @@ skill — is deliberately left as a decision rather than guessed at here.
|
|||||||
|
|
||||||
- **Two runs, one tier, one workflow.** Nothing here generalises to the microVM
|
- **Two runs, one tier, one workflow.** Nothing here generalises to the microVM
|
||||||
or session tiers yet, and this document should not be read as if it does.
|
or session tiers yet, and this document should not be read as if it does.
|
||||||
- **7 of 9 skills scored `not_applicable`** on both observable axes. That is not
|
- **Most skills score `not_applicable`** on both observable axes. That is not a
|
||||||
a pass. It means we cannot currently tell whether those skills changed
|
pass. It means we cannot currently tell whether those skills changed anything.
|
||||||
anything, and the next unit of work is mechanical checks for more of them —
|
`workspace-repo-commit-protocol` now has a Boundary check (writing outside
|
||||||
`workspace-repo-commit-protocol` and `small-focused-commits` both have
|
`/mission/repo`); `small-focused-commits` and `tdd-red-green-refactor` remain
|
||||||
checkable consequences in git history.
|
candidates, and both need the repository diff rather than the turn text.
|
||||||
- **No agent-authored skill has been measured.** Self-authoring shipped in the
|
- **No agent-authored skill has been measured.** Self-authoring shipped in the
|
||||||
same pass; `source_kind` is carried through the scorer specifically so a
|
same pass; `source_kind` is carried through the scorer specifically so a
|
||||||
rising score on agent-authored skills is visible rather than averaged in.
|
rising score on agent-authored skills is visible rather than averaged in.
|
||||||
|
|||||||
@@ -1,54 +1,77 @@
|
|||||||
---
|
---
|
||||||
name: workspace-repo-commit-protocol
|
name: workspace-repo-commit-protocol
|
||||||
description: How to interact with /workspace/repo — the mission's checked-out codebase — and how to commit + push meaningful changes back.
|
description: How to work inside /mission/repo — the mission's checked-out codebase — and how to commit meaningful changes back.
|
||||||
when_to_use: You are a coder, committer, or any role that edits code. Pin this at turn start so you never lose orientation.
|
when_to_use: You are a coder, committer, or any role that edits code. Pin this at turn start so you never lose orientation.
|
||||||
tags: [foundation, coding, git]
|
tags: [foundation, coding, git]
|
||||||
---
|
---
|
||||||
|
|
||||||
# Workspace repo + commit protocol
|
# Mission repo + commit protocol
|
||||||
|
|
||||||
You are running inside a mission's team container. The clawhdf5/backend/whatever repository the mission targets is bind-mounted at **`/workspace/repo`**. That is the ONLY path where source-modifying edits belong.
|
The repository this mission targets is checked out at **`/mission/repo`**. That
|
||||||
|
is the only path where source-modifying edits belong.
|
||||||
|
|
||||||
## Ground rules
|
## Ground rules
|
||||||
|
|
||||||
1. **`cd /workspace/repo` at the start of every substantive turn.** If you `pwd` and it isn't `/workspace/repo`, cd there first — your default CWD is the ZeroClaw agent workspace (`~/workspace`), which is scratch storage, not the codebase.
|
1. **`cd /mission/repo` at the start of every substantive turn.** If you `pwd`
|
||||||
2. **All `file_read` / `file_write` / `shell` calls that touch source use paths under `/workspace/repo`.** Anything else is scratch — the mission will not persist it.
|
and it is somewhere else, cd there first.
|
||||||
3. **Never write files outside `/workspace/repo` and expect them to survive** — the agent workspace resets, `/tmp` is per-container ephemeral.
|
2. **Every read and write that touches source uses a path under
|
||||||
|
`/mission/repo`.** Anything else is scratch and will not be delivered.
|
||||||
|
3. On a mission with **no** repository, `/mission/repo` still exists and is
|
||||||
|
writable — it is a scratch workspace, every file you leave there is
|
||||||
|
collected when the phase ends and published as a mission artifact, and there
|
||||||
|
is nothing to commit or push. Your task text says which kind of mission this
|
||||||
|
is; believe it over any assumption.
|
||||||
|
|
||||||
|
## Use the tool names your prompt gives you
|
||||||
|
|
||||||
|
Your turn runs through Claude Code, so the tools are `Read`, `Edit`, `Write`,
|
||||||
|
`Bash`, `Glob`, `Grep`. Your prompt lists them explicitly — use those names.
|
||||||
|
|
||||||
|
Do not reach for `file_read`, `file_write`, `content_search` or `shell`. Those
|
||||||
|
are ZeroClaw's names, they are not what your subprocess exposes, and agents that
|
||||||
|
tried them spent whole turns describing the mismatch instead of working.
|
||||||
|
|
||||||
## Commit protocol
|
## Commit protocol
|
||||||
|
|
||||||
When (and only when) you have a meaningful, tested change:
|
When, and only when, you have a meaningful, tested change:
|
||||||
|
|
||||||
```
|
```
|
||||||
cd /workspace/repo
|
cd /mission/repo
|
||||||
git status # sanity-check what you touched
|
git status # what did you actually touch
|
||||||
git diff --stat # confirm scope matches the plan
|
git diff --stat # does the scope match the plan
|
||||||
git add -A
|
git add -A
|
||||||
git commit -m "<INT-NN> <one-line title>
|
git commit -m "<INT-NN> <one-line title>
|
||||||
|
|
||||||
<one-paragraph rationale — WHY, not what>
|
<one paragraph on WHY, not what>
|
||||||
|
|
||||||
Refs: INT-NN
|
Refs: INT-NN
|
||||||
"
|
"
|
||||||
git push
|
|
||||||
```
|
```
|
||||||
|
|
||||||
- **Commit message uses the mission's INT-XX marker** on the subject line — the mission's task-card parser (Slice 5) advances state on it.
|
- **Put the INT-XX marker on the subject line.** The task-card parser advances
|
||||||
- **One INT per commit** unless the change genuinely can't be split; split-when-in-doubt.
|
mission state on it.
|
||||||
- **Never `--force`, never rewrite pushed history** without an explicit `HANDOFF: safe to force-push` from the reviewer.
|
- **One INT per commit** unless the change genuinely cannot be split. Split when
|
||||||
|
in doubt: a commit covering three items cannot be reverted for one of them.
|
||||||
|
- **Never `--force`, never rewrite pushed history** without an explicit
|
||||||
|
`HANDOFF: safe to force-push` from the reviewer.
|
||||||
|
- Push only if your task says to. Many missions deliver by having the platform
|
||||||
|
diff your checkout, and a phase that pushes when it should not is harder to
|
||||||
|
undo than one that did not push.
|
||||||
|
|
||||||
## When NOT to commit
|
## When NOT to commit
|
||||||
|
|
||||||
- Tests failing → fix or revert; never commit red.
|
- Tests failing. Fix or revert; never commit red.
|
||||||
- Reviewer emitted `REVIEW_BLOCK: INT-NN — <reason>` for the current item.
|
- The reviewer emitted `REVIEW_BLOCK: INT-NN` for the current item.
|
||||||
- The change is a WIP or exploratory — that lives in the agent's scratch workspace, not the repo.
|
- The change is exploratory. That is not what the mission branch is for.
|
||||||
|
|
||||||
## Emit the completion marker
|
## Emit the completion marker
|
||||||
|
|
||||||
After a successful push, on a line by itself:
|
After the work is genuinely done, on a line by itself:
|
||||||
|
|
||||||
```
|
```
|
||||||
COMPLETED: INT-NN
|
COMPLETED: INT-NN
|
||||||
```
|
```
|
||||||
|
|
||||||
The mission loop advances on that marker. If you didn't push, don't emit it.
|
Exactly one INT id, no bold, no code fence — the parser takes the literal line
|
||||||
|
and rejects anything else. The mission loop advances on it, so emitting one you
|
||||||
|
cannot back up desynchronizes the mission from the repository.
|
||||||
|
|||||||
Reference in New Issue
Block a user