fix(skills): a pinned skill contradicted the platform inside the same prompt

Extending the Skill-Use mechanical checks, per the baseline's own next step,
found something bigger than a missing check.

THE DEFECT

`workspace-repo-commit-protocol` told agents that `/workspace/repo` was "the
ONLY path where source-modifying edits belong". The platform mounts and
advertises `/mission/repo` — 26 references in the code; `/workspace/repo`
appears in none of them.

The skill is bound on 29 role bindings and was delivered TWICE in the run
already measured, so an agent received the real path in its tool preamble
and a skill contradicting it a few hundred tokens later, in one prompt. An
agent that obeyed the skill wrote source into a directory nothing collects
— the phase then delivers nothing, and looks like an agent that did no work.

The same skill instructed `file_read` / `file_write` / `shell`: ZeroClaw's
names, the exact ones `phase_task_text` was fixed to stop advertising after
five agents on a single mission spent 7.4k tokens describing the mismatch
instead of working. The prompt was corrected and the skill kept saying it.

Rewritten against what the code actually does, including the repo-less case
(`/mission/repo` exists, is collected as artifacts, has nothing to push).

THE CLASS, AND THE GUARD

The skills were never checked against the platform they describe. Nothing
compared them, so a skill could contradict the prompt it ships inside and
stay that way indefinitely — the same shape as PLAN_COMPLETE being
documented and never implemented.

Two tests in `skills_loader::contradiction_tests` now hold it: no skill may
name a repo path the platform does not mount, and none may instruct a tool
the agent's subprocess does not expose. The second matches backticked
instructions and skips corrective lines, so a skill may still WARN against
the wrong names — as this one now does. Both negative-controlled by
restoring the old wording.

AND THE CHECK THAT STARTED IT

`workspace-repo-commit-protocol` now has a Boundary check: writing outside
`/mission/repo` fails, and the message names the consequence — a phase that
delivers nothing — rather than just the wrong path.

docs/SKILL-USE-BASELINE.md records this as the fourth defect the
measurement found, and corrects the "next unit of work" note now that this
one is done.

Full workspace suite green: 106 binaries, zero build errors.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-19 12:51:43 -07:00
co-authored by Claude Opus 5
parent 113de610ec
commit 771092b165
4 changed files with 234 additions and 27 deletions
+31 -7
View File
@@ -92,7 +92,7 @@ this one without first establishing that floor.
## What the measurement found
Three defects, none of which any test or log would have surfaced.
Four defects, none of which any test or log would have surfaced.
### 1. The prompt format made its own record unparseable
@@ -115,7 +115,31 @@ A provenance record of something that did not happen is worse than no record: it
is the wrong answer, delivered confidently. Recording now happens inside each
tier, and a test asserts every launcher records the prompt it actually sends.
### 3. The skill documents a marker the platform never implemented
### 3. A pinned skill contradicted the platform in the same prompt
`workspace-repo-commit-protocol` told agents that **`/workspace/repo`** was "the
ONLY path where source-modifying edits belong". The platform mounts and
advertises **`/mission/repo`** — 26 references in the code; `/workspace/repo`
appears in none. The skill is bound on **29 role bindings** and was delivered
twice in run 2, so agents received the real path in the tool preamble and a
skill contradicting it a few hundred tokens later.
It also instructed `file_read` / `file_write` / `shell` — ZeroClaw's tool names,
the exact ones `phase_task_text` was fixed to stop advertising after five agents
on one mission spent 7.4k tokens describing the mismatch instead of working.
An agent that obeyed this skill wrote source into a directory nothing collects,
and reached for tools its subprocess does not expose. Rewritten against what the
code actually does, with two guards in `skills_loader::contradiction_tests`: no
skill may name a repo path the platform does not mount, and none may instruct a
tool the agent does not have. Both negative-controlled.
This is the same shape as the finding below and it is worth stating as a class:
**the skills were never checked against the platform they describe.** Nothing
compared them, so a skill could contradict the prompt it ships inside and stay
that way indefinitely.
### 4. The skill documents a marker the platform never implemented
`int-xx-marker-protocol` lists `PLAN_COMPLETE: INT-NN` in its ladder.
`task_card_parser` has **no such kind** and never has. An agent following the
@@ -136,11 +160,11 @@ skill — is deliberately left as a decision rather than guessed at here.
- **Two runs, one tier, one workflow.** Nothing here generalises to the microVM
or session tiers yet, and this document should not be read as if it does.
- **7 of 9 skills scored `not_applicable`** on both observable axes. That is not
a pass. It means we cannot currently tell whether those skills changed
anything, and the next unit of work is mechanical checks for more of them —
`workspace-repo-commit-protocol` and `small-focused-commits` both have
checkable consequences in git history.
- **Most skills score `not_applicable`** on both observable axes. That is not a
pass. It means we cannot currently tell whether those skills changed anything.
`workspace-repo-commit-protocol` now has a Boundary check (writing outside
`/mission/repo`); `small-focused-commits` and `tdd-red-green-refactor` remain
candidates, and both need the repository diff rather than the turn text.
- **No agent-authored skill has been measured.** Self-authoring shipped in the
same pass; `source_kind` is carried through the scorer specifically so a
rising score on agent-authored skills is visible rather than averaged in.