feat(skills): a skill that must be read cannot be left to be noticed
deploy / test (push) Successful in 5m24s
deploy / build (push) Successful in 5m55s

The `index` arm hands an agent a list of uris and trusts it to fetch what
applies. Measured on the first A/B pair, that is mostly what happens — each
agent fetched the skill bound to its own role and no other, which is the result
that made Trigger observable at all.

`workspace-repo-commit-protocol` is the case it fails on. It scored Trigger=FAIL
beside a PASSING boundary check: the rule was live and unread. A procedure that
applies to everyone who writes reads as nobody's in particular, so no agent
recognises it as theirs and no agent fetches it.

Upstream ZeroClaw arrived at the same place from the other direction and gave
its compact injection mode an `always: true` frontmatter escape hatch (#9520).
This is that hatch as a column: `skills.always_inject`, default FALSE, so
nothing changes for an existing skill and the inline arm is untouched either
way.

Two halves, because delivering it and scoring it are different mistakes:

- Delivery: under `Index`, an `always_inject` skill renders its BODY.
- Scoring: the arm belongs to the PROMPT and `always_inject` belongs to the
  SKILL, so the scorer now asks per skill which one it got. A skill whose body
  is in the prompt was handed over, and a Trigger miss cannot be charged against
  an agent that was never asked to fetch anything.

`skill_was_indexed` reads that off the rendered prompt via `READ_IT`, a
constant now shared with `index_entry` — two spellings of one marker is how a
detector quietly stops detecting.

Suite: 108 binaries, 840 tests, green.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
Omar Sobh
2026-08-27 09:58:01 -05:00
co-authored by Claude Opus 5
parent 563b074116
commit 2f1a870949
5 changed files with 148 additions and 6 deletions
+66 -1
View File
@@ -263,10 +263,19 @@ pub fn score(
.into_iter()
.map(|skill| {
let (compliance, boundary) = check(&skill, evidence);
// The arm belongs to the prompt; `always_inject` belongs to the
// skill. A skill whose BODY is in the prompt was handed over, so
// there is no reaching-for to observe even under `Index` — scoring
// it as a Trigger miss would report a failure against an agent that
// was never asked to fetch anything.
let delivered = match crate::skill_delivery::skill_was_indexed(prompt, &skill) {
Some(false) => crate::skill_delivery::Mode::Inline,
_ => mode,
};
SkillUse {
source_kind: source_kinds(&skill),
trigger: trigger_verdict(
mode,
delivered,
retrieved.contains(&skill),
&compliance,
&boundary,
@@ -1007,6 +1016,62 @@ mod tests {
}
/// The control arm has to be unchanged, or the A/B measures this edit too.
/// The case this whole flag exists for.
///
/// `workspace-repo-commit-protocol` applies to every agent that writes,
/// which is exactly why no agent reads it as *theirs* — it scored
/// Trigger=FAIL beside a passing boundary check on the first A/B pair.
/// Marked `always_inject`, its body is in the prompt under the index arm,
/// and a skill the agent was handed cannot be a reaching-for failure.
#[test]
fn a_skill_delivered_in_full_under_the_index_arm_is_not_a_trigger_miss() {
let prompt = format!(
"{}\n{}{}",
crate::skill_delivery::INDEX_PREAMBLE,
crate::topology_exec::render_pinned_skill(
"workspace-repo-commit-protocol",
"Commit only inside /workspace/repo. Never write outside it.",
),
rendered_index(&[("arxiv-daily", "when sweeping arxiv")]),
);
let tools = [];
let ev = Evidence { text: "", tools: &tools };
let got = score(&prompt, &ev, &|_| "builtin".to_string());
let it = got
.iter()
.find(|u| u.skill == "workspace-repo-commit-protocol")
.expect("the always-injected skill must still be scored");
assert!(
matches!(it.trigger, Verdict::NotObservable(_)),
"handed over, not offered — there is no retrieval to miss: {:?}",
it.trigger
);
}
/// The flag must not leak: a skill still delivered as an index entry keeps
/// being scored on whether it was fetched.
#[test]
fn an_indexed_skill_in_the_same_prompt_is_still_judged_on_retrieval() {
let prompt = format!(
"{}\n{}{}",
crate::skill_delivery::INDEX_PREAMBLE,
crate::topology_exec::render_pinned_skill(
"workspace-repo-commit-protocol",
"Commit only inside /workspace/repo.",
),
rendered_index(&[("arxiv-daily", "when sweeping arxiv")]),
);
for u in score(&prompt, &Evidence { text: "", tools: &[] }, &|_| "builtin".into()) {
if u.skill != "workspace-repo-commit-protocol" {
assert!(
!matches!(u.trigger, Verdict::NotObservable(_)),
"{} was offered by uri, so retrieval is observable for it",
u.skill
);
}
}
}
#[test]
fn the_inline_arm_still_reports_trigger_as_unobservable() {
let prompt = rendered(&[("workspace-repo-commit-protocol", "body")]);