feat(skills): files is the default delivery arm
deploy / test (push) Successful in 4m59s
deploy / build (push) Successful in 5m28s

The A/B has its answer. Across four matched production runs — same recipe,
same task, same three offered skills — the MCP-door arm retrieved 1 in 9 and
the file arm retrieved 3 of 3, with the judge loop closing on the same run
(01a098dd). A signal, not a rate; but 0, 1, 0 → 3 on an otherwise identical
task is not noise, and the mechanism is explained rather than guessed: the
door is a deferred tool the agents never load, and Read is not.

A code default and not CLAWMATES_SKILL_DELIVERY on one server, for the reason
always_inject moved into the skill files: a setting that exists only in one
deployment is a setting nobody can find. The env var still overrides, and
`index` and `inline` stay selectable per mission so the comparison remains
runnable against one binary.

Garbage in the env var still falls to `inline`, not to the default — an
unreadable value must not silently select an arm that needs something
installed. A test pins the default so the next change to it is a decision
made with the numbers in front of you, not a slip.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
Omar Sobh
2026-09-13 09:41:45 -05:00
co-authored by Claude Opus 5
parent 00160739de
commit 7dd3aa0965
2 changed files with 35 additions and 4 deletions
+34 -3
View File
@@ -46,9 +46,25 @@
/// straight to `ReadMcpResourceTool`.
pub const MCP_SERVER: &str = "clawmates_skills";
/// Selects the arm. Unset or unrecognised means [`Mode::Inline`].
/// Selects the arm. Unset means [`DEFAULT`]; unrecognised means [`Mode::Inline`].
pub const ENV_VAR: &str = "CLAWMATES_SKILL_DELIVERY";
/// The arm a deployment runs when nothing selects one.
///
/// `Files` since 2026-09-13. It was `Inline` — the control arm of an A/B has
/// to be the thing already running — until the A/B produced its answer: the
/// MCP-door arm retrieved 1 skill in 9 across three matched production runs,
/// and the file arm retrieved 3 of 3 on the fourth (`01a098dd`), with the
/// judge loop closing on the same run. That is a signal and not a rate, but
/// 0, 1, 0 → 3 on an otherwise identical task is not noise, and a default that
/// hands agents procedures they demonstrably read beats one that hands them
/// bodies they were never asked to look for.
///
/// A code default and not an env var on one server, because a setting that
/// exists only in one deployment is a setting nobody can find — the exact
/// shape `always_inject` had before it moved into the skill files.
pub const DEFAULT: Mode = Mode::Files;
/// The `# Your skills` preamble under [`Mode::Inline`].
///
/// **Byte-identical to what production has always sent.** The A arm of an A/B
@@ -187,13 +203,15 @@ pub fn parse(s: &str) -> Option<Mode> {
/// The arm this deployment asks for, before the door is taken into account.
pub fn requested() -> Mode {
let Ok(raw) = std::env::var(ENV_VAR) else {
return Mode::Inline;
return DEFAULT;
};
if raw.trim().is_empty() {
return Mode::Inline;
return DEFAULT;
}
match parse(&raw) {
Some(m) => m,
// Garbage falls to `Inline`, not to `DEFAULT`: an unreadable value must
// not silently select an arm that needs something installed.
None => {
eprintln!(
"skill_delivery: {ENV_VAR}={raw:?} is not `inline`, `index` or `files` — \
@@ -389,6 +407,19 @@ mod tests {
assert_eq!(resolve(Mode::Inline, true), Mode::Inline);
}
/// The deployment default is a measured decision; changing it should fail
/// a test so it is made on purpose, with the numbers in front of you.
#[test]
fn the_default_arm_is_files_and_garbage_still_falls_to_inline() {
assert_eq!(DEFAULT, Mode::Files);
assert_eq!(requested_for(&serde_json::json!({})), requested());
assert_eq!(
requested_for(&serde_json::json!({ "skill_delivery": "sideways" })),
requested(),
"an unreadable per-mission value defers to the deployment, as before"
);
}
#[test]
fn the_files_arm_parses_resolves_and_reads_back() {
assert_eq!(parse("files"), Some(Mode::Files));