fix(missions): skills can now reach a mission agent at all

Repairing the 55 broken skill bindings made the catalogue correct. This
makes it reachable, which it was not — for any skill, on any mission, since
the catalogue was built.

The skills had exactly ONE delivery channel: the `clawmates_skills` MCP
server. A mission claw could not reach it for three independent reasons:

  1. `provision_claw` wrote the constant `["clawmates_door"]` and ignored
     the template's mcp_bundles — which mission_orchestrator had already
     resolved and stored on the team row.
  2. The runtime config defines no `clawmates_skills` bundle. The live
     local config defines no bundles at all, not even the door.
  3. Mission claws run on `claude_cli`, which the runtime's own config
     comments document as text-only: it cannot surface a tool call, so no
     MCP server is reachable from a mission turn regardless of bundles.

And a mission turn's whole system context is two sentences synthesised from
the role slot in topology_exec::build_prompt. The template's role prose is
not used either — mission_orchestrator documents this, and it means the
role prompts describing which procedures to follow were never read.

Two doc comments in cm-runtime describe the mission path as already having
the summary-and-fetch contract. It never did. The belief was written down
twice and checked zero times, which is why nobody looked — and it is why
the Skill-Use measurement this review planned could only ever have returned
a trigger rate of zero. That would have read as a finding about the agents.

  - provision_claw takes the bundles, with clawmates_door always added: a
    template that forgets to list it must not get an ungated agent
  - all 11 templates now request clawmates_skills; web_fetch removed, since
    a list that is honoured must not name a bundle that does not exist
  - the re-provision sweep re-asserts the team's own stored bundles rather
    than a constant, which would have silently stripped a capability
    mid-mission
  - pinned skill BODIES are injected into the mission prompt, bounded and
    with truncation stated. Bodies, not an index: there is no `skills.read`
    tool on this path, so an index would advertise a capability that does
    not exist — the exact failure this whole change is about

Three tests: the body reaches the prompt, an agent with no skills adds no
heading (an empty "Your skills" section announces skills the agent does not
have), and the composition is exercised separately from the lookup, because
`pinned_skills_text` working and `run_turn` calling it are different claims
and the second is the one that was false.

Also adds the three review documents: CAPABILITY-REVIEW (inventory, what
was repaired, what is deferred and why), PROVENANCE-ASSESSMENT (assess
only, per decision — what each store answers and the two candidate paths),
and RESEARCH-SWEEP (the fortnight's papers and what we did about each,
including the ones we deliberately did nothing about).

Full workspace suite green.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-19 08:24:02 -07:00
co-authored by Claude Opus 5
parent 18dc0b964b
commit e4942ce985
16 changed files with 646 additions and 21 deletions
+2 -2
View File
@@ -49,7 +49,7 @@ pub mod repo_digest;
pub mod root_copy; pub mod root_copy;
mod routes; mod routes;
pub mod runtime_preflight; pub mod runtime_preflight;
mod runtime_provision; pub mod runtime_provision;
pub mod security_scan; pub mod security_scan;
pub mod session_executor; pub mod session_executor;
pub mod skills_loader; pub mod skills_loader;
@@ -59,7 +59,7 @@ pub mod task_card_parser;
pub mod task_card_worker; pub mod task_card_worker;
pub mod team_template_loader; pub mod team_template_loader;
pub mod tool_versions; pub mod tool_versions;
mod topology_exec; pub mod topology_exec;
pub mod topology_worker; pub mod topology_worker;
pub mod validator_preflight; pub mod validator_preflight;
pub mod vm_placement; pub mod vm_placement;
+6 -1
View File
@@ -679,7 +679,12 @@ async fn mint_team_from_template(
// out-of-band via MissionRuntimeProvisioner::pin_agent_workspaces. // out-of-band via MissionRuntimeProvisioner::pin_agent_workspaces.
if let Some(p) = provisioner { if let Some(p) = provisioner {
match p match p
.provision_claw(claw_id, role_model, &template.template.risk_profile) .provision_claw(
claw_id,
role_model,
&template.template.risk_profile,
&template.template.mcp_bundles,
)
.await .await
{ {
Ok(_) => provisioned_claws.push(agent_id), Ok(_) => provisioned_claws.push(agent_id),
+17 -1
View File
@@ -782,7 +782,7 @@ async fn launch_phase(
crate::runtime_provision::RuntimeProvisioner::for_gateway(ec.endpoint.clone()) crate::runtime_provision::RuntimeProvisioner::for_gateway(ec.endpoint.clone())
{ {
let crew = sqlx::query( let crew = sqlx::query(
"SELECT DISTINCT a.id, a.model_binding, t.risk_profile "SELECT DISTINCT a.id, a.model_binding, t.risk_profile, t.mcp_bundles
FROM team_members tm FROM team_members tm
JOIN mission_teams mt ON mt.team_id = tm.team_id JOIN mission_teams mt ON mt.team_id = tm.team_id
JOIN teams t ON t.id = tm.team_id JOIN teams t ON t.id = tm.team_id
@@ -798,11 +798,27 @@ async fn launch_phase(
let aid: uuid::Uuid = row.get("id"); let aid: uuid::Uuid = row.get("id");
let model: Option<String> = row.get("model_binding"); let model: Option<String> = row.get("model_binding");
let risk: Option<String> = row.get("risk_profile"); let risk: Option<String> = row.get("risk_profile");
// Re-assert the team's OWN bundles. Passing a constant
// here would quietly strip `clawmates_skills` from a
// crew that had it, and a re-provision that removes a
// capability is worse than one that never ran: the
// agent keeps working and simply stops being able to
// read its skills, halfway through the mission.
let bundles: Vec<String> = row
.get::<serde_json::Value, _>("mcp_bundles")
.as_array()
.map(|a| {
a.iter()
.filter_map(|v| v.as_str().map(|s| s.to_string()))
.collect()
})
.unwrap_or_default();
match rp match rp
.provision_claw( .provision_claw(
aid, aid,
model.as_deref().unwrap_or("claude"), model.as_deref().unwrap_or("claude"),
risk.as_deref().unwrap_or("research_readonly"), risk.as_deref().unwrap_or("research_readonly"),
&bundles,
) )
.await .await
{ {
+16 -1
View File
@@ -83,6 +83,21 @@ pub(crate) async fn build_team(
.await .await
} }
/// MCP bundles for a team built by the wizard or the planner rather than from a
/// team template.
///
/// These teams have no template, so there is no `mcp_bundles` list to inherit —
/// which previously meant they were provisioned with the door alone and could
/// not reach the skills catalogue at all. `mcp_skills` scopes what it lists to
/// the caller's workspace, so an agent with no template link still sees the
/// global skills, which is the useful half for an ad-hoc team.
fn adhoc_bundles() -> Vec<String> {
vec![
"clawmates_door".to_string(),
"clawmates_skills".to_string(),
]
}
/// Same as `build_team` but with an explicit `lifecycle` (`permanent` | /// Same as `build_team` but with an explicit `lifecycle` (`permanent` |
/// `ephemeral`). Ephemeral teams are torn down by the topology_worker after /// `ephemeral`). Ephemeral teams are torn down by the topology_worker after
/// their last run terminates — used by the Scheduled + Triggered planner modes. /// their last run terminates — used by the Scheduled + Triggered planner modes.
@@ -140,7 +155,7 @@ pub(crate) async fn build_team_with_lifecycle(
// Ad-hoc team-wizard teams aren't mission-bound, so they use the // Ad-hoc team-wizard teams aren't mission-bound, so they use the
// default per-agent workspace under <install>/agents/<alias>/workspace/. // default per-agent workspace under <install>/agents/<alias>/workspace/.
provisioner provisioner
.provision_claw(claw_id, &m.model, risk) .provision_claw(claw_id, &m.model, risk, &adhoc_bundles())
.await .await
.map_err(|e| { .map_err(|e| {
eprintln!("teams: provision claw {claw_id} failed: {e}"); eprintln!("teams: provision claw {claw_id} failed: {e}");
+35 -3
View File
@@ -14,6 +14,24 @@
use uuid::Uuid; use uuid::Uuid;
/// The runtime agent alias for a claw id. /// The runtime agent alias for a claw id.
/// The bundles an agent is provisioned with: whatever the template asked for,
/// plus `clawmates_door`, always.
///
/// The door is not optional. It carries the §15 approval gate, so an agent
/// provisioned without it is not a restricted agent, it is an ungated one —
/// and a template that simply forgot to list it would silently get that.
fn with_door(bundles: &[String]) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
out.push("clawmates_door".to_string());
for b in bundles {
let b = b.trim();
if !b.is_empty() && !out.iter().any(|x| x == b) {
out.push(b.to_string());
}
}
out
}
pub fn claw_alias(claw_id: Uuid) -> String { pub fn claw_alias(claw_id: Uuid) -> String {
format!("claw_{}", claw_id.simple()) format!("claw_{}", claw_id.simple())
} }
@@ -232,8 +250,21 @@ impl RuntimeProvisioner {
/// agent gets: `toolfree` = nothing, `research_readonly` = file_read + /// agent gets: `toolfree` = nothing, `research_readonly` = file_read +
/// content_search + glob_search, `coding_readwrite` = adds file_edit + /// content_search + glob_search, `coding_readwrite` = adds file_edit +
/// git_operations + shell, etc.; see the `[risk_profiles.*]` allowlists /// git_operations + shell, etc.; see the `[risk_profiles.*]` allowlists
/// in `deploy/clawmates-runtime/agent.config.example.toml`), and the /// in `deploy/clawmates-runtime/agent.config.example.toml`), and the MCP
/// `clawmates_door` MCP bundle. /// bundles the team template asked for.
///
/// `bundles` used to be the constant `["clawmates_door"]`, which is how
/// every skill in the catalogue became unreachable from a mission. The
/// skills are delivered by ONE channel — the `clawmates_skills` MCP server
/// (`mcp_skills.rs`) — a template that does not receive that bundle cannot
/// list or read a single skill, and 5 of 11 templates ask for it. Two
/// separate doc comments in `cm-runtime` describe the mission path as
/// already having this, which is why nobody looked: the belief was written
/// down twice and checked zero times.
///
/// `clawmates_door` is always included regardless of what is passed. It
/// carries the §15 approval gate, and an agent provisioned without it does
/// not become safer, it becomes ungated.
/// ///
/// NOTE ON WORKSPACE PINNING: `[agents.<alias>.workspace.path]` is an /// NOTE ON WORKSPACE PINNING: `[agents.<alias>.workspace.path]` is an
/// `Option<PathBuf>` field that the ZeroClaw config prop-schema does NOT /// `Option<PathBuf>` field that the ZeroClaw config prop-schema does NOT
@@ -251,6 +282,7 @@ impl RuntimeProvisioner {
claw_id: Uuid, claw_id: Uuid,
model: &str, model: &str,
risk_profile: &str, risk_profile: &str,
bundles: &[String],
) -> Result<String, String> { ) -> Result<String, String> {
let alias = claw_alias(claw_id); let alias = claw_alias(claw_id);
let model_alias = provider_alias_for(model); let model_alias = provider_alias_for(model);
@@ -285,7 +317,7 @@ impl RuntimeProvisioner {
.await?; .await?;
self.set_prop( self.set_prop(
&format!("agents.{alias}.mcp_bundles"), &format!("agents.{alias}.mcp_bundles"),
serde_json::json!(["clawmates_door"]), serde_json::json!(with_door(bundles)),
) )
.await?; .await?;
+88 -1
View File
@@ -42,6 +42,13 @@ use tokio_tungstenite::tungstenite::Message;
const TURN_TIMEOUT: Duration = Duration::from_secs(3600); const TURN_TIMEOUT: Duration = Duration::from_secs(3600);
/// Drives ZeroClaw role-agents (in one container) to execute topology turns. /// Drives ZeroClaw role-agents (in one container) to execute topology turns.
/// Cap on the pinned-skill text injected into one mission turn.
///
/// Skill bodies average ~3.5 KB and pinning is `idx < 2 || foundation`, so a
/// role lands near 7-10 KB. The cap exists for the role that grows a long
/// foundation set, and it is stated in the prompt when it fires.
const MAX_PINNED_SKILL_BYTES: usize = 24_000;
pub struct ZeroClawDriveExecutor { pub struct ZeroClawDriveExecutor {
/// Gateway base URL, e.g. `http://127.0.0.1:42617`. /// Gateway base URL, e.g. `http://127.0.0.1:42617`.
gateway_url: String, gateway_url: String,
@@ -276,6 +283,65 @@ impl ZeroClawDriveExecutor {
Ok(token) Ok(token)
} }
/// The pinned skills for the claw behind `alias`, rendered for the prompt.
///
/// Missions had NO path to a skill. The catalogue's only delivery channel
/// is the `clawmates_skills` MCP server, and a mission agent cannot reach
/// it for three independent reasons: `provision_claw` wrote a constant
/// bundle list, the runtime config defines no such bundle, and mission
/// claws run on `claude_cli`, which is text-only and cannot surface a tool
/// call at all. Two doc comments in `cm-runtime` describe the mission path
/// as already having this contract. It never did — so every skill authored
/// for a mission role was unreachable prose, and no measurement of whether
/// skills fire could have returned anything but zero.
///
/// Bodies, not an index. The chat path lists names and lets the claw call
/// `skills.read`; there is no such tool here, so an index would advertise a
/// capability that does not exist — the exact failure this whole change is
/// about. Pinned only (`pin_in_context`), because everything else would go
/// in unbounded and unread.
pub async fn pinned_skills_text(&self, alias: &str) -> Option<String> {
let tap = self.tap.as_ref()?;
let agent_id = crate::runtime_provision::claw_from_alias(alias)?;
let link = cm_db::repo::agent_template_link::get(&tap.pool, agent_id)
.await
.ok()
.flatten();
let (tpl_id, slot) = link
.as_ref()
.map(|l| (Some(l.template_id), Some(l.role_slot.as_str())))
.unwrap_or((None, None));
let bindings =
cm_db::repo::skills_catalog::effective_for_agent(&tap.pool, agent_id, tpl_id, slot)
.await
.ok()?;
let mut out = String::new();
let mut n = 0usize;
for b in bindings.iter().filter(|b| b.pin_in_context) {
// Bounded, and truncation is STATED. A silently clipped procedure
// is worse than an absent one: the agent follows the half it can
// see and reports success against a rule it never read.
if out.len() + b.skill.body.len() > MAX_PINNED_SKILL_BYTES {
out.push_str(&format!(
"\n[skill \"{}\" omitted — the pinned set exceeded {} bytes]\n",
b.skill.name, MAX_PINNED_SKILL_BYTES
));
continue;
}
out.push_str("\n## ");
out.push_str(&b.skill.name);
out.push('\n');
out.push_str(&b.skill.body);
out.push('\n');
n += 1;
}
if n == 0 {
return None;
}
Some(out)
}
/// Mirror of `ProviderExecutor`'s prompt, flattened to one `content` string /// Mirror of `ProviderExecutor`'s prompt, flattened to one `content` string
/// (the gateway `message` envelope carries a single content field). /// (the gateway `message` envelope carries a single content field).
fn build_prompt(req: &TurnRequest) -> String { fn build_prompt(req: &TurnRequest) -> String {
@@ -674,11 +740,32 @@ impl TurnExecutor for ZeroClawDriveExecutor {
); );
fallback fallback
}); });
let prompt = Self::build_prompt(&req); let prompt = compose_turn_prompt(
&Self::build_prompt(&req),
self.pinned_skills_text(&alias).await.as_deref(),
);
self.drive(&alias, &prompt).await self.drive(&alias, &prompt).await
} }
} }
/// The base turn prompt with the agent's pinned skills appended, if it has any.
///
/// Split out from `run_turn` so the wiring is testable: `pinned_skills_text`
/// working and `run_turn` actually calling it are different claims, and the
/// second is the one that was false for every skill in the catalogue.
pub fn compose_turn_prompt(base: &str, skills: Option<&str>) -> String {
let Some(skills) = skills.map(str::trim).filter(|s| !s.is_empty()) else {
// No heading when there is nothing under it. An empty "Your skills"
// section tells the model it has skills and then shows it none, which
// is worse than silence.
return base.to_string();
};
format!(
"{base}\n\n# Your skills\n\nThese are procedures you are expected to follow for \
this kind of work. Where one applies to what you are about to do, follow it.\n\n{skills}"
)
}
/// Parse `role=alias,role=alias` into a map (blank/malformed entries skipped). /// Parse `role=alias,role=alias` into a map (blank/malformed entries skipped).
fn parse_agent_map(s: &str) -> HashMap<String, String> { fn parse_agent_map(s: &str) -> HashMap<String, String> {
s.split(',') s.split(',')
@@ -0,0 +1,213 @@
//! Do a mission agent's pinned skills actually reach its prompt?
//!
//! Before this test the honest answer was no, for every skill and every role.
//! The catalogue's only delivery channel was the `clawmates_skills` MCP server,
//! and a mission claw could not reach it: `provision_claw` wrote a constant
//! bundle list, the runtime config defines no such bundle, and mission claws
//! run on `claude_cli`, which is text-only and cannot surface a tool call.
//!
//! So the skills were authored, bound, listed in the boot log as bound — and
//! structurally unreadable. That is why this is a test and not a comment: the
//! failure produced no error anywhere, and every layer reported success.
use cm_api::topology_exec::{MissionTap, ZeroClawDriveExecutor};
use cm_domain::{Agent, AgentId, AgentStatus, Role, User, UserId, Workspace, WorkspaceId};
/// `agents.managed_by` is a real FK, so the owner has to exist.
async fn seed_user(pool: &sqlx::PgPool, ws: WorkspaceId) -> UserId {
let user = User {
id: UserId::new(),
workspace_id: ws,
email: format!("owner-{}@example.com", Uuid::now_v7().simple()),
role: Role::Owner,
display_name: "Owner".into(),
created_at: time::OffsetDateTime::UNIX_EPOCH,
};
cm_db::repo::users::insert(pool, &user).await.unwrap();
user.id
}
use std::collections::HashMap;
use uuid::Uuid;
/// A claw with one template-bound, pinned skill. Returns its runtime alias.
async fn seed_claw_with_pinned_skill(pool: &sqlx::PgPool, body: &str) -> (String, WorkspaceId) {
let ws = Workspace {
id: WorkspaceId::new(),
name: "Skill Delivery Test".into(),
plan: "team".into(),
};
cm_db::repo::workspaces::insert(pool, &ws).await.unwrap();
let user = seed_user(pool, ws.id).await;
let agent = Agent {
id: AgentId::new(),
workspace_id: ws.id,
name: "Scout".into(),
job_title: "researcher".into(),
system_prompt: String::new(),
avatar: String::new(),
accent: String::new(),
wallpaper: String::new(),
managed_by: user,
status: AgentStatus::Online,
};
cm_db::repo::agents::insert(pool, &agent, &cm_domain::AccessPolicy::default())
.await
.unwrap();
// A template with one role, and a skill pinned to it.
let template_id = Uuid::now_v7();
sqlx::query(
"INSERT INTO team_templates (id, key, name, description, category, stack,
default_topology, risk_profile, mcp_bundles, version)
VALUES ($1, $2, 'Delivery Test', 'test', 'research', '{}',
'pipeline', 'research_readonly', '{}', 1)",
)
.bind(template_id)
.bind(format!("delivery_test_{}", template_id.simple()))
.execute(pool)
.await
.unwrap();
sqlx::query(
"INSERT INTO template_roles (template_id, slot, order_idx, system_prompt)
VALUES ($1, 'researcher', 0, 'you research')",
)
.bind(template_id)
.execute(pool)
.await
.unwrap();
let skill_id = Uuid::now_v7();
sqlx::query(
"INSERT INTO skills
(id, workspace_id, name, title, author, description, when_to_use,
tags, source_kind, current_version, body)
VALUES ($1, NULL, $2, $2, 'system',
'a procedure the agent must follow', 'always', '{}',
'builtin', 1, $3)",
)
.bind(skill_id)
.bind(format!("delivery-test-skill-{}", skill_id.simple()))
.bind(body)
.execute(pool)
.await
.unwrap();
sqlx::query(
"INSERT INTO template_role_skills (template_id, slot, skill_id, pin_in_context, order_idx)
VALUES ($1, 'researcher', $2, true, 0)",
)
.bind(template_id)
.bind(skill_id)
.execute(pool)
.await
.unwrap();
cm_db::repo::agent_template_link::upsert(
pool,
agent.id.as_uuid(),
template_id,
1,
"researcher",
)
.await
.unwrap();
(
cm_api::runtime_provision::claw_alias(agent.id.as_uuid()),
ws.id,
)
}
fn executor(pool: &sqlx::PgPool, workspace_id: WorkspaceId) -> ZeroClawDriveExecutor {
ZeroClawDriveExecutor::new(
"http://127.0.0.1:1".into(),
"unused".into(),
HashMap::new(),
"default".into(),
)
.with_tap(MissionTap {
pool: pool.clone(),
workspace_id: workspace_id.as_uuid(),
mission_id: Uuid::now_v7(),
phase_id: None,
run_id: None,
})
}
#[tokio::test]
async fn a_pinned_skill_body_reaches_the_turn_prompt() {
let pool = cm_testkit::test_pool().await;
const MARKER: &str = "Never review a paper from its title alone.";
let (alias, ws) = seed_claw_with_pinned_skill(&pool, MARKER).await;
let text = executor(&pool, ws)
.pinned_skills_text(&alias)
.await
.expect("a claw with a pinned template skill must produce skill text");
assert!(
text.contains(MARKER),
"the skill BODY must be present, not just its name — there is no \
`skills.read` tool on the mission path, so an index would name a \
procedure the agent has no way to fetch. Got:\n{text}"
);
}
#[tokio::test]
async fn an_agent_with_no_pinned_skills_adds_nothing() {
let pool = cm_testkit::test_pool().await;
let ws = Workspace {
id: WorkspaceId::new(),
name: "No Skills".into(),
plan: "team".into(),
};
cm_db::repo::workspaces::insert(&pool, &ws).await.unwrap();
let user = seed_user(&pool, ws.id).await;
let agent = Agent {
id: AgentId::new(),
workspace_id: ws.id,
name: "Bare".into(),
job_title: "researcher".into(),
system_prompt: String::new(),
avatar: String::new(),
accent: String::new(),
wallpaper: String::new(),
managed_by: user,
status: AgentStatus::Online,
};
cm_db::repo::agents::insert(&pool, &agent, &cm_domain::AccessPolicy::default())
.await
.unwrap();
let alias = cm_api::runtime_provision::claw_alias(agent.id.as_uuid());
assert!(
executor(&pool, ws.id)
.pinned_skills_text(&alias)
.await
.is_none(),
"an agent with no bound skills must add no section at all — an empty \
`# Your skills` heading tells the model it has skills and then shows \
it none"
);
}
#[test]
fn the_composed_prompt_carries_the_skill_and_omits_the_heading_when_empty() {
let base = "You are the \"researcher\" agent.\n\nTask: read the papers";
let with = cm_api::topology_exec::compose_turn_prompt(base, Some("## arxiv-daily\nDo not re-search."));
assert!(with.contains("Task: read the papers"), "the base turn must survive");
assert!(with.contains("Do not re-search."), "the skill body must be in the prompt");
assert!(with.contains("# Your skills"), "the section needs a heading");
for empty in [None, Some(""), Some(" \n ")] {
let without = cm_api::topology_exec::compose_turn_prompt(base, empty);
assert_eq!(
without, base,
"with no skills the prompt must be byte-identical to the base — an \
empty heading announces skills the agent does not have"
);
}
}
+158
View File
@@ -0,0 +1,158 @@
# ClawMates capability review
*2026-08-19. What exists, what is wired, what was repaired in this pass, and
what is deferred with reasons.*
## Inventory
| | |
|---|---|
| Crates | 19 libraries + 4 binaries (`clawmates-server`, `-broker`, `-node`, `fcagent`) |
| HTTP | 181 routes / 207 method+handler pairs across 39 route modules |
| Workers | 21 background loops, 2s (approval resume) to 24h (tool versions) |
| Database | 79 migrations, ~90 tables |
| Templates | 6 workflow recipes, 11 team templates, **53 authored skills** (was 30) |
| Frontend | 6 tiers (VIZ · MISSIONS · AGENT · PODCAST · REPOS · INFRA), ~57 dashboard components |
| Execution | 5 paths: container/ZeroClaw, microVM solo, composed microVM graph, local herdr, direct headless session |
Dead-code hygiene is genuinely excellent: no `todo!`, no `unimplemented!`, no
stray `TODO` in production paths. **Every defect found in this review was a
wiring defect** — code that was correct, reachable by nothing, and reported as
working.
## The theme
Everything below is one shape, the project's own "silent-success" class: a
capability is built, a second layer does not connect to it, and every layer
reports success. It is not carelessness. In four of the six cases the *belief*
that the wire existed was written down in a doc comment, and no test asked.
That is the actionable lesson: **a comment asserting a connection is not
evidence of one.** Two of the audit's own findings that fed this plan were
themselves wrong in the same way, and the registry whose job is to record which
config keys are read was inaccurate in both directions.
## What was repaired
### 1. Skill bindings — 55 of 85 resolved to nothing
Only 30 skills were authored; 10 of 11 templates referenced names that did not
exist. Three roles bound **zero** skills while their prompts described
procedures to follow.
Invisible because both existing tests assert `authored ⊆ referenced` (30/30,
green) and one explicitly declines to check the other direction.
Fixed: 23 skills authored, renames onto real skills, 22 aspirational references
deleted. Two tests now hold it — one against the files, one against the database
(different questions: resolution goes through catalogue rows, so a file that
exists but fails to ingest still leaves the role empty).
### 2. Skills could not reach a mission agent at all
The larger finding, and the reason #1 was never noticed. The catalogue's only
delivery channel was the `clawmates_skills` MCP server, and a mission claw could
not reach it for **three independent reasons**:
- `provision_claw` wrote the constant `["clawmates_door"]`, ignoring the
template's `mcp_bundles` — which `mission_orchestrator` had already stored.
- The runtime config defines no `clawmates_skills` bundle. (The live local
config defines **no bundles at all**, not even the door.)
- Mission claws run on `claude_cli`, which the runtime's own config comments
document as text-only: it cannot surface a tool call, so no MCP server is
reachable from a mission turn regardless.
And a mission turn's entire system context is two sentences synthesised from the
role slot (`topology_exec::build_prompt`) — the template's role prose is not
used either, which `mission_orchestrator` documents.
So every skill authored for a mission role was unreachable prose, and the
Skill-Use measurement this review planned could only ever have returned zero.
Fixed: `provision_claw` now provisions the template's bundles (door always
added); the pinned skills are injected as **bodies** into the mission prompt —
not an index, because there is no `skills.read` tool on this path and an index
would advertise a capability that does not exist.
### 3. `upsert_task` raised 42P10 on every call
`mission_tasks_external_uniq` is a **partial** unique index. Postgres will not
match a partial index to an `ON CONFLICT` target unless the statement repeats
the predicate. Both callers — the task-card parser that turns INT markers into
tasks, and the security scanner — map the error to a string their caller logs.
Two features were broken for as long as they have existed, and nothing was red.
### 4. The security scan phase never scanned
`security_scan::run` was reachable only from an operator button. So
`security_hardening.toml` — a workflow whose entire first phase is a scan — ran
an agent that was never told to scan, and never fired the scanner either. Now
swept by `phase_runner`, mirroring the benchmark baseline sweep added earlier
for the identical defect, guarded on a completion marker (a clean scan writes no
findings, so a findings-guard would rescan forever).
### 5. Two recipes could not fail
`security_hardening.toml` and `benchmark.toml` carried no `task` and no
`done_when` on any phase. Without `done_when` a phase never enters `evaluating`,
is never judged, and reports `completed` whatever it did. Both now state the
work and the condition.
### 6. Smaller wiring
- `CLAWMATES_JUDGE_MODEL` had two different defaults and a doc comment naming a
third; one source now.
- `GITEA_TOKEN` absence is stated rather than degrading into the same message a
private repo produces.
- `phase_config`'s registry corrected: `harness` and `tools` were listed NOT
IMPLEMENTED while fully read; `bench_name`/`cmd` added; `test_command`
deleted (no reader **and** no writer).
## Verification
- Full workspace suite green.
- Every fix has a test, and **every test was negative-controlled**: the skills
test failed naming all 55; reverting the `ON CONFLICT` predicate reproduces
42P10 exactly; reverting one skill name fails naming that exact role.
- Local stack boots; test-server path (`scripts/test-server.sh up`) is required
for the integration suite, otherwise `cm-testkit` falls back to a
testcontainer that times out.
## Deferred, with reasons
- **`mission_plan` / `mission_roster` have no frontend.** A fully built
propose → review → approve gate whose review step *is* the safety mechanism,
and which no user can reach. The largest declared-but-unwired feature in the
product. A UI is its own piece of work.
- **Skill-Use measurement (Trigger / Compliance / Boundary).** Deferred not for
cost but because it was **unmeasurable until this pass**: with no delivery
channel, trigger rate was structurally zero. It is now worth running, and it
is the natural next step.
- **Provenance layer.** Assessed only, by decision — see
[PROVENANCE-ASSESSMENT.md](PROVENANCE-ASSESSMENT.md).
- **Graph memory / `clawhdf5-agent`.** Declared in the workspace manifest, used
by no crate. The matched study (`MemoryLake on MemoryArena`) shows structured
memory winning modestly on low absolute numbers, and `Harness the Memory`
finds excessive retrieval actively harms agent decisions. Measure against a
baseline before migrating.
- **Thin test coverage**: `cm-secrets` (4), `cm-billing` (4), `cm-safety` (7),
`cm-telemetry` (1); 6 of `cm-brain`'s 9 tests are `#[ignore]`d.
- **`ZEROCLAW_GATEWAY_URL` / `_TOKEN`** have no default and fail at *first use*,
not boot — a deployment looks healthy until someone clicks run.
- **Runtime config bundles.** `provision_claw` now sends the right bundle names,
but the deployed runtime config must define `[mcp_bundles.clawmates_skills]`
(and a server entry pointing at `/mcp/skills`) for the MCP channel to work at
all. The prompt-injection path does not depend on it, which is why it was the
fix chosen here.
## The one process change worth making
Every defect above was found by **checking a claim instead of reading it**. The
existing `runtime_preflight` module is the pattern that works: it asks the
running container what it actually has and says so at boot, because "the code is
right and the machine is not" produced no error anywhere.
The equivalent check for this pass — does a provisioned agent actually receive
the bundles and skills its template names — does not exist yet, and is the
cheapest guard against all of this recurring.
+94
View File
@@ -0,0 +1,94 @@
# Research sweep — papers from the fortnight to 2026-08-19
Scoped to what could change ClawMates: agent runtimes, sub-agent structure,
memory substrates, agent performance measurement, and provenance.
Each entry says what we did about it. "Nothing" is a legitimate outcome and is
recorded as such — a sweep where every paper is actionable is a sweep that
stopped judging.
---
## Skill-Use (2026-08-05) — *acted on*
Benchmarks whether an agent actually **uses** a skill under progressive
disclosure: it sees a name and description and must retrieve the procedure.
Three scores:
- **Trigger** — did it invoke the skill at all
- **Compliance** — did it follow the procedure
- **Boundary** — did it avoid what the skill forbids
That is exactly our skills shape, and reading it is what prompted asking
whether ours fire. The answer turned out to be structural rather than
behavioural: **they could not**. 55 of 85 bindings resolved to nothing, and the
catalogue had no delivery channel to a mission agent at all. Both are now fixed
(see [CAPABILITY-REVIEW.md](CAPABILITY-REVIEW.md)).
The measurement itself is now the obvious next piece of work, and it is worth
doing on the paper's three axes rather than as a pass/fail — the useful output
is *which* of our 53 skills are inert prose.
## MemoryLake on MemoryArena (2026-08-14) — *held as a finding*
A matched comparison: same framework, same model, same tasks, with the memory
backend the only changed component. Structured memory beat vector RAG and
long-context.
The numbers are the point. Success rates were **9/40, 12/20 and 4/20**, and
every system scored zero on some task category. A useful antidote to expecting
a memory swap to be transformative.
**What we did: nothing, deliberately.** It is the strongest argument against
adopting graph memory on enthusiasm, and the strongest argument for capturing a
baseline first.
## Harness the Memory (2026-08-15) — *held as a finding*
No memory substrate dominates. Broad retrieval helps factual QA, while
**excessive retrieval actively harms agent decision-making**.
Directly relevant to a decision we made this pass: mission prompts now carry
pinned skill *bodies*. This paper is the reason that is bounded
(`MAX_PINNED_SKILL_BYTES`) and restricted to pinned skills rather than "give the
agent everything it might need".
## D²ACCI (2026-08-18) — *the sharpest one for us*
> "End-to-end evaluation reveals that an error occurred, but not which stage
> caused it."
That sentence is this project's recurring defect class stated as a research
problem. The paper proposes **DCR**, a graded metric for whether failures stay
*localizable*.
This reframed the provenance assessment: the property to optimise is not
completeness of the record but **localizability of failure**. A system that
records everything and cannot tell you which stage broke has not solved the
problem. See [PROVENANCE-ASSESSMENT.md](PROVENANCE-ASSESSMENT.md).
---
## On clawhdf5 as the provenance layer
The specific question asked. `clawhdf5-agent` (~21k lines: typed entities,
relations including `RelationType::Causal`, BFS and spreading activation;
`provenance.rs` with `MemorySource` / `content_hash` / `session_id`) is
**already in the workspace manifest and used by no crate**. `cm-brain` exposes
`set_provenance` / `provenance` — a wired slot with **zero callers**.
So the adoption cost is lower than it looks. The assessment still recommends
against leading with it, for reasons that are about our system rather than its
quality: the `.brain` is reaped with the agent, which is the wrong lifetime for
an audit record, and it is file-local, so cross-agent questions need a second
mechanism anyway.
The honest sequencing is the one the memory papers argue for: make the record
complete and correctly retained on the path that matters, capture a baseline,
then measure whether the graph substrate earns its place.
## Not found
Nothing in the fortnight materially changes our **runtime** design (microVM
boot, vsock RPC, per-backend egress) or the sub-agent/topology model. Recording
that so the next sweep does not re-search the same ground.
+1 -1
View File
@@ -4,7 +4,7 @@ description = "Rust backend teams — Postgres, DuckDB, graph databases, AP
stack = ["rust", "postgres", "duckdb", "graph", "api", "middleware"] stack = ["rust", "postgres", "duckdb", "graph", "api", "middleware"]
default_topology = "pipeline" default_topology = "pipeline"
risk_profile = "coding_readwrite" risk_profile = "coding_readwrite"
mcp_bundles = ["clawmates_door", "gitea_forge"] mcp_bundles = ["clawmates_door", "clawmates_skills", "gitea_forge"]
version = 1 version = 1
[[roles]] [[roles]]
+10 -5
View File
@@ -5,11 +5,16 @@ stack = ["research", "papers", "digest", "obsidian", "podcast"]
category = "research" category = "research"
default_topology = "pipeline" default_topology = "pipeline"
risk_profile = "research_readonly" risk_profile = "research_readonly"
# `web_fetch` was declared here and is INERT: runtime_provision.rs binds every # This field IS now read. `runtime_provision::provision_claw` used to write the
# mission claw to `mcp_bundles = ["clawmates_door"]` and never reads this field. # constant `["clawmates_door"]` and ignore it — which is how every skill in the
# Agents reach a paper with `curl` through Bash instead, which the prompts say. # catalogue became unreachable from a mission, since the `clawmates_skills` MCP
# Listing a bundle that does not arrive is how a role ends up instructed to use # server is their only delivery channel. It now provisions what is listed here,
# a tool it does not have. # with the door always added.
#
# `web_fetch` was removed rather than kept: there is no such bundle to deliver,
# and now that this list is honoured, naming one that does not exist is worse
# than naming one that was ignored. Agents reach a paper with `curl` through
# Bash, which the prompts say.
mcp_bundles = ["clawmates_door", "clawmates_skills"] mcp_bundles = ["clawmates_door", "clawmates_skills"]
version = 2 version = 2
+1 -1
View File
@@ -4,7 +4,7 @@ description = "Latest React + TailwindCSS + ShadCN — component authoring,
stack = ["typescript", "react", "tailwindcss", "shadcn", "next"] stack = ["typescript", "react", "tailwindcss", "shadcn", "next"]
default_topology = "pipeline" default_topology = "pipeline"
risk_profile = "coding_readwrite" risk_profile = "coding_readwrite"
mcp_bundles = ["clawmates_door", "gitea_forge"] mcp_bundles = ["clawmates_door", "clawmates_skills", "gitea_forge"]
version = 1 version = 1
[[roles]] [[roles]]
+1 -1
View File
@@ -4,7 +4,7 @@ description = "CUDA, Metal, ROCm from Rust — low-level GPU application de
stack = ["rust", "cuda", "metal", "rocm", "gpu"] stack = ["rust", "cuda", "metal", "rocm", "gpu"]
default_topology = "pipeline" default_topology = "pipeline"
risk_profile = "coding_readwrite" risk_profile = "coding_readwrite"
mcp_bundles = ["clawmates_door", "gitea_forge"] mcp_bundles = ["clawmates_door", "clawmates_skills", "gitea_forge"]
version = 1 version = 1
[[roles]] [[roles]]
+1 -1
View File
@@ -4,7 +4,7 @@ description = "Expo + React Native for iOS/Android — camera, comms, netwo
stack = ["typescript", "react-native", "expo", "ios", "android"] stack = ["typescript", "react-native", "expo", "ios", "android"]
default_topology = "pipeline" default_topology = "pipeline"
risk_profile = "coding_readwrite" risk_profile = "coding_readwrite"
mcp_bundles = ["clawmates_door", "gitea_forge"] mcp_bundles = ["clawmates_door", "clawmates_skills", "gitea_forge"]
version = 1 version = 1
[[roles]] [[roles]]
+1 -1
View File
@@ -4,7 +4,7 @@ description = "Full software lifecycle for Rust projects — planning, impl
stack = ["rust", "systems", "distributed", "backend"] stack = ["rust", "systems", "distributed", "backend"]
default_topology = "pipeline" default_topology = "pipeline"
risk_profile = "coding_readwrite" risk_profile = "coding_readwrite"
mcp_bundles = ["clawmates_door", "gitea_forge"] mcp_bundles = ["clawmates_door", "clawmates_skills", "gitea_forge"]
version = 1 version = 1
[[roles]] [[roles]]
+1 -1
View File
@@ -4,7 +4,7 @@ description = "Immersive graphics + game dev in the browser — 3D, isometr
stack = ["typescript", "threejs", "webgl", "webgpu", "gsap"] stack = ["typescript", "threejs", "webgl", "webgpu", "gsap"]
default_topology = "pipeline" default_topology = "pipeline"
risk_profile = "coding_readwrite" risk_profile = "coding_readwrite"
mcp_bundles = ["clawmates_door", "gitea_forge"] mcp_bundles = ["clawmates_door", "clawmates_skills", "gitea_forge"]
version = 1 version = 1
[[roles]] [[roles]]