door: live governor agent (LLM veto) — self-governing autonomy

Adds Runtime::judge (the configured model returns ALLOW/DENY + reason,
fail-open) and wires it into the door policy behind CLAWMATES_DOOR_GOVERNOR: a
governor agent judges each outbound action and can veto exfiltration / spam /
secret-leakage, atop the deterministic rules. Realizes the self-governing-
topology path — authority decided by an agent, not a human, still audited.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Omar Sobh
2026-06-17 03:27:09 -07:00
co-authored by Claude Opus 4.8
parent 15f187e031
commit e0e150084d
2 changed files with 54 additions and 4 deletions
+22 -4
View File
@@ -92,7 +92,7 @@ async fn policy_decide(
state: &AppState,
workspace: cm_domain::WorkspaceId,
mcp_tool: &str,
_category: Option<cm_domain::GatedCategory>,
category: Option<cm_domain::GatedCategory>,
args: &Value,
) -> PolicyOutcome {
// 1. Kill switch.
@@ -138,9 +138,27 @@ async fn policy_decide(
}
}
// 4. Governor hook (extension point): a deterministic rule set or a governor
// agent (an LLM that judges the payload) can veto here — the seam for the
// "self-governing topology" story. Default: no veto.
// 4. Governor agent: when CLAWMATES_DOOR_GOVERNOR is set, an LLM judges the
// action and can veto — the "self-governing topology" path. Fail-open
// (a governor outage doesn't halt agents); deterministic rules above are
// the hard floor.
if std::env::var("CLAWMATES_DOOR_GOVERNOR").is_ok() {
let system = "You are a security governor for an autonomous agent's outbound actions. \
Reply with exactly ALLOW or DENY on the first line, then one short reason. \
DENY if the action looks like data exfiltration, spam, credential/secret leakage, \
or sending sensitive or internal data to an untrusted external recipient. \
Otherwise ALLOW.";
let request = format!(
"Tool: {mcp_tool}\nCategory: {}\nPayload: {}",
category.map(|c| c.as_str()).unwrap_or("none"),
serde_json::to_string(args).unwrap_or_default()
);
let (allow, reason) = state.runtime.judge(system, &request).await;
if !allow {
return PolicyOutcome::Deny(format!("governor agent vetoed — {reason}"));
}
}
PolicyOutcome::Approve
}