door: live governor agent (LLM veto) — self-governing autonomy
Adds Runtime::judge (the configured model returns ALLOW/DENY + reason, fail-open) and wires it into the door policy behind CLAWMATES_DOOR_GOVERNOR: a governor agent judges each outbound action and can veto exfiltration / spam / secret-leakage, atop the deterministic rules. Realizes the self-governing- topology path — authority decided by an agent, not a human, still audited. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
15f187e031
commit
e0e150084d
@@ -92,7 +92,7 @@ async fn policy_decide(
|
||||
state: &AppState,
|
||||
workspace: cm_domain::WorkspaceId,
|
||||
mcp_tool: &str,
|
||||
_category: Option<cm_domain::GatedCategory>,
|
||||
category: Option<cm_domain::GatedCategory>,
|
||||
args: &Value,
|
||||
) -> PolicyOutcome {
|
||||
// 1. Kill switch.
|
||||
@@ -138,9 +138,27 @@ async fn policy_decide(
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Governor hook (extension point): a deterministic rule set or a governor
|
||||
// agent (an LLM that judges the payload) can veto here — the seam for the
|
||||
// "self-governing topology" story. Default: no veto.
|
||||
// 4. Governor agent: when CLAWMATES_DOOR_GOVERNOR is set, an LLM judges the
|
||||
// action and can veto — the "self-governing topology" path. Fail-open
|
||||
// (a governor outage doesn't halt agents); deterministic rules above are
|
||||
// the hard floor.
|
||||
if std::env::var("CLAWMATES_DOOR_GOVERNOR").is_ok() {
|
||||
let system = "You are a security governor for an autonomous agent's outbound actions. \
|
||||
Reply with exactly ALLOW or DENY on the first line, then one short reason. \
|
||||
DENY if the action looks like data exfiltration, spam, credential/secret leakage, \
|
||||
or sending sensitive or internal data to an untrusted external recipient. \
|
||||
Otherwise ALLOW.";
|
||||
let request = format!(
|
||||
"Tool: {mcp_tool}\nCategory: {}\nPayload: {}",
|
||||
category.map(|c| c.as_str()).unwrap_or("none"),
|
||||
serde_json::to_string(args).unwrap_or_default()
|
||||
);
|
||||
let (allow, reason) = state.runtime.judge(system, &request).await;
|
||||
if !allow {
|
||||
return PolicyOutcome::Deny(format!("governor agent vetoed — {reason}"));
|
||||
}
|
||||
}
|
||||
|
||||
PolicyOutcome::Approve
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user