feat(door): a calibrated governor with three outcomes — allow, deny, HELD for a person
cm_decide:🚪 three Nouls per outbound action (data leaving the organisation, a credential in the content, unsolicited/abusive), the max is the deny probability. Measured on 24 hand-labelled door actions (eval/door-actions.json): AUROC 1.000, [email protected] 0.96, no false denies, no misses, 4 of 24 in the review band — three deny-labelled actions it would not refuse alone (db dump 0.71, delegate-exfil 0.74, threat 0.77) and the one genuinely borderline allow (repo name to a contractor 0.56). 168 ms, ~600 tokens per action, off the z.ai quota. mcp_door: PolicyOutcome::Hold. With TYPESAFE_API_KEY set, above DENY_AT (0.8) refused, below ALLOW_BELOW (0.2) executed, between them the action gets a pending approval (session_key door:<id>) and the agent is told it is queued and not to retry. The approvals route recognises a held door action and executes it on approve — the grant decide mints, the tool consumes — rather than resuming a chat run. The chat-model governor stays as the fallback without a key; it has no middle band. Fail-closed on an unreachable or malformed answer. Thresholds overridable per deployment (CLAWMATES_DOOR_DENY_AT / _ALLOW_BELOW). decide-eval --kind door reports the band outcome, not only a threshold. Harness: a door scenario exercising all three bands directly against /mcp with email_send (its effect is an outbox row), then approving the held one and checking it executes then and not before. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
ddd3972aa3
commit
2656d73def
+216
-46
@@ -8,9 +8,18 @@
|
||||
//! is journaled to the append-only audit log, and a central policy decides each
|
||||
//! call — but the **human approver is replaced by an automated policy/governor**
|
||||
//! ("agents control their destiny"). Recipient allowlists, spend caps, taint
|
||||
//! blocks, or a governor agent plug into [`policy_decide`]. Since 2026-09-20
|
||||
//! the door is **closed by default**: a call is approved only by a governor
|
||||
//! that answered ALLOW, or by an explicit `CLAWMATES_DOOR_POLICY=allow`.
|
||||
//! blocks, or a governor plug into [`policy_decide`]. Since 2026-09-20 the
|
||||
//! door is **closed by default**: a call is approved only by a governor that
|
||||
//! answered ALLOW, or by an explicit `CLAWMATES_DOOR_POLICY=allow`.
|
||||
//!
|
||||
//! Since 2026-09-21 the governor has THREE outcomes, not two. With
|
||||
//! `TYPESAFE_API_KEY` set the decision is a calibrated one
|
||||
//! (`cm_decide::door`: three Nouls, the max is the deny probability):
|
||||
//! above `DENY_AT` refused, below `ALLOW_BELOW` executed, and in between
|
||||
//! **held** — a pending approval a person decides, executed on approve.
|
||||
//! Measured on 24 labelled actions: AUROC 1.0, no false denies, no misses,
|
||||
//! 4 held. The chat-model governor (`CLAWMATES_DOOR_GOVERNOR`) remains the
|
||||
//! fallback when no key is set; it has no middle band.
|
||||
//!
|
||||
//! v1 exposes `email_send` (runtime-executed → `outbox`, observable, no external
|
||||
//! creds). Broker-backed tools (e.g. `slack_post`) are the next increment — they
|
||||
@@ -78,6 +87,9 @@ fn tool_result(id: Option<Value>, is_error: bool, text: String) -> Json<Value> {
|
||||
enum PolicyOutcome {
|
||||
Approve,
|
||||
Deny(String),
|
||||
/// Not refused, not executed: a person decides. Carries the reason a
|
||||
/// reviewer reads.
|
||||
Hold(String),
|
||||
}
|
||||
|
||||
/// Decides each door call in place of a human. The human is removed; autonomy
|
||||
@@ -144,7 +156,14 @@ async fn policy_decide(
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Governor agent: when CLAWMATES_DOOR_GOVERNOR is set, an LLM judges the
|
||||
// 4. Calibrated governor: three questions, one call, a probability with
|
||||
// a middle band. Preferred over the chat governor whenever a key is
|
||||
// set. Fail-CLOSED: an unreachable or malformed answer denies.
|
||||
if let Some(jev) = cm_decide::jev::Jev::from_env() {
|
||||
return calibrated_decision(&jev, mcp_tool, args).await;
|
||||
}
|
||||
|
||||
// 4b. Governor agent: when CLAWMATES_DOOR_GOVERNOR is set, an LLM judges the
|
||||
// action — the "self-governing topology" path. Fail-CLOSED: a governor
|
||||
// that cannot be reached, or that does not say ALLOW, denies. The
|
||||
// deterministic rules above are the hard floor; this is the only
|
||||
@@ -193,6 +212,65 @@ async fn policy_decide(
|
||||
ungoverned_default(std::env::var("CLAWMATES_DOOR_POLICY").ok().as_deref())
|
||||
}
|
||||
|
||||
/// How long the calibrated governor may take. It measures ~170 ms; a door
|
||||
/// that waits ten seconds on it is a door whose provider is down.
|
||||
const DECISION_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
|
||||
|
||||
/// Ask the three door questions and read the band. `DENY_AT` and
|
||||
/// `ALLOW_BELOW` are `cm_decide::door`'s, overridable per deployment by
|
||||
/// `CLAWMATES_DOOR_DENY_AT` / `CLAWMATES_DOOR_ALLOW_BELOW`.
|
||||
async fn calibrated_decision(
|
||||
jev: &cm_decide::jev::Jev,
|
||||
mcp_tool: &str,
|
||||
args: &Value,
|
||||
) -> PolicyOutcome {
|
||||
use cm_decide::Decider as _;
|
||||
let deny_at = env_f64("CLAWMATES_DOOR_DENY_AT", cm_decide::door::DENY_AT);
|
||||
let allow_below = env_f64("CLAWMATES_DOOR_ALLOW_BELOW", cm_decide::door::ALLOW_BELOW);
|
||||
let state = cm_decide::door::state(mcp_tool, args);
|
||||
let decision = match tokio::time::timeout(
|
||||
DECISION_TIMEOUT,
|
||||
jev.decide(&state, &cm_decide::door::questions()),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(Ok(d)) => d,
|
||||
Ok(Err(e)) => {
|
||||
eprintln!("mcp_door: the calibrated governor failed, DENYING {mcp_tool}: {e}");
|
||||
return PolicyOutcome::Deny(format!("the door's governor could not decide ({e})"));
|
||||
}
|
||||
Err(_) => {
|
||||
eprintln!("mcp_door: the calibrated governor timed out, DENYING {mcp_tool}");
|
||||
return PolicyOutcome::Deny("the door's governor did not answer in time".into());
|
||||
}
|
||||
};
|
||||
let Some(risk) = cm_decide::door::Risk::from_answers(&decision.answers) else {
|
||||
return PolicyOutcome::Deny("the door's governor answered in an unexpected shape".into());
|
||||
};
|
||||
let why = format!(
|
||||
"{} judged this {:.0}% likely to be {} (exfil {:.2}, secret {:.2}, spam {:.2})",
|
||||
decision.model,
|
||||
risk.deny * 100.0,
|
||||
risk.dominant(),
|
||||
risk.exfil,
|
||||
risk.secret,
|
||||
risk.spam
|
||||
);
|
||||
match cm_decide::patterns::gate_noul(risk.deny, allow_below, deny_at) {
|
||||
cm_decide::patterns::Gate::Act => PolicyOutcome::Deny(why),
|
||||
cm_decide::patterns::Gate::Dismiss => PolicyOutcome::Approve,
|
||||
cm_decide::patterns::Gate::Review => PolicyOutcome::Hold(why),
|
||||
}
|
||||
}
|
||||
|
||||
fn env_f64(key: &str, default: f64) -> f64 {
|
||||
std::env::var(key)
|
||||
.ok()
|
||||
.and_then(|v| v.trim().parse::<f64>().ok())
|
||||
.filter(|v| (0.0..=1.0).contains(v))
|
||||
.unwrap_or(default)
|
||||
}
|
||||
|
||||
/// The posture with no governor configured. Only the literal `allow` opens
|
||||
/// the door; unset, empty, or anything else keeps it shut and says how to
|
||||
/// open it. `deny` is handled earlier as the kill switch and lands here too.
|
||||
@@ -219,31 +297,7 @@ async fn mint_grant(
|
||||
category: Option<cm_domain::GatedCategory>,
|
||||
args: &Value,
|
||||
) -> Result<uuid::Uuid, String> {
|
||||
// approvals.run_id / requested_by_agent are strict FKs → agent → session → run.
|
||||
let session =
|
||||
cm_db::repo::sessions::create(&state.pool, agent_id, user.workspace_id, "mcp-door")
|
||||
.await
|
||||
.map_err(|e| e.to_string())?;
|
||||
let run_id = cm_db::repo::runs::create(&state.pool, session.id)
|
||||
.await
|
||||
.map_err(|e| e.to_string())?;
|
||||
let approval = cm_safety::approvals::create(
|
||||
&state.pool,
|
||||
cm_safety::NewApproval {
|
||||
workspace_id: user.workspace_id,
|
||||
run_id,
|
||||
session_key: session.id.as_uuid().to_string(),
|
||||
action_type: internal.to_string(),
|
||||
category: category.unwrap_or(cm_domain::GatedCategory::OutboundMessage),
|
||||
payload: args.clone(),
|
||||
preview: state.runtime.tool_preview(internal, args),
|
||||
requested_by_agent: agent_id,
|
||||
taint_sources: Vec::new(),
|
||||
expires_at: Some(time::OffsetDateTime::now_utc() + time::Duration::hours(1)),
|
||||
},
|
||||
)
|
||||
.await
|
||||
.map_err(|e| e.to_string())?;
|
||||
let approval = door_approval(state, user, agent_id, internal, category, args, "mcp-door").await?;
|
||||
// Auto-decide (policy already approved above): mints the single-use grant
|
||||
// and writes the decision to the audit log.
|
||||
cm_safety::approvals::decide(
|
||||
@@ -257,6 +311,87 @@ async fn mint_grant(
|
||||
Ok(approval.id)
|
||||
}
|
||||
|
||||
/// Marks a held door action's approval so the approve route knows to
|
||||
/// execute the tool rather than resume a chat run. It is the `session_key`
|
||||
/// prefix; a chat approval's key is a `SessionKey`, which never starts so.
|
||||
pub const HELD_SESSION_KEY_PREFIX: &str = "door:";
|
||||
|
||||
/// The approval row a door action gets. Pending; the caller decides it —
|
||||
/// immediately for an approved action ([`mint_grant`]), or a person later
|
||||
/// for a held one. `title` names the session so the row is recognisable.
|
||||
async fn door_approval(
|
||||
state: &AppState,
|
||||
user: &cm_auth::AuthedUser,
|
||||
agent_id: cm_domain::AgentId,
|
||||
internal: &str,
|
||||
category: Option<cm_domain::GatedCategory>,
|
||||
args: &Value,
|
||||
title: &str,
|
||||
) -> Result<cm_safety::Approval, String> {
|
||||
// approvals.run_id / requested_by_agent are strict FKs → agent → session → run.
|
||||
let session = cm_db::repo::sessions::create(&state.pool, agent_id, user.workspace_id, title)
|
||||
.await
|
||||
.map_err(|e| e.to_string())?;
|
||||
let run_id = cm_db::repo::runs::create(&state.pool, session.id)
|
||||
.await
|
||||
.map_err(|e| e.to_string())?;
|
||||
let session_key = if title == "mcp-door-held" {
|
||||
format!("{HELD_SESSION_KEY_PREFIX}{}", session.id.as_uuid())
|
||||
} else {
|
||||
session.id.as_uuid().to_string()
|
||||
};
|
||||
cm_safety::approvals::create(
|
||||
&state.pool,
|
||||
cm_safety::NewApproval {
|
||||
workspace_id: user.workspace_id,
|
||||
run_id,
|
||||
session_key,
|
||||
action_type: internal.to_string(),
|
||||
category: category.unwrap_or(cm_domain::GatedCategory::OutboundMessage),
|
||||
payload: args.clone(),
|
||||
preview: state.runtime.tool_preview(internal, args),
|
||||
requested_by_agent: agent_id,
|
||||
taint_sources: Vec::new(),
|
||||
expires_at: Some(time::OffsetDateTime::now_utc() + time::Duration::hours(24)),
|
||||
},
|
||||
)
|
||||
.await
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Execute a held door action that a person has just approved. Called by
|
||||
/// the approvals route; the grant was minted by the decide it just made.
|
||||
pub async fn execute_held(state: &AppState, approval: &cm_safety::Approval) -> Result<Value, String> {
|
||||
let out = state
|
||||
.runtime
|
||||
.execute_door_tool(
|
||||
approval.workspace_id,
|
||||
approval.requested_by_agent,
|
||||
&approval.action_type,
|
||||
approval.payload.clone(),
|
||||
Some(approval.id),
|
||||
)
|
||||
.await;
|
||||
let (event, detail) = match &out {
|
||||
Ok(output) => (
|
||||
"door.executed",
|
||||
json!({ "auto_decided": false, "approval_id": approval.id, "args": approval.payload, "output": output }),
|
||||
),
|
||||
Err(e) => ("door.error", json!({ "approval_id": approval.id, "error": e, "args": approval.payload })),
|
||||
};
|
||||
let _ = cm_db::repo::audit::append(
|
||||
&state.pool,
|
||||
approval.workspace_id,
|
||||
cm_db::repo::audit::Actor::Agent(approval.requested_by_agent),
|
||||
event,
|
||||
"tool",
|
||||
&approval.action_type,
|
||||
detail,
|
||||
)
|
||||
.await;
|
||||
out
|
||||
}
|
||||
|
||||
/// Authenticate the bearer header → workspace/user. `None` if missing/invalid.
|
||||
///
|
||||
/// Accepts [`cm_auth::SCOPE_AGENT_DOOR`] as well as a person's session. This
|
||||
@@ -531,23 +666,6 @@ pub async fn mcp(
|
||||
|
||||
let category = state.runtime.tool_gate_category(internal);
|
||||
|
||||
// The gate — human replaced by automated policy.
|
||||
if let PolicyOutcome::Deny(reason) =
|
||||
policy_decide(&state, user.workspace_id, mcp_name, category, &args).await
|
||||
{
|
||||
let _ = cm_db::repo::audit::append(
|
||||
&state.pool,
|
||||
user.workspace_id,
|
||||
cm_db::repo::audit::Actor::System,
|
||||
"door.denied",
|
||||
"tool",
|
||||
mcp_name,
|
||||
json!({ "category": category.map(|c| c.as_str()), "reason": reason, "args": args }),
|
||||
)
|
||||
.await;
|
||||
return tool_result(req.id, true, format!("denied by policy: {reason}"));
|
||||
}
|
||||
|
||||
// Attribute the action to the specific calling claw (X-ZeroClaw-Agent
|
||||
// header), or the workspace's first agent as a legacy fallback.
|
||||
let agent_id = match caller_agent(&state, &user, &headers).await {
|
||||
@@ -555,6 +673,58 @@ pub async fn mcp(
|
||||
Err(msg) => return tool_result(req.id, true, msg),
|
||||
};
|
||||
|
||||
// The gate — human replaced by automated policy, with a way back
|
||||
// to the human for the actions the policy will not decide alone.
|
||||
match policy_decide(&state, user.workspace_id, mcp_name, category, &args).await {
|
||||
PolicyOutcome::Approve => {}
|
||||
PolicyOutcome::Deny(reason) => {
|
||||
let _ = cm_db::repo::audit::append(
|
||||
&state.pool,
|
||||
user.workspace_id,
|
||||
cm_db::repo::audit::Actor::System,
|
||||
"door.denied",
|
||||
"tool",
|
||||
mcp_name,
|
||||
json!({ "category": category.map(|c| c.as_str()), "reason": reason, "args": args }),
|
||||
)
|
||||
.await;
|
||||
return tool_result(req.id, true, format!("denied by policy: {reason}"));
|
||||
}
|
||||
PolicyOutcome::Hold(reason) => {
|
||||
let internal_name = internal;
|
||||
let approval = match door_approval(
|
||||
&state, &user, agent_id, internal_name, category, &args, "mcp-door-held",
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(a) => a,
|
||||
Err(e) => {
|
||||
return tool_result(req.id, true, format!("held, but could not queue it for review: {e}"))
|
||||
}
|
||||
};
|
||||
let _ = cm_db::repo::audit::append(
|
||||
&state.pool,
|
||||
user.workspace_id,
|
||||
cm_db::repo::audit::Actor::System,
|
||||
"door.held",
|
||||
"tool",
|
||||
mcp_name,
|
||||
json!({ "category": category.map(|c| c.as_str()), "reason": reason, "approval_id": approval.id, "args": args }),
|
||||
)
|
||||
.await;
|
||||
return tool_result(
|
||||
req.id,
|
||||
true,
|
||||
format!(
|
||||
"held for human review — NOT executed. {reason}. It is in the approvals \
|
||||
queue as {}; if a person approves it, it will be executed then. Do not \
|
||||
retry it with different wording.",
|
||||
approval.id
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Gated delegation bridge: `delegate` causes a sibling claw to run a
|
||||
// full turn and returns its result, gated + audited here rather than
|
||||
// via ZeroClaw's in-memory DelegateTool (which would bypass the door).
|
||||
|
||||
@@ -56,6 +56,27 @@ async fn decide(
|
||||
workspace_approval(&state, &user, id).await?;
|
||||
let approval = approvals::decide(&state.pool, id, user.user_id, decision).await?;
|
||||
|
||||
// A held DOOR action has no chat run to resume: the tool itself is what
|
||||
// was waiting. Approve executes it now, with the grant decide just
|
||||
// minted; reject leaves the audit trail decide already wrote.
|
||||
if approval
|
||||
.session_key
|
||||
.starts_with(crate::mcp_door::HELD_SESSION_KEY_PREFIX)
|
||||
{
|
||||
let executed = if decision == Decision::Approve {
|
||||
Some(crate::mcp_door::execute_held(&state, &approval).await)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
return Ok(Json(json!({
|
||||
"id": approval.id,
|
||||
"status": approval.status,
|
||||
"door_action": approval.action_type,
|
||||
"executed": executed.as_ref().map(|r| r.is_ok()),
|
||||
"error": executed.and_then(|r| r.err()),
|
||||
})));
|
||||
}
|
||||
|
||||
// Kick the resume before returning. resume_run's awaited portion is only
|
||||
// the setup (claim + checkpoint load + open the broadcast channel); it
|
||||
// spawns the actual multi-step work internally, so this doesn't block the
|
||||
|
||||
Reference in New Issue
Block a user