feat(door): a calibrated governor with three outcomes — allow, deny, HELD for a person
cm_decide:🚪 three Nouls per outbound action (data leaving the organisation, a credential in the content, unsolicited/abusive), the max is the deny probability. Measured on 24 hand-labelled door actions (eval/door-actions.json): AUROC 1.000, [email protected] 0.96, no false denies, no misses, 4 of 24 in the review band — three deny-labelled actions it would not refuse alone (db dump 0.71, delegate-exfil 0.74, threat 0.77) and the one genuinely borderline allow (repo name to a contractor 0.56). 168 ms, ~600 tokens per action, off the z.ai quota. mcp_door: PolicyOutcome::Hold. With TYPESAFE_API_KEY set, above DENY_AT (0.8) refused, below ALLOW_BELOW (0.2) executed, between them the action gets a pending approval (session_key door:<id>) and the agent is told it is queued and not to retry. The approvals route recognises a held door action and executes it on approve — the grant decide mints, the tool consumes — rather than resuming a chat run. The chat-model governor stays as the fallback without a key; it has no middle band. Fail-closed on an unreachable or malformed answer. Thresholds overridable per deployment (CLAWMATES_DOOR_DENY_AT / _ALLOW_BELOW). decide-eval --kind door reports the band outcome, not only a threshold. Harness: a door scenario exercising all three bands directly against /mcp with email_send (its effect is an outbox row), then approving the held one and checking it executes then and not before. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
ddd3972aa3
commit
2656d73def
@@ -56,6 +56,27 @@ async fn decide(
|
||||
workspace_approval(&state, &user, id).await?;
|
||||
let approval = approvals::decide(&state.pool, id, user.user_id, decision).await?;
|
||||
|
||||
// A held DOOR action has no chat run to resume: the tool itself is what
|
||||
// was waiting. Approve executes it now, with the grant decide just
|
||||
// minted; reject leaves the audit trail decide already wrote.
|
||||
if approval
|
||||
.session_key
|
||||
.starts_with(crate::mcp_door::HELD_SESSION_KEY_PREFIX)
|
||||
{
|
||||
let executed = if decision == Decision::Approve {
|
||||
Some(crate::mcp_door::execute_held(&state, &approval).await)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
return Ok(Json(json!({
|
||||
"id": approval.id,
|
||||
"status": approval.status,
|
||||
"door_action": approval.action_type,
|
||||
"executed": executed.as_ref().map(|r| r.is_ok()),
|
||||
"error": executed.and_then(|r| r.err()),
|
||||
})));
|
||||
}
|
||||
|
||||
// Kick the resume before returning. resume_run's awaited portion is only
|
||||
// the setup (claim + checkpoint load + open the broadcast channel); it
|
||||
// spawns the actual multi-step work internally, so this doesn't block the
|
||||
|
||||
Reference in New Issue
Block a user