sec(door): closed by default, governor fails closed; self-authoring off by default
Three fail-open paths on the §15 door: no env at all meant allow-all; a
governor that could not be reached approved with a WARNING; and a reply
that never said DENY — empty, truncated, a refusal — approved, because the
rule was !contains("DENY"). On the two days the judge plan emptied every
outbound action was approved by nobody.
Now: governor_allows() needs an explicit ALLOW and no DENY; both judge()
implementations return false when unreachable; with no governor the door
opens only on CLAWMATES_DOOR_POLICY=allow. Open Agent Passport (arXiv
2603.20953): 74.6% social-engineering success under a permissive policy,
0 of 879 under a restrictive one. Local override gains the governor prod
already runs.
skill_self_authoring: default flipped to OFF. No agent-authored skill has
ever been delivered to a mission or scored; prod held zero proposals.
Enable with CLAWMATES_SKILL_SELF_AUTHORING=1 once promoted skills go
through the files arm and get a Skill-Use score.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
co-authored by
Claude Opus 5
parent
9b7680a605
commit
76ac3714f1
@@ -14,7 +14,8 @@ mod tools;
|
||||
pub use events::{RunEventBody, RunEventEnvelope};
|
||||
pub use outbox::{drain_once, spawn_drainer, EmailSender, LettreSender, SmtpConfig};
|
||||
pub use runtime::{
|
||||
judge_model, ProviderRegistry, Runtime, RuntimeConfig, RuntimeError, StartedRun,
|
||||
governor_allows, judge_model, ProviderRegistry, Runtime, RuntimeConfig, RuntimeError,
|
||||
StartedRun,
|
||||
};
|
||||
pub use sandboxes::{NodeDriverProvider, SandboxManager};
|
||||
pub use terminals::{DriveConfig, TerminalManager};
|
||||
|
||||
@@ -30,6 +30,16 @@ const APPROVAL_TTL: time::Duration = time::Duration::hours(24);
|
||||
/// comparison scorer). Judges use the strongest model — default
|
||||
/// `claude-opus-5` — while everything else runs on the configured default
|
||||
/// model (`claude-sonnet-4-6`). Override with `CLAWMATES_JUDGE_MODEL`.
|
||||
/// Read a governor's verdict. Only an explicit `ALLOW` with no `DENY`
|
||||
/// anywhere approves; an empty reply, a reply that never says either, or one
|
||||
/// that says both is a denial. The previous rule was `!contains("DENY")`,
|
||||
/// under which a model that answered nothing at all — a truncated stream, a
|
||||
/// refusal, a reply in the wrong shape — approved the action.
|
||||
pub fn governor_allows(reply: &str) -> bool {
|
||||
let upper = reply.to_ascii_uppercase();
|
||||
upper.contains("ALLOW") && !upper.contains("DENY")
|
||||
}
|
||||
|
||||
pub fn judge_model() -> String {
|
||||
std::env::var("CLAWMATES_JUDGE_MODEL").unwrap_or_else(|_| "claude-opus-5".to_string())
|
||||
}
|
||||
@@ -298,13 +308,18 @@ impl Runtime {
|
||||
}
|
||||
|
||||
/// Governor agent: ask the **judge model** to judge an action. Returns
|
||||
/// `(allow, reason)`. The verdict is the first token (`ALLOW`/`DENY`) of the
|
||||
/// model's reply. Best-effort and **fail-open** — if the model is
|
||||
/// unreachable it returns `(true, …)` so a governor outage doesn't halt
|
||||
/// autonomous agents (the governor is an extra soft check atop deterministic
|
||||
/// policy, not the only gate). Judges use the strongest model
|
||||
/// ([`judge_model`], default `claude-opus-4-8`); everything else runs on the
|
||||
/// configured default model.
|
||||
/// `(allow, reason)`, parsed by [`governor_allows`]: only an explicit
|
||||
/// `ALLOW` with no `DENY` approves.
|
||||
///
|
||||
/// **Fail-closed** since 2026-09-20. This returned `(true, …)` when the
|
||||
/// model was unreachable so that a governor outage would not halt
|
||||
/// autonomous agents — which meant that on the day the judge plan emptied
|
||||
/// (2026-08-29, 2026-09-09) every outbound action was approved unjudged,
|
||||
/// with nothing but a log line saying so. Open Agent Passport (arXiv
|
||||
/// 2603.20953) measured the difference a restrictive default makes:
|
||||
/// social-engineered actions succeeded 74.6% of the time under a
|
||||
/// permissive policy and 0 of 879 under a restrictive one. A door that
|
||||
/// cannot reach its governor now says so and denies.
|
||||
pub async fn judge(&self, system: &str, user: &str) -> (bool, String) {
|
||||
let (provider, model) = self.resolve_provider(&judge_model());
|
||||
let request = ChatRequest {
|
||||
@@ -327,10 +342,10 @@ impl Runtime {
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => return (true, format!("governor unreachable (fail-open): {e}")),
|
||||
Err(e) => return (false, format!("governor unreachable (fail-closed): {e}")),
|
||||
}
|
||||
let allow = !text.to_uppercase().contains("DENY");
|
||||
(allow, text.trim().to_string())
|
||||
let text = text.trim().to_string();
|
||||
(governor_allows(&text), text)
|
||||
}
|
||||
|
||||
/// One-shot completion: send `system`+`user` to `model`, collect the full
|
||||
@@ -1077,3 +1092,21 @@ fn chat_messages(history: &[MessageWithSteps], user_text: &str) -> Vec<ChatMessa
|
||||
});
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod governor_verdict_tests {
|
||||
use super::governor_allows;
|
||||
|
||||
/// Only an explicit ALLOW approves. The old rule, `!contains("DENY")`,
|
||||
/// approved every reply in the first three rows.
|
||||
#[test]
|
||||
fn silence_and_off_contract_replies_deny() {
|
||||
assert!(!governor_allows(""));
|
||||
assert!(!governor_allows("Sure, that looks reasonable to me."));
|
||||
assert!(!governor_allows("I cannot evaluate this request."));
|
||||
assert!(!governor_allows("DENY\nrecipient is external"));
|
||||
assert!(!governor_allows("ALLOW? No — DENY, the payload holds a token"));
|
||||
assert!(governor_allows("ALLOW\nroutine status update to a known channel"));
|
||||
assert!(governor_allows("allow"));
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user