fix(fleet): a VM reaches its OWN provider and no other, measured not assumed

The GLM backend works — and proving it produced a better boundary than the one
I shipped an hour ago.

WHAT THE FIRST GLM MISSION SHOWED. It completed, and the delivered file said the
model was "claude-opus-5". The node's egress log said the VM had dialled
`api.anthropic.com` five times before `api.z.ai`. Either reading alone is
consistent with a "GLM backend" that silently runs Anthropic — the exact
silent-success shape this project keeps closing — so I did not accept either.

THE ABLATION, run on tank rather than reasoned about: deny `anthropic.com` at the
proxy and run the same mission again. It **completed**, dialling only
`api.z.ai`. So the completions genuinely come from z.ai; Claude Code's calls to
anthropic.com are its own telemetry, not its model traffic.

And that same agent — served exclusively by z.ai, with Anthropic unreachable —
still described itself as "Claude Opus 5 (1M context)". **A model's account of
which model it is has no evidential value.** The proxy's log of which host it
dialled does. This is the `uname -r` lesson again in a new place: ask the
infrastructure, not the agent.

So the allow-list is now PER BACKEND rather than a union: a `claude` VM reaches
Anthropic and the forge, a `glm` VM reaches z.ai and the forge, and neither can
reach the other's endpoint. A union was defensible when it was one host; once the
measurement showed a GLM VM never needs Anthropic, keeping it would mean a
credential mix-up upstream could still put one provider's secret on another
provider's wire. Now it fails at a closed door instead.

An unknown backend gets the forge and NO model API — it cannot run anyway, and
borrowing somebody else's door is the failure this split prevents. An explicit
`CLAWMATES_FC_EGRESS_ALLOW` still wins outright: an operator who set it drew a
boundary on purpose.

`DEFAULT_ALLOW` is deleted rather than left beside the new function, so there is
one answer to "what may a mission reach" and not two.

534 tests pass, clippy clean.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-06 17:26:32 -07:00
co-authored by Claude Opus 5
parent f7f3dfe495
commit d3a53e7bf1
2 changed files with 78 additions and 24 deletions
+77 -23
View File
@@ -43,39 +43,58 @@ use tokio::net::{TcpStream, UnixListener, UnixStream};
/// Port the guest dials. Must match `fcagent`'s `EGRESS_PORT`. /// Port the guest dials. Must match `fcagent`'s `EGRESS_PORT`.
pub const EGRESS_PORT: u32 = 9002; pub const EGRESS_PORT: u32 = 9002;
/// Hosts a mission may reach, unless `CLAWMATES_FC_EGRESS_ALLOW` overrides it.
/// What every backend gets, whatever it is.
const COMMON_ALLOW: &[&str] = &["git.redclaw.dev"];
/// The model host a backend's CLI must reach, and NOTHING else.
/// ///
/// Narrow on purpose: this is the whole egress surface of a mission, and the /// Per backend rather than a union, and that is not tidiness. MEASURED on tank:
/// point of the exercise is that it is smaller than "the internet". An entry /// a `glm` VM completed a whole mission with `api.anthropic.com` denied at this
/// beginning with `.` matches subdomains. /// proxy, dialling only `api.z.ai` — Claude Code's calls to anthropic.com are
const DEFAULT_ALLOW: &[&str] = &[ /// its own telemetry, not its completions. So a GLM VM has no need of Anthropic
// The model API. Without it there is no agent. /// at all, and a union allow-list would let a credential mix-up reach the wrong
"api.anthropic.com", /// provider's endpoint instead of failing at a closed door.
".anthropic.com", ///
// z.ai, for a `glm` backend VM. Claude Code speaks to it through /// The measurement also settled something a self-report could not: that same
// ANTHROPIC_BASE_URL, so the binary is the same and only the host differs — /// agent, served only by z.ai, still described itself as "Claude Opus 5". A
// and a host the proxy denies is an agent that cannot reach any model at /// model's account of which model it is has no evidential value here; the
// all. Adding it here rather than requiring an operator to set /// proxy's log of which host it dialled does.
// CLAWMATES_FC_EGRESS_ALLOW: the image exists to be used, and a default that fn provider_hosts(backend: Option<&str>) -> &'static [&'static str] {
// cannot run the images we ship is a trap, not a policy. match backend {
"api.z.ai", None | Some("") | Some("default") | Some("claude") => {
// Our forge: clone, push, PRs. &["api.anthropic.com", ".anthropic.com"]
"git.redclaw.dev", }
]; Some("glm") => &["api.z.ai"],
// Fail closed: a backend nobody taught this function about reaches the
// forge and no model API. It cannot silently borrow another provider's
// door, which is the failure this split exists to prevent.
Some(_) => &[],
}
}
/// Parse the allow-list once per VM. /// Parse the allow-list once per VM.
/// ///
/// An empty `CLAWMATES_FC_EGRESS_ALLOW` means **deny everything**, not "fall back /// An empty `CLAWMATES_FC_EGRESS_ALLOW` means **deny everything**, not "fall back
/// to the default": an operator who blanked it asked for no egress, and quietly /// to the default": an operator who blanked it asked for no egress, and quietly
/// restoring the default would hand a mission the network they just took away. /// restoring the default would hand a mission the network they just took away.
fn allow_list() -> Vec<String> { /// The allow-list for a VM running `backend`.
///
/// An explicit `CLAWMATES_FC_EGRESS_ALLOW` still wins outright: an operator who
/// set it asked for exactly that list, and quietly adding a provider host to it
/// would widen a boundary they had drawn on purpose.
fn allow_list_for(backend: Option<&str>) -> Vec<String> {
match std::env::var("CLAWMATES_FC_EGRESS_ALLOW") { match std::env::var("CLAWMATES_FC_EGRESS_ALLOW") {
Ok(raw) => raw Ok(raw) => raw
.split(',') .split(',')
.map(|s| s.trim().to_ascii_lowercase()) .map(|s| s.trim().to_ascii_lowercase())
.filter(|s| !s.is_empty()) .filter(|s| !s.is_empty())
.collect(), .collect(),
Err(_) => DEFAULT_ALLOW.iter().map(|s| s.to_string()).collect(), Err(_) => COMMON_ALLOW
.iter()
.chain(provider_hosts(backend).iter())
.map(|s| s.to_string())
.collect(),
} }
} }
@@ -272,7 +291,11 @@ async fn reply(s: &mut UnixStream, code: u16, text: &str) -> std::io::Result<()>
/// ///
/// Bound **before** firecracker starts, because a guest that dials before the /// Bound **before** firecracker starts, because a guest that dials before the
/// host is listening gets a connection refused it will not retry. /// host is listening gets a connection refused it will not retry.
pub fn start(uds: &Path, vm_id: &str) -> Result<(PathBuf, tokio::task::JoinHandle<()>), String> { pub fn start(
uds: &Path,
vm_id: &str,
backend: Option<&str>,
) -> Result<(PathBuf, tokio::task::JoinHandle<()>), String> {
let path = PathBuf::from(format!("{}_{}", uds.display(), EGRESS_PORT)); let path = PathBuf::from(format!("{}_{}", uds.display(), EGRESS_PORT));
// Firecracker does not clean these up any more than it cleans up its own // Firecracker does not clean these up any more than it cleans up its own
// socket, and a stale file makes bind fail with EADDRINUSE. // socket, and a stale file makes bind fail with EADDRINUSE.
@@ -280,7 +303,7 @@ pub fn start(uds: &Path, vm_id: &str) -> Result<(PathBuf, tokio::task::JoinHandl
let listener = let listener =
UnixListener::bind(&path).map_err(|e| format!("bind {}: {e}", path.display()))?; UnixListener::bind(&path).map_err(|e| format!("bind {}: {e}", path.display()))?;
let allow = Arc::new(allow_list()); let allow = Arc::new(allow_list_for(backend));
eprintln!( eprintln!(
"microvm {vm_id}: egress proxy on {} allowing {:?}", "microvm {vm_id}: egress proxy on {} allowing {:?}",
path.display(), path.display(),
@@ -323,7 +346,38 @@ mod tests {
use super::*; use super::*;
fn allow() -> Vec<String> { fn allow() -> Vec<String> {
DEFAULT_ALLOW.iter().map(|s| s.to_string()).collect() allow_list_for(None)
}
/// MEASURED on tank, not assumed: a `glm` VM ran a whole mission to
/// completion with `api.anthropic.com` denied at this proxy, dialling only
/// `api.z.ai`. So Anthropic's host is not something a GLM agent needs — and
/// a VM that cannot reach it cannot send z.ai's key there, or Anthropic's
/// subscription token to z.ai, whatever a credential bug does upstream.
#[test]
fn each_backend_reaches_its_own_provider_and_no_other() {
let claude = allow_list_for(Some("claude"));
assert!(claude.iter().any(|h| h == "api.anthropic.com"), "{claude:?}");
assert!(!claude.iter().any(|h| h == "api.z.ai"), "{claude:?}");
let glm = allow_list_for(Some("glm"));
assert!(glm.iter().any(|h| h == "api.z.ai"), "{glm:?}");
assert!(
!glm.iter().any(|h| h.contains("anthropic")),
"a GLM VM must not be able to reach Anthropic: {glm:?}"
);
// Both still reach the forge — delivery is host-side, but a mission that
// clones or fetches needs it.
for l in [&claude, &glm] {
assert!(l.iter().any(|h| h == "git.redclaw.dev"), "{l:?}");
}
// An unknown backend gets no model API at all rather than borrowing
// somebody's: it cannot run anyway, and failing at a closed door beats
// reaching the wrong endpoint with a credential.
let unknown = allow_list_for(Some("rootfs-opus"));
assert_eq!(unknown, vec!["git.redclaw.dev".to_string()], "{unknown:?}");
} }
#[test] #[test]
+1 -1
View File
@@ -291,7 +291,7 @@ pub async fn create(
// Bound BEFORE firecracker starts: a guest that dials the host before the // Bound BEFORE firecracker starts: a guest that dials the host before the
// host is listening gets a connection refused it will not retry. // host is listening gets a connection refused it will not retry.
let (egress_uds, egress_task) = match crate::egress::start(&uds, vm_id) { let (egress_uds, egress_task) = match crate::egress::start(&uds, vm_id, backend) {
Ok((p, t)) => (p, Some(t)), Ok((p, t)) => (p, Some(t)),
// Not fatal — a VM is still useful for work that needs no network — but // Not fatal — a VM is still useful for work that needs no network — but
// it must be visible. `create`'s reply says whether egress exists, and // it must be visible. `create`'s reply says whether egress exists, and