fix(fleet): a VM reaches its OWN provider and no other, measured not assumed
The GLM backend works — and proving it produced a better boundary than the one I shipped an hour ago. WHAT THE FIRST GLM MISSION SHOWED. It completed, and the delivered file said the model was "claude-opus-5". The node's egress log said the VM had dialled `api.anthropic.com` five times before `api.z.ai`. Either reading alone is consistent with a "GLM backend" that silently runs Anthropic — the exact silent-success shape this project keeps closing — so I did not accept either. THE ABLATION, run on tank rather than reasoned about: deny `anthropic.com` at the proxy and run the same mission again. It **completed**, dialling only `api.z.ai`. So the completions genuinely come from z.ai; Claude Code's calls to anthropic.com are its own telemetry, not its model traffic. And that same agent — served exclusively by z.ai, with Anthropic unreachable — still described itself as "Claude Opus 5 (1M context)". **A model's account of which model it is has no evidential value.** The proxy's log of which host it dialled does. This is the `uname -r` lesson again in a new place: ask the infrastructure, not the agent. So the allow-list is now PER BACKEND rather than a union: a `claude` VM reaches Anthropic and the forge, a `glm` VM reaches z.ai and the forge, and neither can reach the other's endpoint. A union was defensible when it was one host; once the measurement showed a GLM VM never needs Anthropic, keeping it would mean a credential mix-up upstream could still put one provider's secret on another provider's wire. Now it fails at a closed door instead. An unknown backend gets the forge and NO model API — it cannot run anyway, and borrowing somebody else's door is the failure this split prevents. An explicit `CLAWMATES_FC_EGRESS_ALLOW` still wins outright: an operator who set it drew a boundary on purpose. `DEFAULT_ALLOW` is deleted rather than left beside the new function, so there is one answer to "what may a mission reach" and not two. 534 tests pass, clippy clean. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f7f3dfe495
commit
d3a53e7bf1
@@ -43,39 +43,58 @@ use tokio::net::{TcpStream, UnixListener, UnixStream};
|
|||||||
/// Port the guest dials. Must match `fcagent`'s `EGRESS_PORT`.
|
/// Port the guest dials. Must match `fcagent`'s `EGRESS_PORT`.
|
||||||
pub const EGRESS_PORT: u32 = 9002;
|
pub const EGRESS_PORT: u32 = 9002;
|
||||||
|
|
||||||
/// Hosts a mission may reach, unless `CLAWMATES_FC_EGRESS_ALLOW` overrides it.
|
|
||||||
|
/// What every backend gets, whatever it is.
|
||||||
|
const COMMON_ALLOW: &[&str] = &["git.redclaw.dev"];
|
||||||
|
|
||||||
|
/// The model host a backend's CLI must reach, and NOTHING else.
|
||||||
///
|
///
|
||||||
/// Narrow on purpose: this is the whole egress surface of a mission, and the
|
/// Per backend rather than a union, and that is not tidiness. MEASURED on tank:
|
||||||
/// point of the exercise is that it is smaller than "the internet". An entry
|
/// a `glm` VM completed a whole mission with `api.anthropic.com` denied at this
|
||||||
/// beginning with `.` matches subdomains.
|
/// proxy, dialling only `api.z.ai` — Claude Code's calls to anthropic.com are
|
||||||
const DEFAULT_ALLOW: &[&str] = &[
|
/// its own telemetry, not its completions. So a GLM VM has no need of Anthropic
|
||||||
// The model API. Without it there is no agent.
|
/// at all, and a union allow-list would let a credential mix-up reach the wrong
|
||||||
"api.anthropic.com",
|
/// provider's endpoint instead of failing at a closed door.
|
||||||
".anthropic.com",
|
///
|
||||||
// z.ai, for a `glm` backend VM. Claude Code speaks to it through
|
/// The measurement also settled something a self-report could not: that same
|
||||||
// ANTHROPIC_BASE_URL, so the binary is the same and only the host differs —
|
/// agent, served only by z.ai, still described itself as "Claude Opus 5". A
|
||||||
// and a host the proxy denies is an agent that cannot reach any model at
|
/// model's account of which model it is has no evidential value here; the
|
||||||
// all. Adding it here rather than requiring an operator to set
|
/// proxy's log of which host it dialled does.
|
||||||
// CLAWMATES_FC_EGRESS_ALLOW: the image exists to be used, and a default that
|
fn provider_hosts(backend: Option<&str>) -> &'static [&'static str] {
|
||||||
// cannot run the images we ship is a trap, not a policy.
|
match backend {
|
||||||
"api.z.ai",
|
None | Some("") | Some("default") | Some("claude") => {
|
||||||
// Our forge: clone, push, PRs.
|
&["api.anthropic.com", ".anthropic.com"]
|
||||||
"git.redclaw.dev",
|
}
|
||||||
];
|
Some("glm") => &["api.z.ai"],
|
||||||
|
// Fail closed: a backend nobody taught this function about reaches the
|
||||||
|
// forge and no model API. It cannot silently borrow another provider's
|
||||||
|
// door, which is the failure this split exists to prevent.
|
||||||
|
Some(_) => &[],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Parse the allow-list once per VM.
|
/// Parse the allow-list once per VM.
|
||||||
///
|
///
|
||||||
/// An empty `CLAWMATES_FC_EGRESS_ALLOW` means **deny everything**, not "fall back
|
/// An empty `CLAWMATES_FC_EGRESS_ALLOW` means **deny everything**, not "fall back
|
||||||
/// to the default": an operator who blanked it asked for no egress, and quietly
|
/// to the default": an operator who blanked it asked for no egress, and quietly
|
||||||
/// restoring the default would hand a mission the network they just took away.
|
/// restoring the default would hand a mission the network they just took away.
|
||||||
fn allow_list() -> Vec<String> {
|
/// The allow-list for a VM running `backend`.
|
||||||
|
///
|
||||||
|
/// An explicit `CLAWMATES_FC_EGRESS_ALLOW` still wins outright: an operator who
|
||||||
|
/// set it asked for exactly that list, and quietly adding a provider host to it
|
||||||
|
/// would widen a boundary they had drawn on purpose.
|
||||||
|
fn allow_list_for(backend: Option<&str>) -> Vec<String> {
|
||||||
match std::env::var("CLAWMATES_FC_EGRESS_ALLOW") {
|
match std::env::var("CLAWMATES_FC_EGRESS_ALLOW") {
|
||||||
Ok(raw) => raw
|
Ok(raw) => raw
|
||||||
.split(',')
|
.split(',')
|
||||||
.map(|s| s.trim().to_ascii_lowercase())
|
.map(|s| s.trim().to_ascii_lowercase())
|
||||||
.filter(|s| !s.is_empty())
|
.filter(|s| !s.is_empty())
|
||||||
.collect(),
|
.collect(),
|
||||||
Err(_) => DEFAULT_ALLOW.iter().map(|s| s.to_string()).collect(),
|
Err(_) => COMMON_ALLOW
|
||||||
|
.iter()
|
||||||
|
.chain(provider_hosts(backend).iter())
|
||||||
|
.map(|s| s.to_string())
|
||||||
|
.collect(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -272,7 +291,11 @@ async fn reply(s: &mut UnixStream, code: u16, text: &str) -> std::io::Result<()>
|
|||||||
///
|
///
|
||||||
/// Bound **before** firecracker starts, because a guest that dials before the
|
/// Bound **before** firecracker starts, because a guest that dials before the
|
||||||
/// host is listening gets a connection refused it will not retry.
|
/// host is listening gets a connection refused it will not retry.
|
||||||
pub fn start(uds: &Path, vm_id: &str) -> Result<(PathBuf, tokio::task::JoinHandle<()>), String> {
|
pub fn start(
|
||||||
|
uds: &Path,
|
||||||
|
vm_id: &str,
|
||||||
|
backend: Option<&str>,
|
||||||
|
) -> Result<(PathBuf, tokio::task::JoinHandle<()>), String> {
|
||||||
let path = PathBuf::from(format!("{}_{}", uds.display(), EGRESS_PORT));
|
let path = PathBuf::from(format!("{}_{}", uds.display(), EGRESS_PORT));
|
||||||
// Firecracker does not clean these up any more than it cleans up its own
|
// Firecracker does not clean these up any more than it cleans up its own
|
||||||
// socket, and a stale file makes bind fail with EADDRINUSE.
|
// socket, and a stale file makes bind fail with EADDRINUSE.
|
||||||
@@ -280,7 +303,7 @@ pub fn start(uds: &Path, vm_id: &str) -> Result<(PathBuf, tokio::task::JoinHandl
|
|||||||
let listener =
|
let listener =
|
||||||
UnixListener::bind(&path).map_err(|e| format!("bind {}: {e}", path.display()))?;
|
UnixListener::bind(&path).map_err(|e| format!("bind {}: {e}", path.display()))?;
|
||||||
|
|
||||||
let allow = Arc::new(allow_list());
|
let allow = Arc::new(allow_list_for(backend));
|
||||||
eprintln!(
|
eprintln!(
|
||||||
"microvm {vm_id}: egress proxy on {} allowing {:?}",
|
"microvm {vm_id}: egress proxy on {} allowing {:?}",
|
||||||
path.display(),
|
path.display(),
|
||||||
@@ -323,7 +346,38 @@ mod tests {
|
|||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
fn allow() -> Vec<String> {
|
fn allow() -> Vec<String> {
|
||||||
DEFAULT_ALLOW.iter().map(|s| s.to_string()).collect()
|
allow_list_for(None)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// MEASURED on tank, not assumed: a `glm` VM ran a whole mission to
|
||||||
|
/// completion with `api.anthropic.com` denied at this proxy, dialling only
|
||||||
|
/// `api.z.ai`. So Anthropic's host is not something a GLM agent needs — and
|
||||||
|
/// a VM that cannot reach it cannot send z.ai's key there, or Anthropic's
|
||||||
|
/// subscription token to z.ai, whatever a credential bug does upstream.
|
||||||
|
#[test]
|
||||||
|
fn each_backend_reaches_its_own_provider_and_no_other() {
|
||||||
|
let claude = allow_list_for(Some("claude"));
|
||||||
|
assert!(claude.iter().any(|h| h == "api.anthropic.com"), "{claude:?}");
|
||||||
|
assert!(!claude.iter().any(|h| h == "api.z.ai"), "{claude:?}");
|
||||||
|
|
||||||
|
let glm = allow_list_for(Some("glm"));
|
||||||
|
assert!(glm.iter().any(|h| h == "api.z.ai"), "{glm:?}");
|
||||||
|
assert!(
|
||||||
|
!glm.iter().any(|h| h.contains("anthropic")),
|
||||||
|
"a GLM VM must not be able to reach Anthropic: {glm:?}"
|
||||||
|
);
|
||||||
|
|
||||||
|
// Both still reach the forge — delivery is host-side, but a mission that
|
||||||
|
// clones or fetches needs it.
|
||||||
|
for l in [&claude, &glm] {
|
||||||
|
assert!(l.iter().any(|h| h == "git.redclaw.dev"), "{l:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
// An unknown backend gets no model API at all rather than borrowing
|
||||||
|
// somebody's: it cannot run anyway, and failing at a closed door beats
|
||||||
|
// reaching the wrong endpoint with a credential.
|
||||||
|
let unknown = allow_list_for(Some("rootfs-opus"));
|
||||||
|
assert_eq!(unknown, vec!["git.redclaw.dev".to_string()], "{unknown:?}");
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
@@ -291,7 +291,7 @@ pub async fn create(
|
|||||||
|
|
||||||
// Bound BEFORE firecracker starts: a guest that dials the host before the
|
// Bound BEFORE firecracker starts: a guest that dials the host before the
|
||||||
// host is listening gets a connection refused it will not retry.
|
// host is listening gets a connection refused it will not retry.
|
||||||
let (egress_uds, egress_task) = match crate::egress::start(&uds, vm_id) {
|
let (egress_uds, egress_task) = match crate::egress::start(&uds, vm_id, backend) {
|
||||||
Ok((p, t)) => (p, Some(t)),
|
Ok((p, t)) => (p, Some(t)),
|
||||||
// Not fatal — a VM is still useful for work that needs no network — but
|
// Not fatal — a VM is still useful for work that needs no network — but
|
||||||
// it must be visible. `create`'s reply says whether egress exists, and
|
// it must be visible. `create`'s reply says whether egress exists, and
|
||||||
|
|||||||
Reference in New Issue
Block a user