feat(backend): local-ornith — a mission backend served by the node's own GPU

Claude Code pointed at the Ollama already installed on every GPU node. Ollama
has served a native Anthropic-compatible /v1/messages since v0.14, so this is
an env contract rather than a translation layer — the fourth variation on the
same idea as agent-glm and agent-kimi.

The route is NOT the egress proxy, and that is the design. `egress` speaks
CONNECT, takes a destination from the guest, resolves it and decides; every one
of those powers is a liability, which is why it refuses non-443 ports and IP
literals after a unit test caught them being bypassed. Routing a local model
through it would have meant relaxing both.

`local_model` is the opposite shape: there is no destination in the protocol.
fcagent listens on guest 127.0.0.1:11434 and pumps to vsock 9003; the node
splices that onto its own 127.0.0.1:11434 and copies bytes. A compromised guest
cannot redirect it because there is nothing to redirect — it is a pipe, not a
proxy, and strictly narrower than anything an allow-list could express. The
bytes never touch a network, so there is no wire for TLS to protect, and Ollama
stays bound to loopback rather than being exposed on the tailnet.

The socket is bound only for a backend declared to use a local model, so a
`local-ornith` VM reaches the forge through egress and nothing else, while every
other backend's guest port simply refuses. Both halves have negative controls.

`scripts/fleet-model-setup.sh` exists because of one measurement: stock
ornith:9b reported input_tokens=2050 for a 48000-word prompt and answered as
though nothing had been dropped. Ollama's default window is ~2K whatever the
model card says, and it truncates silently — the exact failure an agent turn
would hit and never report. The script pins num_ctx=131072 into a derived tag
and then PROVES both the window and tool calling before declaring success.
Verified on architect: ~65536 words -> 65604 input tokens, stop_reason=tool_use.

Placement needs no new capability key: building the rootfs only on GPU nodes
means `nodes::online_for_backend`'s existing `rootfs @> ["local-ornith"]`
predicate does the affinity, so morpheus never offers the backend.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-09 13:13:54 -07:00
co-authored by Claude Opus 5
parent e96c5143bc
commit f56d41f5b7
9 changed files with 498 additions and 10 deletions
+30
View File
@@ -79,6 +79,15 @@ fn provider_hosts(backend: Option<&str>) -> &'static [&'static str] {
// a different account namespace and rejects that key. Only the host the // a different account namespace and rejects that key. Only the host the
// `agent-kimi` image bakes in. // `agent-kimi` image bakes in.
Some("kimi") => &["api.kimi.com"], Some("kimi") => &["api.kimi.com"],
// A locally-hosted model reaches NOTHING through this proxy. Its route
// is `crate::local_model` — a vsock pipe to the node's own loopback,
// with no destination in the protocol — so the correct allow-list here
// is the empty one, and it falls through to the branch below.
//
// Spelled out rather than left implicit because the temptation was to
// widen this proxy instead: an entry here would have meant relaxing the
// 443-only rule AND the IP-literal refusal, both of which exist because
// a unit test caught them being bypassed.
// Fail closed: a backend nobody taught this function about reaches the // Fail closed: a backend nobody taught this function about reaches the
// forge and no model API. It cannot silently borrow another provider's // forge and no model API. It cannot silently borrow another provider's
// door, which is the failure this split exists to prevent. // door, which is the failure this split exists to prevent.
@@ -449,6 +458,27 @@ mod tests {
} }
/// A raw address must not sidestep a list written in names. /// A raw address must not sidestep a list written in names.
/// A local-model backend gets NO egress, and the 443 rule is untouched.
///
/// The alternative design routed the node's Ollama through this proxy, which
/// would have meant permitting port 11434 and an address the guest names.
/// Both are refused here, still, and a `local-ornith` VM reaches the forge
/// and nothing else — its model lives on the other socket entirely.
#[test]
fn a_local_model_backend_gets_no_egress_and_no_new_port() {
let allow = allow_list_for(Some("local-ornith"));
assert!(
allow.iter().all(|a| a == "git.redclaw.dev"),
"a local backend must reach only the forge, got {allow:?}"
);
for h in ["api.anthropic.com", "api.z.ai", "api.kimi.com", "127.0.0.1"] {
assert!(!host_allowed(h, &allow), "{h} must NOT be reachable");
}
// The rules this design exists to avoid loosening.
assert!(parse_target("anything:11434").is_err());
assert!(parse_target("127.0.0.1:443").is_ok_and(|(h, _)| !host_allowed(&h, &allow)));
}
#[test] #[test]
fn an_ip_literal_is_not_allowed() { fn an_ip_literal_is_not_allowed() {
let a = vec![".anthropic.com".to_string()]; let a = vec![".anthropic.com".to_string()];
@@ -0,0 +1,167 @@
//! Host side of a microVM's route to the node's OWN locally-hosted model.
//!
//! # Why this is not the egress proxy
//!
//! [`crate::egress`] exists so an agent can reach the public internet under an
//! allow-list: it speaks HTTP `CONNECT`, takes a destination from the guest,
//! resolves it, and decides. Every one of those powers is a liability, which is
//! why that module is careful about ports, IP literals and suffix matching.
//!
//! This is the opposite shape. There is **no destination in the protocol**. The
//! guest opens a socket; the host connects it to `127.0.0.1:11434` on the node
//! and copies bytes. A compromised guest can ask for nothing else, because there
//! is nothing to ask — it is a pipe, not a proxy. That is strictly narrower than
//! anything the allow-list could express, and it is why routing a local model
//! through `egress` would have been the worse design: it would have meant
//! relaxing the 443-only rule and the IP-literal refusal, both of which exist
//! because a unit test caught them being bypassed.
//!
//! # Why plaintext is right here
//!
//! The bytes go guest loopback → vsock → host loopback. They never touch a
//! network, so there is no wire for TLS to protect. Ollama stays bound to
//! `127.0.0.1` on the node and is never exposed to the tailnet, which is a
//! stronger position than terminating TLS in front of it would have been.
//!
//! # Why it is per-backend
//!
//! The node binds this socket only for a backend declared to use a local model.
//! On every other backend the guest's listener is still there and simply gets a
//! refusal — the same fail-closed default `provider_hosts` applies to egress.
use std::path::{Path, PathBuf};
use tokio::net::{TcpStream, UnixListener};
/// Host-side vsock port. Must match `fcagent`'s `MODEL_VSOCK_PORT`.
pub const MODEL_PORT: u32 = 9003;
/// Where the node's model server listens. Loopback, and not configurable from
/// the guest by design — see the module docs.
const OLLAMA_ADDR: &str = "127.0.0.1:11434";
/// Whether a backend is served by a model running on the node itself.
///
/// Named individually rather than by prefix. An unrecognised backend must not
/// acquire a route to anything by accident, which is the same rule
/// `egress::provider_hosts` and `mission_runtime::microvm_credential_for`
/// already apply from their own side.
pub fn uses_local_model(backend: Option<&str>) -> bool {
matches!(backend, Some("local-ornith"))
}
/// Bind the guest's local-model socket, if this backend has one.
///
/// `Ok(None)` means "this backend does not use a local model" and is the normal
/// case. An error means it should have had one and could not — reported by the
/// caller, never silently swallowed, because the symptom otherwise is an agent
/// that hangs on its first turn.
pub fn start(
uds: &Path,
vm_id: &str,
backend: Option<&str>,
) -> Result<Option<(PathBuf, tokio::task::JoinHandle<()>)>, String> {
if !uses_local_model(backend) {
return Ok(None);
}
let path = PathBuf::from(format!("{}_{}", uds.display(), MODEL_PORT));
// Firecracker leaves these behind exactly as it does its own socket, and a
// stale file makes bind fail with EADDRINUSE.
let _ = std::fs::remove_file(&path);
let listener =
UnixListener::bind(&path).map_err(|e| format!("bind {}: {e}", path.display()))?;
eprintln!(
"microvm {vm_id}: local model socket on {} -> {OLLAMA_ADDR}",
path.display()
);
let vm = vm_id.to_string();
let task = tokio::spawn(async move {
loop {
match listener.accept().await {
Ok((s, _)) => {
let vm = vm.clone();
tokio::spawn(async move {
if let Err(e) = pipe(s).await {
// Loud, because the failure a mission sees is a turn
// that never answers. A refused connection here means
// the node's model server is down, and that is worth
// saying out loud rather than leaving to a timeout.
eprintln!("microvm {vm}: local model pipe failed: {e}");
}
});
}
Err(e) => {
eprintln!("microvm {vm}: local model accept failed: {e}");
return;
}
}
}
});
Ok(Some((path, task)))
}
/// Splice one guest connection onto a fresh connection to the node's model.
async fn pipe(mut guest: tokio::net::UnixStream) -> Result<(), String> {
let mut model = TcpStream::connect(OLLAMA_ADDR)
.await
.map_err(|e| format!("connect {OLLAMA_ADDR}: {e}"))?;
tokio::io::copy_bidirectional(&mut guest, &mut model)
.await
.map(|_| ())
.map_err(|e| format!("copy: {e}"))
}
#[cfg(test)]
mod tests {
use super::*;
/// Only the backends that are meant to have a local model get one.
///
/// The negative half is the point: an unrecognised backend acquiring a route
/// to the node's own model server would be a hole opened by a typo, and it
/// would be invisible because the mission would simply work.
#[test]
fn a_local_route_is_never_granted_by_accident() {
assert!(uses_local_model(Some("local-ornith")));
for other in [
None,
Some(""),
Some("default"),
Some("claude"),
Some("canary-claude"),
Some("glm"),
Some("kimi"),
Some("local"),
Some("local-ornith-typo"),
Some("ornith"),
] {
assert!(
!uses_local_model(other),
"{other:?} must not reach the node's model server"
);
}
}
/// The guest cannot name a destination, so there is nothing to validate.
///
/// This asserts the property that makes this module safe enough to skip the
/// allow-list entirely: the upstream address is a constant. If it ever
/// becomes a parameter, this file needs everything `egress` has.
#[test]
fn the_upstream_address_is_a_constant_not_an_input() {
let src = include_str!("local_model.rs");
// Needles are split so they do not match themselves in this file.
assert_eq!(
src.matches(concat!("TcpStream", "::connect(")).count(),
1,
"exactly one dial site, and it must use the constant"
);
assert!(src.contains(concat!("TcpStream", "::connect(OLLAMA_ADDR)")));
assert!(
OLLAMA_ADDR.starts_with("127.0.0.1:"),
"the model server must be reached on loopback only"
);
}
}
+1
View File
@@ -20,6 +20,7 @@ use tokio::sync::{mpsc, Mutex};
use tokio_tungstenite::tungstenite::Message; use tokio_tungstenite::tungstenite::Message;
mod egress; mod egress;
mod local_model;
mod microvm; mod microvm;
mod rtc; mod rtc;
+41 -3
View File
@@ -58,6 +58,14 @@ pub struct Vm {
/// one's identity. /// one's identity.
egress: Option<tokio::task::JoinHandle<()>>, egress: Option<tokio::task::JoinHandle<()>>,
egress_uds: PathBuf, egress_uds: PathBuf,
/// The local-model listener, held for the SAME reason as `egress`: a
/// listener that outlives its VM would accept a connection from the next VM
/// to reuse the path and serve it under the dead one's identity.
model: Option<tokio::task::JoinHandle<()>>,
/// Present only for a backend served by a model on this node. Removed on
/// destroy alongside the egress socket — a leaked unix socket is the same
/// class of host litter the TAP approach was rejected for.
model_uds: Option<PathBuf>,
} }
pub type Vms = Arc<Mutex<HashMap<String, Vm>>>; pub type Vms = Arc<Mutex<HashMap<String, Vm>>>;
@@ -388,6 +396,23 @@ pub async fn create(
} }
}; };
// The node's own model, for a backend that has one. Bound before firecracker
// starts for the same reason egress is: a guest that dials before the host
// listens gets a refusal it will not retry.
let (model_uds, model_task) = match crate::local_model::start(&uds, vm_id, backend) {
Ok(Some((p, t))) => (Some(p), Some(t)),
Ok(None) => (None, None),
// This backend was supposed to have a local model and does not. Not
// fatal, but the mission WILL fail on its first turn, so say why here
// rather than leaving it to look like a hung agent.
Err(e) => {
eprintln!(
"microvm {vm_id}: NO LOCAL MODEL ({e}) — a {backend:?} turn cannot reach one"
);
(None, None)
}
};
let log = std::fs::File::create(workdir.join("console.log")) let log = std::fs::File::create(workdir.join("console.log"))
.map_err(|e| format!("create console.log: {e}"))?; .map_err(|e| format!("create console.log: {e}"))?;
let errlog = log let errlog = log
@@ -418,6 +443,8 @@ pub async fn create(
uds: uds.clone(), uds: uds.clone(),
egress: egress_task, egress: egress_task,
egress_uds: egress_uds.clone(), egress_uds: egress_uds.clone(),
model: model_task,
model_uds: model_uds.clone(),
}; };
// Poll for the agent. 10s is generous: the measured boot-to-agent is under // Poll for the agent. 10s is generous: the measured boot-to-agent is under
@@ -548,14 +575,17 @@ async fn kill_group(pgid: i32) {
pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> { pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
check_id(vm_id)?; check_id(vm_id)?;
let vm = vms.lock().await.remove(vm_id); let vm = vms.lock().await.remove(vm_id);
let (pgid, workdir, uds, egress_uds) = match vm { let (pgid, workdir, uds, egress_uds, model_uds) = match vm {
Some(v) => { Some(v) => {
// Abort first: a live listener would keep accepting on a path the // Abort first: a live listener would keep accepting on a path the
// next VM is about to reuse. // next VM is about to reuse.
if let Some(t) = v.egress { if let Some(t) = v.egress {
t.abort(); t.abort();
} }
(Some(v.pgid), v.workdir, v.uds, v.egress_uds) if let Some(t) = v.model {
t.abort();
}
(Some(v.pgid), v.workdir, v.uds, v.egress_uds, v.model_uds)
} }
// Not registered: still clean the paths, so a VM created by a previous // Not registered: still clean the paths, so a VM created by a previous
// incarnation of the daemon can be reaped rather than orphaned forever. // incarnation of the daemon can be reaped rather than orphaned forever.
@@ -563,7 +593,12 @@ pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
let wd = work_root().join("vms").join(vm_id); let wd = work_root().join("vms").join(vm_id);
let uds = wd.join("v.sock"); let uds = wd.join("v.sock");
let eg = PathBuf::from(format!("{}_{}", uds.display(), crate::egress::EGRESS_PORT)); let eg = PathBuf::from(format!("{}_{}", uds.display(), crate::egress::EGRESS_PORT));
(None, wd.clone(), uds, eg) let md = PathBuf::from(format!(
"{}_{}",
uds.display(),
crate::local_model::MODEL_PORT
));
(None, wd.clone(), uds, eg, Some(md))
} }
}; };
// Killed means OBSERVED GONE, not asked-to-die. // Killed means OBSERVED GONE, not asked-to-die.
@@ -596,6 +631,9 @@ pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
// Same trap as firecracker's own socket: nothing unlinks these for us, and a // Same trap as firecracker's own socket: nothing unlinks these for us, and a
// stale file makes the next bind fail with EADDRINUSE. // stale file makes the next bind fail with EADDRINUSE.
let _ = tokio::fs::remove_file(&egress_uds).await; let _ = tokio::fs::remove_file(&egress_uds).await;
if let Some(m) = &model_uds {
let _ = tokio::fs::remove_file(m).await;
}
let removed = tokio::fs::remove_dir_all(&workdir).await.is_ok(); let removed = tokio::fs::remove_dir_all(&workdir).await.is_ok();
// `killed` is now observed rather than assumed; `signalled` keeps the old // `killed` is now observed rather than assumed; `signalled` keeps the old
// meaning so a caller can tell "there was nothing to kill" from // meaning so a caller can tell "there was nothing to kill" from
+42 -6
View File
@@ -51,6 +51,25 @@ const PROXY_PORT: u16 = 3128;
/// Host-side vsock port the tunnel lands on. Firecracker's convention for a /// Host-side vsock port the tunnel lands on. Firecracker's convention for a
/// guest-initiated connection is that the HOST listens on `<uds_path>_<port>`. /// guest-initiated connection is that the HOST listens on `<uds_path>_<port>`.
const EGRESS_PORT: u32 = 9002; const EGRESS_PORT: u32 = 9002;
/// Guest-side port for a LOCALLY HOSTED model, and the vsock port it lands on.
///
/// Separate from the egress proxy on purpose, and simpler than it. The egress
/// path exists to let an agent reach the public internet under an allow-list;
/// this one reaches exactly one thing — the Ollama the node itself is running,
/// on its own loopback — and can reach nothing else, because the host end is a
/// pipe to a fixed address rather than a proxy that takes a destination.
///
/// It therefore needs no `CONNECT`, no TLS and no allow-list. The bytes travel
/// guest loopback → vsock → host loopback and never touch a network, so there is
/// nothing on a wire for TLS to protect. `NO_PROXY` already contains
/// `127.0.0.1`, so an agent pointed at `http://127.0.0.1:11434` bypasses the
/// egress proxy entirely rather than trying to CONNECT through it.
///
/// The guest always listens. Whether anything answers is the HOST's decision:
/// the node only binds the vsock end for a backend that is meant to have a
/// local model, so on every other backend this port simply refuses.
const MODEL_PORT: u16 = 11434;
const MODEL_VSOCK_PORT: u32 = 9003;
/// `VMADDR_CID_HOST` — the hypervisor side of the vsock. /// `VMADDR_CID_HOST` — the hypervisor side of the vsock.
const HOST_CID: u32 = 2; const HOST_CID: u32 = 2;
@@ -193,7 +212,24 @@ fn start_egress_proxy() {
PROXY_UP.store(true, Ordering::Relaxed); PROXY_UP.store(true, Ordering::Relaxed);
println!("FC-AGENT-PROXY listening on 127.0.0.1:{PROXY_PORT} -> vsock {EGRESS_PORT}"); println!("FC-AGENT-PROXY listening on 127.0.0.1:{PROXY_PORT} -> vsock {EGRESS_PORT}");
let _ = std::io::stdout().flush(); let _ = std::io::stdout().flush();
pump(listener, EGRESS_PORT, "PROXY");
// The local-model port. Failure to bind is reported and non-fatal, exactly
// like the egress proxy: a VM whose backend does not use a local model is
// still perfectly useful, and a fatal error here would take out every
// backend to serve one.
match TcpListener::bind(("127.0.0.1", MODEL_PORT)) {
Ok(l) => {
println!("FC-AGENT-MODEL listening on 127.0.0.1:{MODEL_PORT} -> vsock {MODEL_VSOCK_PORT}");
let _ = std::io::stdout().flush();
pump(l, MODEL_VSOCK_PORT, "MODEL");
}
Err(e) => eprintln!("FC-AGENT-NO-MODEL could not listen on 127.0.0.1:{MODEL_PORT}: {e}"),
}
}
/// Accept forever, splicing each connection onto its own vsock stream.
fn pump(listener: TcpListener, vsock_port: u32, tag: &'static str) {
std::thread::spawn(move || { std::thread::spawn(move || {
for c in listener.incoming() { for c in listener.incoming() {
match c { match c {
@@ -201,12 +237,12 @@ fn start_egress_proxy() {
// and serving them in sequence would look like a hang. // and serving them in sequence would look like a hang.
Ok(tcp) => { Ok(tcp) => {
std::thread::spawn(move || { std::thread::spawn(move || {
if let Err(e) = tunnel(tcp) { if let Err(e) = tunnel(tcp, vsock_port) {
eprintln!("FC-AGENT-PROXY-ERROR {e}"); eprintln!("FC-AGENT-{tag}-ERROR {e}");
} }
}); });
} }
Err(e) => eprintln!("FC-AGENT-PROXY-ERROR accept: {e}"), Err(e) => eprintln!("FC-AGENT-{tag}-ERROR accept: {e}"),
} }
} }
}); });
@@ -217,9 +253,9 @@ fn start_egress_proxy() {
/// No parsing: whatever the client sent — `CONNECT host:443`, or an absolute-form /// No parsing: whatever the client sent — `CONNECT host:443`, or an absolute-form
/// request — is the host's business. The host answers with real HTTP, so a /// request — is the host's business. The host answers with real HTTP, so a
/// refusal reaches the client as a status code rather than a dropped socket. /// refusal reaches the client as a status code rather than a dropped socket.
fn tunnel(tcp: std::net::TcpStream) -> Result<(), String> { fn tunnel(tcp: std::net::TcpStream, vsock_port: u32) -> Result<(), String> {
let vs = vsock::VsockStream::connect_with_cid_port(HOST_CID, EGRESS_PORT) let vs = vsock::VsockStream::connect_with_cid_port(HOST_CID, vsock_port)
.map_err(|e| format!("vsock connect to host:{EGRESS_PORT}: {e}"))?; .map_err(|e| format!("vsock connect to host:{vsock_port}: {e}"))?;
let (mut tcp_r, mut tcp_w) = ( let (mut tcp_r, mut tcp_w) = (
tcp.try_clone().map_err(|e| format!("clone tcp: {e}"))?, tcp.try_clone().map_err(|e| format!("clone tcp: {e}"))?,
+54
View File
@@ -199,6 +199,20 @@ fn microvm_credential_for(backend: Option<&str>) -> Result<Credential, String> {
source: "KIMI_API_KEY", source: "KIMI_API_KEY",
target: "ANTHROPIC_AUTH_TOKEN", target: "ANTHROPIC_AUTH_TOKEN",
}), }),
// A model running on the NODE ITSELF. There is no credential and there
// is nothing to protect: the guest reaches it over a vsock pipe to the
// node's own loopback (`clawmates-node::local_model`), which has no
// destination in its protocol and can therefore reach nothing else.
//
// It still goes through this map rather than around it. Ollama ignores
// the bearer but Claude Code refuses to start without one, so the value
// is a literal — and routing it here keeps the rule that a backend with
// no entry cannot launch, which is what stops a typo'd backend from
// quietly inheriting the subscription token.
Some("local-ornith") => Ok(Credential {
source: "CLAWMATES_LOCAL_MODEL_TOKEN",
target: "ANTHROPIC_AUTH_TOKEN",
}),
// Kimi, on the same split as GLM: endpoint in the image // Kimi, on the same split as GLM: endpoint in the image
// (`https://api.kimi.com/coding`), credential in the turn. // (`https://api.kimi.com/coding`), credential in the turn.
// //
@@ -222,6 +236,14 @@ fn microvm_provider_env_from(
lookup: impl Fn(&str) -> Option<String>, lookup: impl Fn(&str) -> Option<String>,
) -> Result<Vec<(String, String)>, String> { ) -> Result<Vec<(String, String)>, String> {
let want = microvm_credential_for(backend)?; let want = microvm_credential_for(backend)?;
// A local model has no secret to look up. Defaulted rather than demanded:
// requiring an operator to set a variable whose value is ignored is a step
// that only ever fails, and its failure mode here is a hung agent.
let lookup = |k: &str| {
lookup(k).or_else(|| {
(k == "CLAWMATES_LOCAL_MODEL_TOKEN").then(|| "local-model-no-auth".to_string())
})
};
let token = lookup(want.source) let token = lookup(want.source)
.filter(|v| !v.trim().is_empty()) .filter(|v| !v.trim().is_empty())
.ok_or_else(|| { .ok_or_else(|| {
@@ -1199,6 +1221,38 @@ mod tests {
/// `CLAWMATES_RUNTIME_AUTH` at all. If it did, the single unset variable on /// `CLAWMATES_RUNTIME_AUTH` at all. If it did, the single unset variable on
/// gw-04 today would put an API key inside every VM, and Claude Code ranks /// gw-04 today would put an API key inside every VM, and Claude Code ranks
/// the key above the subscription token: it would work, and bill per-token /// the key above the subscription token: it would work, and bill per-token
/// A local backend carries no secret, and no secret reaches it.
///
/// The credential map is the one place a backend can acquire a token, and
/// the point of routing `local-ornith` through it — rather than special-
/// casing it earlier — is that the "unknown backend cannot launch" rule
/// still holds. A typo like `local-ornit` must fail closed, not inherit the
/// subscription token on its way to somebody else's endpoint.
#[test]
fn a_local_backend_gets_a_placeholder_and_never_a_real_credential() {
let env = microvm_provider_env_from(Some("local-ornith"), |_| None)
.expect("a local backend needs nothing from the operator");
let token = env
.iter()
.find(|(k, _)| k == "ANTHROPIC_AUTH_TOKEN")
.map(|(_, v)| v.clone())
.expect("Claude Code refuses to start without a bearer");
assert!(
!token.starts_with("sk-"),
"a local backend must never be handed a real credential, got {token:?}"
);
assert!(!env.iter().any(|(k, _)| k == "ANTHROPIC_API_KEY"));
assert!(!env.iter().any(|(k, _)| k == "CLAUDE_CODE_OAUTH_TOKEN"));
// And a near-miss still cannot launch.
for typo in ["local-ornit", "ornith", "local", "local-ornith2"] {
assert!(
microvm_credential_for(Some(typo)).is_err(),
"{typo} must be refused, not resolved to something"
);
}
}
/// against a plan already paid for, with no symptom but the invoice. /// against a plan already paid for, with no symptom but the invoice.
#[test] #[test]
fn a_microvm_never_receives_an_anthropic_api_key() { fn a_microvm_never_receives_an_anthropic_api_key() {
+55
View File
@@ -0,0 +1,55 @@
# Claude Code pointed at a model running on the NODE ITSELF.
#
# The third variation on one idea: `agent-claude` talks to Anthropic,
# `agent-glm` to z.ai, `agent-kimi` to Moonshot, and this one to the Ollama
# already installed on every GPU node. Ollama has served a native
# Anthropic-compatible `/v1/messages` since v0.14, so this costs an env contract
# rather than a translation layer — MEASURED on both nodes: a tools request
# comes back with a well-formed `tool_use` block and `stop_reason=tool_use`.
#
# Why the base URL is loopback INSIDE the guest, not a hostname:
#
# The VM has no network interface. `fcagent` listens on 127.0.0.1:11434 and
# pumps to vsock 9003, where `clawmates-node::local_model` splices it onto the
# node's own `127.0.0.1:11434`. There is no destination anywhere in that path —
# it is a pipe to a fixed address, so a compromised guest cannot redirect it.
# `NO_PROXY` already contains 127.0.0.1, so this bypasses the egress proxy
# rather than trying to CONNECT through it.
#
# Build (on a node with a GPU — architect or tank):
#
# ssh architect "cd ~/clawmates && \
# docker build -f images/agent-ornith/Dockerfile -t clawmates/agent-ornith:dev images/agent-ornith/"
# scripts/fc-build-rootfs.sh architect clawmates/agent-ornith:dev local-ornith 8G
#
# Building it ONLY on GPU nodes is deliberate and is the whole placement story:
# a node advertises `rootfs-local-ornith.ext4` in `capabilities.rootfs`, and
# `nodes::online_for_backend` already filters on exactly that. No GPU capability
# key, no migration — morpheus simply never offers this backend.
FROM clawmates/agent-toolchain:dev
# Pinned to the SAME version as agent-claude and agent-glm. A run that differs
# by provider should differ by nothing else, or "the local backend behaved
# differently" is ambiguous between the model and the harness.
ARG CLAUDE_CODE_VERSION=2.1.223
RUN npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" \
&& npm cache clean --force \
&& rm -rf /root/.npm \
&& claude --version
# `ornith-fleet:9b`, NOT `ornith:9b`. Ollama defaults to a ~2K context window
# whatever the model card says, and it truncates silently: MEASURED, stock
# ornith:9b reported input_tokens=2050 for a 48000-word prompt and answered as
# though nothing had been dropped. The fleet tag pins num_ctx=131072 and is
# created by `scripts/fleet-model-setup.sh`. An agent turn is exactly the
# workload that would hit the default and never say so.
ENV HOME=/root \
CLAWMATES_AGENT_CLI=claude \
ANTHROPIC_BASE_URL=http://127.0.0.1:11434 \
ANTHROPIC_MODEL=ornith-fleet:9b \
ANTHROPIC_SMALL_FAST_MODEL=ornith-fleet:9b
RUN mkdir -p /root/.claude
# No provider key of any kind. `microvm_credential_for` hands this backend a
# literal placeholder because Ollama ignores the bearer and Claude Code refuses
# to start without one. There is no secret in this image and none reaches it.
+4 -1
View File
@@ -54,7 +54,10 @@ case "${FC_CLI-unset}" in
# image is `claude` for all three. Moonshot ship their own `kimi` # image is `claude` for all three. Moonshot ship their own `kimi`
# CLI, and this map used to expect it — a leftover from before the # CLI, and this map used to expect it — a leftover from before the
# endpoint was measured, which failed a perfectly good rootfs. # endpoint was measured, which failed a perfectly good rootfs.
claude|glm|kimi) FC_CLI="claude --version" ;; # `local-ornith` joins them: Claude Code again, pointed at the
# node's own Ollama. The binary in the image is `claude` for all
# four; only ANTHROPIC_BASE_URL differs.
claude|glm|kimi|local-ornith) FC_CLI="claude --version" ;;
*) FC_CLI="" ;; *) FC_CLI="" ;;
esac ;; esac ;;
esac esac
+104
View File
@@ -0,0 +1,104 @@
#!/usr/bin/env bash
# Put a locally-hosted model on a GPU node, configured the way the fleet needs it.
#
# The whole reason this is a script and not two commands in a README:
#
# MEASURED on tank — stock `ornith:9b` reported input_tokens=2050 for a
# 48000-word prompt, and answered as though nothing had been dropped.
#
# Ollama defaults to a ~2K context window whatever the model card says, and it
# truncates SILENTLY. An agent turn is exactly the workload that hits that
# default, and the symptom is not an error: it is a model that answers
# confidently about a prompt it never saw. So the fleet never uses the stock tag.
# It uses a derived one with `num_ctx` pinned, and that tag is what
# `images/agent-ornith` bakes into ANTHROPIC_MODEL.
#
# Sizing, measured on a 16 GB RTX 5060 Ti with a 5.6 GB model:
#
# num_ctx 32768 -> 6.3 GB resident
# num_ctx 65536 -> 7.2 GB
# num_ctx 131072 -> 9.3 GB <- the default here
# num_ctx 262144 -> 13.6 GB (fits, 100% GPU, but leaves little headroom)
#
# Usage:
# scripts/fleet-model-setup.sh <ssh-host> [base-model] [fleet-tag] [num_ctx]
# scripts/fleet-model-setup.sh architect
set -uo pipefail
HOST="${1:?usage: $0 <ssh-host> [base-model] [fleet-tag] [num_ctx]}"
BASE="${2:-ornith:9b}"
TAG="${3:-ornith-fleet:9b}"
CTX="${4:-131072}"
die() { printf 'ABORT %s\n' "$*" >&2; exit 2; }
ok() { printf 'OK %s\n' "$*"; }
ssh -o BatchMode=yes -o ConnectTimeout=10 "$HOST" true 2>/dev/null || die "cannot ssh to $HOST"
# A GPU is not optional. This model on CPU is slow enough that a mission would
# time out rather than fail, which is the worse outcome.
ssh "$HOST" 'command -v nvidia-smi >/dev/null && nvidia-smi -L' \
|| die "$HOST has no NVIDIA GPU — a local backend belongs on a GPU node"
ssh "$HOST" 'command -v ollama >/dev/null' \
|| die "$HOST has no ollama (the fleet page's update button installs it)"
# Version gate, with the reason. architect ran 0.30.5 and could not pull this
# model AT ALL — "requires a newer version of Ollama" — which is at least a loud
# failure. The quiet one is older still: /v1/messages only exists from v0.14.
have=$(ssh "$HOST" 'ollama --version' | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)
[ -n "$have" ] || die "could not read ollama's version on $HOST"
printf 'ollama %s on %s\n' "$have" "$HOST"
ssh "$HOST" "ollama pull '$BASE'" >/dev/null 2>&1 \
|| die "could not pull $BASE on $HOST — is ollama new enough for it?"
ok "$BASE pulled"
# The derived tag. `ollama create` is idempotent, so re-running is safe.
ssh "$HOST" "printf 'FROM %s\nPARAMETER num_ctx %s\n' '$BASE' '$CTX' > /tmp/Modelfile.fleet \
&& ollama create '$TAG' -f /tmp/Modelfile.fleet" >/dev/null 2>&1 \
|| die "could not create $TAG on $HOST"
ok "$TAG created with num_ctx=$CTX"
# PROVE the window, do not assume it. This is the check the whole script exists
# for: a Modelfile that silently failed to apply looks exactly like one that
# worked, right up until an agent loses its context mid-mission.
words=$((CTX / 2))
reported=$(ssh "$HOST" "python3 - <<'PY'
import json, urllib.request
filler = 'the quick brown fox jumps over the lazy dog ' * ($words // 9)
body = {'model': '$TAG', 'max_tokens': 8,
'messages': [{'role': 'user', 'content': 'Log:\n' + filler + '\nReply OK.'}]}
req = urllib.request.Request('http://127.0.0.1:11434/v1/messages',
data=json.dumps(body).encode(),
headers={'content-type': 'application/json', 'x-api-key': 'ollama'})
try:
print(json.load(urllib.request.urlopen(req, timeout=1800))['usage']['input_tokens'])
except Exception as e:
print('ERR', e)
PY")
case "$reported" in
''|*[!0-9]*) die "context probe failed on $HOST: $reported" ;;
esac
# Allow for tokenisation slack, but nothing like a truncation. 2050 is what the
# stock tag reports for ANY prompt above it; anything near that means the
# num_ctx did not take.
floor=$((words / 2))
[ "$reported" -gt "$floor" ] \
|| die "$TAG reported only $reported input tokens for a ~$words-word prompt — \
the context window did not take, and it would truncate silently in a mission"
ok "context proven: ~$words words -> $reported input tokens"
# Tool calling, the other thing a mission agent cannot work without.
tools=$(ssh "$HOST" "curl -s -m 300 http://127.0.0.1:11434/v1/messages \
-H 'content-type: application/json' -H 'x-api-key: ollama' \
-d '{\"model\":\"$TAG\",\"max_tokens\":300,\"tools\":[{\"name\":\"read_file\",\"description\":\"Read a file\",\"input_schema\":{\"type\":\"object\",\"properties\":{\"path\":{\"type\":\"string\"}},\"required\":[\"path\"]}}],\"messages\":[{\"role\":\"user\",\"content\":\"Read src/lib.rs using the tool.\"}]}' \
| python3 -c \"import json,sys; d=json.load(sys.stdin); print(d.get('stop_reason'), [b.get('type') for b in d.get('content',[])])\"")
case "$tools" in
tool_use*) ok "tool calling proven: $tools" ;;
*) die "no tool_use from $TAG ($tools) — Claude Code cannot drive a model that cannot call tools" ;;
esac
printf '\n%s is ready on %s. Next: build the rootfs there —\n' "$TAG" "$HOST"
printf ' ssh %s "cd ~/clawmates && docker build -f images/agent-ornith/Dockerfile -t clawmates/agent-ornith:dev images/agent-ornith/"\n' "$HOST"
printf ' scripts/fc-build-rootfs.sh %s clawmates/agent-ornith:dev local-ornith 8G\n' "$HOST"