osobh/clawmates
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(backend): local-ornith — a mission backend served by the node's own GPU

Browse Source 
Claude Code pointed at the Ollama already installed on every GPU node. Ollama
has served a native Anthropic-compatible /v1/messages since v0.14, so this is
an env contract rather than a translation layer — the fourth variation on the
same idea as agent-glm and agent-kimi.

The route is NOT the egress proxy, and that is the design. `egress` speaks
CONNECT, takes a destination from the guest, resolves it and decides; every one
of those powers is a liability, which is why it refuses non-443 ports and IP
literals after a unit test caught them being bypassed. Routing a local model
through it would have meant relaxing both.

`local_model` is the opposite shape: there is no destination in the protocol.
fcagent listens on guest 127.0.0.1:11434 and pumps to vsock 9003; the node
splices that onto its own 127.0.0.1:11434 and copies bytes. A compromised guest
cannot redirect it because there is nothing to redirect — it is a pipe, not a
proxy, and strictly narrower than anything an allow-list could express. The
bytes never touch a network, so there is no wire for TLS to protect, and Ollama
stays bound to loopback rather than being exposed on the tailnet.

The socket is bound only for a backend declared to use a local model, so a
`local-ornith` VM reaches the forge through egress and nothing else, while every
other backend's guest port simply refuses. Both halves have negative controls.

`scripts/fleet-model-setup.sh` exists because of one measurement: stock
ornith:9b reported input_tokens=2050 for a 48000-word prompt and answered as
though nothing had been dropped. Ollama's default window is ~2K whatever the
model card says, and it truncates silently — the exact failure an agent turn
would hit and never report. The script pins num_ctx=131072 into a derived tag
and then PROVES both the window and tool calling before declaring success.
Verified on architect: ~65536 words -> 65604 input tokens, stop_reason=tool_use.

Placement needs no new capability key: building the rootfs only on GPU nodes
means `nodes::online_for_backend`'s existing `rootfs @> ["local-ornith"]`
predicate does the affinity, so morpheus never offers the backend.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-09 13:13:54 -07:00
co-authored by Claude Opus 5
parent e96c5143bc
commit f56d41f5b7
9 changed files with 498 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/bins/fcagent/src/main.rs
+42 -6
View File
@@ -51,6 +51,25 @@ const PROXY_PORT: u16 = 3128;
/// Host-side vsock port the tunnel lands on. Firecracker's convention for a
/// guest-initiated connection is that the HOST listens on `<uds_path>_<port>`.
const EGRESS_PORT: u32 = 9002;
/// Guest-side port for a LOCALLY HOSTED model, and the vsock port it lands on.
///
/// Separate from the egress proxy on purpose, and simpler than it. The egress
/// path exists to let an agent reach the public internet under an allow-list;
/// this one reaches exactly one thing — the Ollama the node itself is running,
/// on its own loopback — and can reach nothing else, because the host end is a
/// pipe to a fixed address rather than a proxy that takes a destination.
///
/// It therefore needs no `CONNECT`, no TLS and no allow-list. The bytes travel
/// guest loopback → vsock → host loopback and never touch a network, so there is
/// nothing on a wire for TLS to protect. `NO_PROXY` already contains
/// `127.0.0.1`, so an agent pointed at `http://127.0.0.1:11434` bypasses the
/// egress proxy entirely rather than trying to CONNECT through it.
///
/// The guest always listens. Whether anything answers is the HOST's decision:
/// the node only binds the vsock end for a backend that is meant to have a
/// local model, so on every other backend this port simply refuses.
const MODEL_PORT: u16 = 11434;
const MODEL_VSOCK_PORT: u32 = 9003;
/// `VMADDR_CID_HOST` — the hypervisor side of the vsock.
const HOST_CID: u32 = 2;
@@ -193,7 +212,24 @@ fn start_egress_proxy() {
PROXY_UP.store(true, Ordering::Relaxed);
println!("FC-AGENT-PROXY listening on 127.0.0.1:{PROXY_PORT} -> vsock {EGRESS_PORT}");
let _ = std::io::stdout().flush();
pump(listener, EGRESS_PORT, "PROXY");
// The local-model port. Failure to bind is reported and non-fatal, exactly
// like the egress proxy: a VM whose backend does not use a local model is
// still perfectly useful, and a fatal error here would take out every
// backend to serve one.
match TcpListener::bind(("127.0.0.1", MODEL_PORT)) {
Ok(l) => {
println!("FC-AGENT-MODEL listening on 127.0.0.1:{MODEL_PORT} -> vsock {MODEL_VSOCK_PORT}");
let _ = std::io::stdout().flush();
pump(l, MODEL_VSOCK_PORT, "MODEL");
}
Err(e) => eprintln!("FC-AGENT-NO-MODEL could not listen on 127.0.0.1:{MODEL_PORT}: {e}"),
}
}
/// Accept forever, splicing each connection onto its own vsock stream.
fn pump(listener: TcpListener, vsock_port: u32, tag: &'static str) {
std::thread::spawn(move || {
for c in listener.incoming() {
match c {
@@ -201,12 +237,12 @@ fn start_egress_proxy() {
// and serving them in sequence would look like a hang.
Ok(tcp) => {
std::thread::spawn(move || {
if let Err(e) = tunnel(tcp) {
eprintln!("FC-AGENT-PROXY-ERROR {e}");
if let Err(e) = tunnel(tcp, vsock_port) {
eprintln!("FC-AGENT-{tag}-ERROR {e}");
}
});
}
Err(e) => eprintln!("FC-AGENT-PROXY-ERROR accept: {e}"),
Err(e) => eprintln!("FC-AGENT-{tag}-ERROR accept: {e}"),
}
}
});
@@ -217,9 +253,9 @@ fn start_egress_proxy() {
/// No parsing: whatever the client sent — `CONNECT host:443`, or an absolute-form
/// request — is the host's business. The host answers with real HTTP, so a
/// refusal reaches the client as a status code rather than a dropped socket.
fn tunnel(tcp: std::net::TcpStream) -> Result<(), String> {
let vs = vsock::VsockStream::connect_with_cid_port(HOST_CID, EGRESS_PORT)
.map_err(|e| format!("vsock connect to host:{EGRESS_PORT}: {e}"))?;
fn tunnel(tcp: std::net::TcpStream, vsock_port: u32) -> Result<(), String> {
let vs = vsock::VsockStream::connect_with_cid_port(HOST_CID, vsock_port)
.map_err(|e| format!("vsock connect to host:{vsock_port}: {e}"))?;
let (mut tcp_r, mut tcp_w) = (
tcp.try_clone().map_err(|e| format!("clone tcp: {e}"))?,