feat(backend): local-ornith — a mission backend served by the node's own GPU

Claude Code pointed at the Ollama already installed on every GPU node. Ollama
has served a native Anthropic-compatible /v1/messages since v0.14, so this is
an env contract rather than a translation layer — the fourth variation on the
same idea as agent-glm and agent-kimi.

The route is NOT the egress proxy, and that is the design. `egress` speaks
CONNECT, takes a destination from the guest, resolves it and decides; every one
of those powers is a liability, which is why it refuses non-443 ports and IP
literals after a unit test caught them being bypassed. Routing a local model
through it would have meant relaxing both.

`local_model` is the opposite shape: there is no destination in the protocol.
fcagent listens on guest 127.0.0.1:11434 and pumps to vsock 9003; the node
splices that onto its own 127.0.0.1:11434 and copies bytes. A compromised guest
cannot redirect it because there is nothing to redirect — it is a pipe, not a
proxy, and strictly narrower than anything an allow-list could express. The
bytes never touch a network, so there is no wire for TLS to protect, and Ollama
stays bound to loopback rather than being exposed on the tailnet.

The socket is bound only for a backend declared to use a local model, so a
`local-ornith` VM reaches the forge through egress and nothing else, while every
other backend's guest port simply refuses. Both halves have negative controls.

`scripts/fleet-model-setup.sh` exists because of one measurement: stock
ornith:9b reported input_tokens=2050 for a 48000-word prompt and answered as
though nothing had been dropped. Ollama's default window is ~2K whatever the
model card says, and it truncates silently — the exact failure an agent turn
would hit and never report. The script pins num_ctx=131072 into a derived tag
and then PROVES both the window and tool calling before declaring success.
Verified on architect: ~65536 words -> 65604 input tokens, stop_reason=tool_use.

Placement needs no new capability key: building the rootfs only on GPU nodes
means `nodes::online_for_backend`'s existing `rootfs @> ["local-ornith"]`
predicate does the affinity, so morpheus never offers the backend.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-09 13:13:54 -07:00
co-authored by Claude Opus 5
parent e96c5143bc
commit f56d41f5b7
9 changed files with 498 additions and 10 deletions
+41 -3
View File
@@ -58,6 +58,14 @@ pub struct Vm {
/// one's identity.
egress: Option<tokio::task::JoinHandle<()>>,
egress_uds: PathBuf,
/// The local-model listener, held for the SAME reason as `egress`: a
/// listener that outlives its VM would accept a connection from the next VM
/// to reuse the path and serve it under the dead one's identity.
model: Option<tokio::task::JoinHandle<()>>,
/// Present only for a backend served by a model on this node. Removed on
/// destroy alongside the egress socket — a leaked unix socket is the same
/// class of host litter the TAP approach was rejected for.
model_uds: Option<PathBuf>,
}
pub type Vms = Arc<Mutex<HashMap<String, Vm>>>;
@@ -388,6 +396,23 @@ pub async fn create(
}
};
// The node's own model, for a backend that has one. Bound before firecracker
// starts for the same reason egress is: a guest that dials before the host
// listens gets a refusal it will not retry.
let (model_uds, model_task) = match crate::local_model::start(&uds, vm_id, backend) {
Ok(Some((p, t))) => (Some(p), Some(t)),
Ok(None) => (None, None),
// This backend was supposed to have a local model and does not. Not
// fatal, but the mission WILL fail on its first turn, so say why here
// rather than leaving it to look like a hung agent.
Err(e) => {
eprintln!(
"microvm {vm_id}: NO LOCAL MODEL ({e}) — a {backend:?} turn cannot reach one"
);
(None, None)
}
};
let log = std::fs::File::create(workdir.join("console.log"))
.map_err(|e| format!("create console.log: {e}"))?;
let errlog = log
@@ -418,6 +443,8 @@ pub async fn create(
uds: uds.clone(),
egress: egress_task,
egress_uds: egress_uds.clone(),
model: model_task,
model_uds: model_uds.clone(),
};
// Poll for the agent. 10s is generous: the measured boot-to-agent is under
@@ -548,14 +575,17 @@ async fn kill_group(pgid: i32) {
pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
check_id(vm_id)?;
let vm = vms.lock().await.remove(vm_id);
let (pgid, workdir, uds, egress_uds) = match vm {
let (pgid, workdir, uds, egress_uds, model_uds) = match vm {
Some(v) => {
// Abort first: a live listener would keep accepting on a path the
// next VM is about to reuse.
if let Some(t) = v.egress {
t.abort();
}
(Some(v.pgid), v.workdir, v.uds, v.egress_uds)
if let Some(t) = v.model {
t.abort();
}
(Some(v.pgid), v.workdir, v.uds, v.egress_uds, v.model_uds)
}
// Not registered: still clean the paths, so a VM created by a previous
// incarnation of the daemon can be reaped rather than orphaned forever.
@@ -563,7 +593,12 @@ pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
let wd = work_root().join("vms").join(vm_id);
let uds = wd.join("v.sock");
let eg = PathBuf::from(format!("{}_{}", uds.display(), crate::egress::EGRESS_PORT));
(None, wd.clone(), uds, eg)
let md = PathBuf::from(format!(
"{}_{}",
uds.display(),
crate::local_model::MODEL_PORT
));
(None, wd.clone(), uds, eg, Some(md))
}
};
// Killed means OBSERVED GONE, not asked-to-die.
@@ -596,6 +631,9 @@ pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
// Same trap as firecracker's own socket: nothing unlinks these for us, and a
// stale file makes the next bind fail with EADDRINUSE.
let _ = tokio::fs::remove_file(&egress_uds).await;
if let Some(m) = &model_uds {
let _ = tokio::fs::remove_file(m).await;
}
let removed = tokio::fs::remove_dir_all(&workdir).await.is_ok();
// `killed` is now observed rather than assumed; `signalled` keeps the old
// meaning so a caller can tell "there was nothing to kill" from