feat(backend): local-ornith — a mission backend served by the node's own GPU
Claude Code pointed at the Ollama already installed on every GPU node. Ollama has served a native Anthropic-compatible /v1/messages since v0.14, so this is an env contract rather than a translation layer — the fourth variation on the same idea as agent-glm and agent-kimi. The route is NOT the egress proxy, and that is the design. `egress` speaks CONNECT, takes a destination from the guest, resolves it and decides; every one of those powers is a liability, which is why it refuses non-443 ports and IP literals after a unit test caught them being bypassed. Routing a local model through it would have meant relaxing both. `local_model` is the opposite shape: there is no destination in the protocol. fcagent listens on guest 127.0.0.1:11434 and pumps to vsock 9003; the node splices that onto its own 127.0.0.1:11434 and copies bytes. A compromised guest cannot redirect it because there is nothing to redirect — it is a pipe, not a proxy, and strictly narrower than anything an allow-list could express. The bytes never touch a network, so there is no wire for TLS to protect, and Ollama stays bound to loopback rather than being exposed on the tailnet. The socket is bound only for a backend declared to use a local model, so a `local-ornith` VM reaches the forge through egress and nothing else, while every other backend's guest port simply refuses. Both halves have negative controls. `scripts/fleet-model-setup.sh` exists because of one measurement: stock ornith:9b reported input_tokens=2050 for a 48000-word prompt and answered as though nothing had been dropped. Ollama's default window is ~2K whatever the model card says, and it truncates silently — the exact failure an agent turn would hit and never report. The script pins num_ctx=131072 into a derived tag and then PROVES both the window and tool calling before declaring success. Verified on architect: ~65536 words -> 65604 input tokens, stop_reason=tool_use. Placement needs no new capability key: building the rootfs only on GPU nodes means `nodes::online_for_backend`'s existing `rootfs @> ["local-ornith"]` predicate does the affinity, so morpheus never offers the backend. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
e96c5143bc
commit
f56d41f5b7
@@ -58,6 +58,14 @@ pub struct Vm {
|
||||
/// one's identity.
|
||||
egress: Option<tokio::task::JoinHandle<()>>,
|
||||
egress_uds: PathBuf,
|
||||
/// The local-model listener, held for the SAME reason as `egress`: a
|
||||
/// listener that outlives its VM would accept a connection from the next VM
|
||||
/// to reuse the path and serve it under the dead one's identity.
|
||||
model: Option<tokio::task::JoinHandle<()>>,
|
||||
/// Present only for a backend served by a model on this node. Removed on
|
||||
/// destroy alongside the egress socket — a leaked unix socket is the same
|
||||
/// class of host litter the TAP approach was rejected for.
|
||||
model_uds: Option<PathBuf>,
|
||||
}
|
||||
|
||||
pub type Vms = Arc<Mutex<HashMap<String, Vm>>>;
|
||||
@@ -388,6 +396,23 @@ pub async fn create(
|
||||
}
|
||||
};
|
||||
|
||||
// The node's own model, for a backend that has one. Bound before firecracker
|
||||
// starts for the same reason egress is: a guest that dials before the host
|
||||
// listens gets a refusal it will not retry.
|
||||
let (model_uds, model_task) = match crate::local_model::start(&uds, vm_id, backend) {
|
||||
Ok(Some((p, t))) => (Some(p), Some(t)),
|
||||
Ok(None) => (None, None),
|
||||
// This backend was supposed to have a local model and does not. Not
|
||||
// fatal, but the mission WILL fail on its first turn, so say why here
|
||||
// rather than leaving it to look like a hung agent.
|
||||
Err(e) => {
|
||||
eprintln!(
|
||||
"microvm {vm_id}: NO LOCAL MODEL ({e}) — a {backend:?} turn cannot reach one"
|
||||
);
|
||||
(None, None)
|
||||
}
|
||||
};
|
||||
|
||||
let log = std::fs::File::create(workdir.join("console.log"))
|
||||
.map_err(|e| format!("create console.log: {e}"))?;
|
||||
let errlog = log
|
||||
@@ -418,6 +443,8 @@ pub async fn create(
|
||||
uds: uds.clone(),
|
||||
egress: egress_task,
|
||||
egress_uds: egress_uds.clone(),
|
||||
model: model_task,
|
||||
model_uds: model_uds.clone(),
|
||||
};
|
||||
|
||||
// Poll for the agent. 10s is generous: the measured boot-to-agent is under
|
||||
@@ -548,14 +575,17 @@ async fn kill_group(pgid: i32) {
|
||||
pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
|
||||
check_id(vm_id)?;
|
||||
let vm = vms.lock().await.remove(vm_id);
|
||||
let (pgid, workdir, uds, egress_uds) = match vm {
|
||||
let (pgid, workdir, uds, egress_uds, model_uds) = match vm {
|
||||
Some(v) => {
|
||||
// Abort first: a live listener would keep accepting on a path the
|
||||
// next VM is about to reuse.
|
||||
if let Some(t) = v.egress {
|
||||
t.abort();
|
||||
}
|
||||
(Some(v.pgid), v.workdir, v.uds, v.egress_uds)
|
||||
if let Some(t) = v.model {
|
||||
t.abort();
|
||||
}
|
||||
(Some(v.pgid), v.workdir, v.uds, v.egress_uds, v.model_uds)
|
||||
}
|
||||
// Not registered: still clean the paths, so a VM created by a previous
|
||||
// incarnation of the daemon can be reaped rather than orphaned forever.
|
||||
@@ -563,7 +593,12 @@ pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
|
||||
let wd = work_root().join("vms").join(vm_id);
|
||||
let uds = wd.join("v.sock");
|
||||
let eg = PathBuf::from(format!("{}_{}", uds.display(), crate::egress::EGRESS_PORT));
|
||||
(None, wd.clone(), uds, eg)
|
||||
let md = PathBuf::from(format!(
|
||||
"{}_{}",
|
||||
uds.display(),
|
||||
crate::local_model::MODEL_PORT
|
||||
));
|
||||
(None, wd.clone(), uds, eg, Some(md))
|
||||
}
|
||||
};
|
||||
// Killed means OBSERVED GONE, not asked-to-die.
|
||||
@@ -596,6 +631,9 @@ pub async fn destroy(vms: &Vms, vm_id: &str) -> Result<Value, String> {
|
||||
// Same trap as firecracker's own socket: nothing unlinks these for us, and a
|
||||
// stale file makes the next bind fail with EADDRINUSE.
|
||||
let _ = tokio::fs::remove_file(&egress_uds).await;
|
||||
if let Some(m) = &model_uds {
|
||||
let _ = tokio::fs::remove_file(m).await;
|
||||
}
|
||||
let removed = tokio::fs::remove_dir_all(&workdir).await.is_ok();
|
||||
// `killed` is now observed rather than assumed; `signalled` keeps the old
|
||||
// meaning so a caller can tell "there was nothing to kill" from
|
||||
|
||||
Reference in New Issue
Block a user