Omar SobhandClaude Opus 5
f56d41f5b7
Claude Code pointed at the Ollama already installed on every GPU node. Ollama has served a native Anthropic-compatible /v1/messages since v0.14, so this is an env contract rather than a translation layer — the fourth variation on the same idea as agent-glm and agent-kimi. The route is NOT the egress proxy, and that is the design. `egress` speaks CONNECT, takes a destination from the guest, resolves it and decides; every one of those powers is a liability, which is why it refuses non-443 ports and IP literals after a unit test caught them being bypassed. Routing a local model through it would have meant relaxing both. `local_model` is the opposite shape: there is no destination in the protocol. fcagent listens on guest 127.0.0.1:11434 and pumps to vsock 9003; the node splices that onto its own 127.0.0.1:11434 and copies bytes. A compromised guest cannot redirect it because there is nothing to redirect — it is a pipe, not a proxy, and strictly narrower than anything an allow-list could express. The bytes never touch a network, so there is no wire for TLS to protect, and Ollama stays bound to loopback rather than being exposed on the tailnet. The socket is bound only for a backend declared to use a local model, so a `local-ornith` VM reaches the forge through egress and nothing else, while every other backend's guest port simply refuses. Both halves have negative controls. `scripts/fleet-model-setup.sh` exists because of one measurement: stock ornith:9b reported input_tokens=2050 for a 48000-word prompt and answered as though nothing had been dropped. Ollama's default window is ~2K whatever the model card says, and it truncates silently — the exact failure an agent turn would hit and never report. The script pins num_ctx=131072 into a derived tag and then PROVES both the window and tool calling before declaring success. Verified on architect: ~65536 words -> 65604 input tokens, stop_reason=tool_use. Placement needs no new capability key: building the rootfs only on GPU nodes means `nodes::online_for_backend`'s existing `rootfs @> ["local-ornith"]` predicate does the affinity, so morpheus never offers the backend. Co-Authored-By: Claude Opus 5 <[email protected]>
56 lines
2.9 KiB
Docker
56 lines
2.9 KiB
Docker
# Claude Code pointed at a model running on the NODE ITSELF.
|
|
#
|
|
# The third variation on one idea: `agent-claude` talks to Anthropic,
|
|
# `agent-glm` to z.ai, `agent-kimi` to Moonshot, and this one to the Ollama
|
|
# already installed on every GPU node. Ollama has served a native
|
|
# Anthropic-compatible `/v1/messages` since v0.14, so this costs an env contract
|
|
# rather than a translation layer — MEASURED on both nodes: a tools request
|
|
# comes back with a well-formed `tool_use` block and `stop_reason=tool_use`.
|
|
#
|
|
# Why the base URL is loopback INSIDE the guest, not a hostname:
|
|
#
|
|
# The VM has no network interface. `fcagent` listens on 127.0.0.1:11434 and
|
|
# pumps to vsock 9003, where `clawmates-node::local_model` splices it onto the
|
|
# node's own `127.0.0.1:11434`. There is no destination anywhere in that path —
|
|
# it is a pipe to a fixed address, so a compromised guest cannot redirect it.
|
|
# `NO_PROXY` already contains 127.0.0.1, so this bypasses the egress proxy
|
|
# rather than trying to CONNECT through it.
|
|
#
|
|
# Build (on a node with a GPU — architect or tank):
|
|
#
|
|
# ssh architect "cd ~/clawmates && \
|
|
# docker build -f images/agent-ornith/Dockerfile -t clawmates/agent-ornith:dev images/agent-ornith/"
|
|
# scripts/fc-build-rootfs.sh architect clawmates/agent-ornith:dev local-ornith 8G
|
|
#
|
|
# Building it ONLY on GPU nodes is deliberate and is the whole placement story:
|
|
# a node advertises `rootfs-local-ornith.ext4` in `capabilities.rootfs`, and
|
|
# `nodes::online_for_backend` already filters on exactly that. No GPU capability
|
|
# key, no migration — morpheus simply never offers this backend.
|
|
FROM clawmates/agent-toolchain:dev
|
|
|
|
# Pinned to the SAME version as agent-claude and agent-glm. A run that differs
|
|
# by provider should differ by nothing else, or "the local backend behaved
|
|
# differently" is ambiguous between the model and the harness.
|
|
ARG CLAUDE_CODE_VERSION=2.1.223
|
|
RUN npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" \
|
|
&& npm cache clean --force \
|
|
&& rm -rf /root/.npm \
|
|
&& claude --version
|
|
|
|
# `ornith-fleet:9b`, NOT `ornith:9b`. Ollama defaults to a ~2K context window
|
|
# whatever the model card says, and it truncates silently: MEASURED, stock
|
|
# ornith:9b reported input_tokens=2050 for a 48000-word prompt and answered as
|
|
# though nothing had been dropped. The fleet tag pins num_ctx=131072 and is
|
|
# created by `scripts/fleet-model-setup.sh`. An agent turn is exactly the
|
|
# workload that would hit the default and never say so.
|
|
ENV HOME=/root \
|
|
CLAWMATES_AGENT_CLI=claude \
|
|
ANTHROPIC_BASE_URL=http://127.0.0.1:11434 \
|
|
ANTHROPIC_MODEL=ornith-fleet:9b \
|
|
ANTHROPIC_SMALL_FAST_MODEL=ornith-fleet:9b
|
|
RUN mkdir -p /root/.claude
|
|
|
|
# No provider key of any kind. `microvm_credential_for` hands this backend a
|
|
# literal placeholder because Ollama ignores the bearer and Claude Code refuses
|
|
# to start without one. There is no secret in this image and none reaches it.
|