test(fleet): a mission served entirely by the node's own GPU

`local-ornith` scenario, green on its first real run against tank:

  local-ornith: a locally-served model delivered a guest kernel (6.1.128)
  local-ornith: no Anthropic egress from a locally-served mission
  local-ornith: the node bound its local-model socket for this VM
  local-ornith: checkout has exactly one writer (uid=65532)

Three things had to be true at once and only a real run shows all three: the
agent reached a model at all (a pipe to a closed port produces a turn that HANGS
rather than errors, which is why this is a scenario and not a unit test), the
work came back and landed on a branch, and the VM still could not reach
api.anthropic.com.

That last one is not theoretical. The node log for this VM is a column of
`egress DENIED api.anthropic.com` — Claude Code's own telemetry, correctly
refused — while the model traffic went through the vsock pipe and Ollama logged
loading ornith-fleet:9b at 100% GPU with CONTEXT 131072. A local backend that
quietly kept Anthropic egress would be a credential path nobody asked for.

The egress check asks the NODE's proxy log rather than the agent, for the same
reason the GLM measurement did: a model's account of where its tokens came from
has no evidential value, and the proxy's record of what it dialled does.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-09 13:21:57 -07:00
co-authored by Claude Opus 5
parent f56d41f5b7
commit 774f17d194
+79 -1
View File
@@ -991,6 +991,81 @@ exercise the guard (agent ignored the instruction; re-run)"; continue ;;
return 0 return 0
} }
# ── Scenario: a mission served by the node's OWN GPU ──────────────
#
# `local-ornith` is Claude Code pointed at the Ollama on the node itself, over a
# vsock pipe rather than the egress proxy (`clawmates-node::local_model`). Three
# things have to be true at once and only a real run shows all three:
#
# 1. the agent reached a model AT ALL — a pipe to a closed port produces a
# turn that hangs rather than errors, which is why this is a scenario and
# not a unit test;
# 2. the work came back and landed on a branch, so the local model actually
# drove the CLI rather than merely answering;
# 3. the VM still could NOT reach api.anthropic.com. That is the measured
# property of every other backend and the reason `provider_hosts` is
# per-backend; a local backend that quietly kept Anthropic egress would be
# a credential path nobody asked for.
LOCAL_BODY=$(cat <<JSON
{"title":"verify: a mission on the node's own GPU",
"template_kind":"research_and_code",
"repo_id":"$REPO_ID",
"runtime_kind":"microvm",
"backend":"local-ornith",
"description":"Prove a coding phase runs against a locally-hosted model.",
"phases":[
{"kind":"coding","order_idx":0,"config":{"commit_policy":"always","max_iterations":1,
"done_when":"LOCAL.md exists at the repository root and contains a kernel version such as 6.1.128.",
"task":"Create a file named LOCAL.md at the repository root containing exactly one line: the output of running uname -r. Create no other files."}}
]}
JSON
)
assert_local() { # <token> <mission> <report>
local token="$1" mission="$2" report="$3" delivered node
while read -r idx status files pushed _branch cerr perr; do
[ "$status" = "completed" ] || fail "local-ornith: phase $idx status=$status"
[ "$pushed" = "True" ] || fail "local-ornith: phase $idx not pushed (commit_error=$cerr push_error=$perr)"
case "$files" in 0|-) fail "local-ornith: phase $idx delivered no files" ;; esac
done <<<"$report"
delivered=$(fetch_delivered "$token" "$mission" LOCAL.md) \
|| { fail "local-ornith: could not read LOCAL.md from the pushed branch"; return 1; }
# A GUEST kernel, so this also proves it ran in a VM rather than on a host.
case "$(printf '%s' "$delivered" | tr -d '[:space:]')" in
"$GW_KERNEL"|"$NODE_KERNEL")
fail "local-ornith: LOCAL.md holds a HOST kernel ($delivered) — that phase did not run in a guest" ;;
*[0-9].[0-9]*)
pass "local-ornith: a locally-served model delivered a guest kernel ($(printf '%s' "$delivered" | tr -d '[:space:]'))" ;;
*) fail "local-ornith: LOCAL.md is not a kernel version: $delivered" ;;
esac
# The negative control. Asked of the NODE's proxy log, which is the only
# record of what was actually dialled — a model's own account of where it got
# its tokens has no evidential value here.
node=$(ssh "$HOST" "docker exec clawmates_postgres_1 psql -U postgres -d clawmates -tAc \
\"select coalesce(n.name,'') from missions m left join nodes n on n.id = m.target_node_id where m.id='$mission';\"" \
| head -1 | tr -d '[:space:]')
if [ -z "$node" ]; then
norun "local-ornith: could not tell which node ran it — egress cannot be checked"
else
local target; case "$node" in tank) target=osobh@tank ;; *) target="$node" ;; esac
if ssh "$target" "journalctl -u clawmates-node --since '-30 min' --no-pager 2>/dev/null \
| grep -q 'egress -> api.anthropic.com'"; then
fail "local-ornith: the VM reached api.anthropic.com — a local backend must not"
else
pass "local-ornith: no Anthropic egress from a locally-served mission"
fi
if ssh "$target" "journalctl -u clawmates-node --since '-30 min' --no-pager 2>/dev/null \
| grep -q 'local model socket'"; then
pass "local-ornith: the node bound its local-model socket for this VM"
else
fail "local-ornith: the node never bound a local-model socket — the guest had nothing to talk to"
fi
fi
}
# ── Scenario: a burst larger than the fleet QUEUES, and spreads ─── # ── Scenario: a burst larger than the fleet QUEUES, and spreads ───
# #
# Phase 1 of the fleet-intelligence plan shipped placement-at-phase-launch and # Phase 1 of the fleet-intelligence plan shipped placement-at-phase-launch and
@@ -1410,6 +1485,9 @@ case "${1:-all}" in
roster) roster)
scenario_roster scenario_roster
;; ;;
local-ornith)
run_scenario local-ornith "$(echo "$LOCAL_BODY" | tr -d '\n')" assert_local
;;
capacity) capacity)
scenario_capacity scenario_capacity
;; ;;
@@ -1435,7 +1513,7 @@ case "${1:-all}" in
scenario_drain_midmission scenario_drain_midmission
;; ;;
*) *)
die "unknown scenario: $1 (selftest|uids|chain|multirole|noop|microvm|canary|gatecap|research-only|benchmark|security|refactor|composed|roster|capacity|drain-midmission|all)" die "unknown scenario: $1 (selftest|uids|chain|multirole|noop|microvm|canary|gatecap|research-only|benchmark|security|refactor|composed|roster|local-ornith|capacity|drain-midmission|all)"
;; ;;
esac esac