test(fleet): a mission served entirely by the node's own GPU
`local-ornith` scenario, green on its first real run against tank: local-ornith: a locally-served model delivered a guest kernel (6.1.128) local-ornith: no Anthropic egress from a locally-served mission local-ornith: the node bound its local-model socket for this VM local-ornith: checkout has exactly one writer (uid=65532) Three things had to be true at once and only a real run shows all three: the agent reached a model at all (a pipe to a closed port produces a turn that HANGS rather than errors, which is why this is a scenario and not a unit test), the work came back and landed on a branch, and the VM still could not reach api.anthropic.com. That last one is not theoretical. The node log for this VM is a column of `egress DENIED api.anthropic.com` — Claude Code's own telemetry, correctly refused — while the model traffic went through the vsock pipe and Ollama logged loading ornith-fleet:9b at 100% GPU with CONTEXT 131072. A local backend that quietly kept Anthropic egress would be a credential path nobody asked for. The egress check asks the NODE's proxy log rather than the agent, for the same reason the GLM measurement did: a model's account of where its tokens came from has no evidential value, and the proxy's record of what it dialled does. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f56d41f5b7
commit
774f17d194
@@ -991,6 +991,81 @@ exercise the guard (agent ignored the instruction; re-run)"; continue ;;
|
|||||||
return 0
|
return 0
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# ── Scenario: a mission served by the node's OWN GPU ──────────────
|
||||||
|
#
|
||||||
|
# `local-ornith` is Claude Code pointed at the Ollama on the node itself, over a
|
||||||
|
# vsock pipe rather than the egress proxy (`clawmates-node::local_model`). Three
|
||||||
|
# things have to be true at once and only a real run shows all three:
|
||||||
|
#
|
||||||
|
# 1. the agent reached a model AT ALL — a pipe to a closed port produces a
|
||||||
|
# turn that hangs rather than errors, which is why this is a scenario and
|
||||||
|
# not a unit test;
|
||||||
|
# 2. the work came back and landed on a branch, so the local model actually
|
||||||
|
# drove the CLI rather than merely answering;
|
||||||
|
# 3. the VM still could NOT reach api.anthropic.com. That is the measured
|
||||||
|
# property of every other backend and the reason `provider_hosts` is
|
||||||
|
# per-backend; a local backend that quietly kept Anthropic egress would be
|
||||||
|
# a credential path nobody asked for.
|
||||||
|
|
||||||
|
LOCAL_BODY=$(cat <<JSON
|
||||||
|
{"title":"verify: a mission on the node's own GPU",
|
||||||
|
"template_kind":"research_and_code",
|
||||||
|
"repo_id":"$REPO_ID",
|
||||||
|
"runtime_kind":"microvm",
|
||||||
|
"backend":"local-ornith",
|
||||||
|
"description":"Prove a coding phase runs against a locally-hosted model.",
|
||||||
|
"phases":[
|
||||||
|
{"kind":"coding","order_idx":0,"config":{"commit_policy":"always","max_iterations":1,
|
||||||
|
"done_when":"LOCAL.md exists at the repository root and contains a kernel version such as 6.1.128.",
|
||||||
|
"task":"Create a file named LOCAL.md at the repository root containing exactly one line: the output of running uname -r. Create no other files."}}
|
||||||
|
]}
|
||||||
|
JSON
|
||||||
|
)
|
||||||
|
|
||||||
|
assert_local() { # <token> <mission> <report>
|
||||||
|
local token="$1" mission="$2" report="$3" delivered node
|
||||||
|
while read -r idx status files pushed _branch cerr perr; do
|
||||||
|
[ "$status" = "completed" ] || fail "local-ornith: phase $idx status=$status"
|
||||||
|
[ "$pushed" = "True" ] || fail "local-ornith: phase $idx not pushed (commit_error=$cerr push_error=$perr)"
|
||||||
|
case "$files" in 0|-) fail "local-ornith: phase $idx delivered no files" ;; esac
|
||||||
|
done <<<"$report"
|
||||||
|
|
||||||
|
delivered=$(fetch_delivered "$token" "$mission" LOCAL.md) \
|
||||||
|
|| { fail "local-ornith: could not read LOCAL.md from the pushed branch"; return 1; }
|
||||||
|
# A GUEST kernel, so this also proves it ran in a VM rather than on a host.
|
||||||
|
case "$(printf '%s' "$delivered" | tr -d '[:space:]')" in
|
||||||
|
"$GW_KERNEL"|"$NODE_KERNEL")
|
||||||
|
fail "local-ornith: LOCAL.md holds a HOST kernel ($delivered) — that phase did not run in a guest" ;;
|
||||||
|
*[0-9].[0-9]*)
|
||||||
|
pass "local-ornith: a locally-served model delivered a guest kernel ($(printf '%s' "$delivered" | tr -d '[:space:]'))" ;;
|
||||||
|
*) fail "local-ornith: LOCAL.md is not a kernel version: $delivered" ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# The negative control. Asked of the NODE's proxy log, which is the only
|
||||||
|
# record of what was actually dialled — a model's own account of where it got
|
||||||
|
# its tokens has no evidential value here.
|
||||||
|
node=$(ssh "$HOST" "docker exec clawmates_postgres_1 psql -U postgres -d clawmates -tAc \
|
||||||
|
\"select coalesce(n.name,'') from missions m left join nodes n on n.id = m.target_node_id where m.id='$mission';\"" \
|
||||||
|
| head -1 | tr -d '[:space:]')
|
||||||
|
if [ -z "$node" ]; then
|
||||||
|
norun "local-ornith: could not tell which node ran it — egress cannot be checked"
|
||||||
|
else
|
||||||
|
local target; case "$node" in tank) target=osobh@tank ;; *) target="$node" ;; esac
|
||||||
|
if ssh "$target" "journalctl -u clawmates-node --since '-30 min' --no-pager 2>/dev/null \
|
||||||
|
| grep -q 'egress -> api.anthropic.com'"; then
|
||||||
|
fail "local-ornith: the VM reached api.anthropic.com — a local backend must not"
|
||||||
|
else
|
||||||
|
pass "local-ornith: no Anthropic egress from a locally-served mission"
|
||||||
|
fi
|
||||||
|
if ssh "$target" "journalctl -u clawmates-node --since '-30 min' --no-pager 2>/dev/null \
|
||||||
|
| grep -q 'local model socket'"; then
|
||||||
|
pass "local-ornith: the node bound its local-model socket for this VM"
|
||||||
|
else
|
||||||
|
fail "local-ornith: the node never bound a local-model socket — the guest had nothing to talk to"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
# ── Scenario: a burst larger than the fleet QUEUES, and spreads ───
|
# ── Scenario: a burst larger than the fleet QUEUES, and spreads ───
|
||||||
#
|
#
|
||||||
# Phase 1 of the fleet-intelligence plan shipped placement-at-phase-launch and
|
# Phase 1 of the fleet-intelligence plan shipped placement-at-phase-launch and
|
||||||
@@ -1410,6 +1485,9 @@ case "${1:-all}" in
|
|||||||
roster)
|
roster)
|
||||||
scenario_roster
|
scenario_roster
|
||||||
;;
|
;;
|
||||||
|
local-ornith)
|
||||||
|
run_scenario local-ornith "$(echo "$LOCAL_BODY" | tr -d '\n')" assert_local
|
||||||
|
;;
|
||||||
capacity)
|
capacity)
|
||||||
scenario_capacity
|
scenario_capacity
|
||||||
;;
|
;;
|
||||||
@@ -1435,7 +1513,7 @@ case "${1:-all}" in
|
|||||||
scenario_drain_midmission
|
scenario_drain_midmission
|
||||||
;;
|
;;
|
||||||
*)
|
*)
|
||||||
die "unknown scenario: $1 (selftest|uids|chain|multirole|noop|microvm|canary|gatecap|research-only|benchmark|security|refactor|composed|roster|capacity|drain-midmission|all)"
|
die "unknown scenario: $1 (selftest|uids|chain|multirole|noop|microvm|canary|gatecap|research-only|benchmark|security|refactor|composed|roster|local-ornith|capacity|drain-midmission|all)"
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user