Omar SobhandClaude Opus 4.8
a23a525aef
Replaces the never-executed "LiteRT-LM on UNO Q 4GB" spike plan with a measured record of the local-fallback path we actually run. Measured on board 65301572 (Qwen2.5-0.5B-Instruct, -c 8192, 4 threads), using llama-server's own timings rather than wall clock: prefill ~17-20 tok/s (linear), decode ~6-11 tok/s (degrades with KV) warm prefix-cached tool call: 3.8s, 6/6 correct structured calls Two findings that changed the deployment: 1. The board had drifted onto Qwen2.5-Coder-1.5B - larger and tuned for the wrong task. Reverting to the repo's 0.5B made tool calls ~6x faster (24s -> 3.8s) and freed ~700MB. The repo was right. 2. The harness, not the model, was the bottleneck. The default agent profile sent a 4718-token prompt (~4.6 min prefill) and the client cancelled before the model could answer. A lean runtime profile cuts that to 706 tokens, lifts prefix-cache match 0.435 -> 0.966, and completes a full agentic turn with a real tool call in 11s warm. The ZeroClaw text parser was never at fault. Prompt cost model for budgeting profiles: ~706 base (1 tool), ~244/additional tool, +315 for uno_q_flash (schema + flash imperative), ~53/skill in compact mode. Also standardises context on -c 8192 across all three provisioning paths (a 16k window costs ~16 min to fill at this speed and doubles KV for nothing), and fixes stale references to the deleted src/lib/harness.ts. Adds bench-prefill.sh and bench-tools.py as reproducible baselines. Co-Authored-By: Claude Opus 4.8 <[email protected]>
59 lines
3.1 KiB
Bash
Executable File
59 lines
3.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Provision an Arduino Uno Q as an APESS workshop node — host-driven via adb,
|
|
# no root needed on the board. Installs config, brings up the on-board Qwen and
|
|
# the ZeroClaw daemon, pairs for a bearer token, and registers with APESS.
|
|
#
|
|
# APESS_ADMIN_CODE=adm-xxxx ./provision-uno-q.sh <adb-serial> <team-id> [apess-url]
|
|
#
|
|
# Env overrides: CLOUD_URI, CLOUD_MODEL (default: local shim :8090 / sonnet).
|
|
set -euo pipefail
|
|
|
|
SERIAL="${1:?usage: provision-uno-q.sh <adb-serial> <team-id> [apess-url]}"
|
|
TEAM="${2:?team id required}"
|
|
APESS="${3:-https://apess-api.redclaw.dev}"
|
|
CLOUD_URI="${CLOUD_URI:-http://127.0.0.1:8090/v1}"
|
|
CLOUD_MODEL="${CLOUD_MODEL:-sonnet}"
|
|
ADMIN_CODE="${APESS_ADMIN_CODE:?set APESS_ADMIN_CODE to register with APESS}"
|
|
|
|
HERE="$(cd "$(dirname "$0")" && pwd)"
|
|
a() { adb -s "$SERIAL" "$@"; }
|
|
|
|
echo "==> [1/5] install config (cloud=$CLOUD_URI)"
|
|
sed -e "s#__CLOUD_URI__#${CLOUD_URI}#g" -e "s#__CLOUD_MODEL__#${CLOUD_MODEL}#g" \
|
|
"$HERE/config.template.toml" > /tmp/apess-node-config.toml
|
|
a shell 'mkdir -p /home/arduino/.zeroclaw'
|
|
a push /tmp/apess-node-config.toml /home/arduino/.zeroclaw/config.toml >/dev/null
|
|
|
|
echo "==> [2/5] start on-board Qwen (llama-server :8083)"
|
|
a shell 'pgrep -f llama-server >/dev/null 2>&1 || (cd /home/arduino/llama && \
|
|
LD_LIBRARY_PATH=/home/arduino/llama nohup ./llama-server -m /home/arduino/models/qwen.gguf \
|
|
--host 127.0.0.1 --port 8083 -np 1 -c 8192 --jinja --mlock >/tmp/llama8083.log 2>&1 &)'
|
|
|
|
echo "==> [3/5] start zeroclaw daemon (:8080)"
|
|
a shell 'pkill -f "zeroclaw daemon" 2>/dev/null; pkill -f "zeroclaw gateway" 2>/dev/null; sleep 2; \
|
|
nohup /home/arduino/zeroclaw daemon >/tmp/zc-daemon.log 2>&1 &'
|
|
sleep 8
|
|
|
|
echo "==> [4/5] pair → bearer token"
|
|
a forward tcp:8080 tcp:8080 >/dev/null
|
|
until curl -sf --max-time 3 http://127.0.0.1:8080/health >/dev/null; do sleep 1; done
|
|
# Under LAN-open setup (require_pairing=false, the template default) the gateway
|
|
# mints no code — the open board ignores auth, so register a placeholder token.
|
|
CODE=$(a shell '/home/arduino/zeroclaw gateway get-paircode --new --port 8080' 2>/dev/null | grep -oE '[0-9]{6}' | head -1 || true)
|
|
TOKEN=""
|
|
if [ -n "$CODE" ]; then
|
|
TOKEN=$(curl -s -X POST http://127.0.0.1:8080/pair -H "X-Pairing-Code: ${CODE}" \
|
|
| python3 -c 'import sys,json;print(json.load(sys.stdin).get("token",""))' 2>/dev/null || true)
|
|
fi
|
|
TOKEN="${TOKEN:-open-lan}" # placeholder while the board is LAN-open
|
|
|
|
echo "==> [5/5] register team '$TEAM' with APESS"
|
|
# Board's LAN IP — what participants reach in a real fleet (falls back to localhost).
|
|
IP=$(a shell "ip -4 -o addr show 2>/dev/null | grep -oE 'inet [0-9.]+' | grep -v '127.0.0.1' | awk '{print \$2}' | head -1" | tr -d '\r')
|
|
URL="http://${IP:-127.0.0.1}:8080"
|
|
curl -sf -X POST "${APESS}/nodes" -H "x-access-code: ${ADMIN_CODE}" -H 'content-type: application/json' \
|
|
-d "{\"teamId\":\"${TEAM}\",\"url\":\"${URL}\",\"token\":\"${TOKEN}\"}" >/dev/null \
|
|
&& echo " registered at ${URL}" || echo " WARN: APESS registration failed (is ${APESS} reachable?)"
|
|
|
|
echo "==> done — node '$TEAM' is live. Agents: default (cloud+fallback), cloud, local."
|