Files
apress/deploy/uno-q/README.md
T
Omar SobhandClaude Opus 4.8 a23a525aef docs(uno-q): measure on-board inference; retire the LiteRT spike
Replaces the never-executed "LiteRT-LM on UNO Q 4GB" spike plan with a
measured record of the local-fallback path we actually run.

Measured on board 65301572 (Qwen2.5-0.5B-Instruct, -c 8192, 4 threads),
using llama-server's own timings rather than wall clock:

  prefill ~17-20 tok/s (linear), decode ~6-11 tok/s (degrades with KV)
  warm prefix-cached tool call: 3.8s, 6/6 correct structured calls

Two findings that changed the deployment:

1. The board had drifted onto Qwen2.5-Coder-1.5B - larger and tuned for
   the wrong task. Reverting to the repo's 0.5B made tool calls ~6x
   faster (24s -> 3.8s) and freed ~700MB. The repo was right.

2. The harness, not the model, was the bottleneck. The default agent
   profile sent a 4718-token prompt (~4.6 min prefill) and the client
   cancelled before the model could answer. A lean runtime profile cuts
   that to 706 tokens, lifts prefix-cache match 0.435 -> 0.966, and
   completes a full agentic turn with a real tool call in 11s warm.
   The ZeroClaw text parser was never at fault.

Prompt cost model for budgeting profiles: ~706 base (1 tool),
~244/additional tool, +315 for uno_q_flash (schema + flash imperative),
~53/skill in compact mode.

Also standardises context on -c 8192 across all three provisioning paths
(a 16k window costs ~16 min to fill at this speed and doubles KV for
nothing), and fixes stale references to the deleted src/lib/harness.ts.

Adds bench-prefill.sh and bench-tools.py as reproducible baselines.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-07-19 15:39:10 -07:00

16 KiB

Uno Q workshop node — provisioning

Turns an Arduino Uno Q into an APESS workshop node: a ZeroClaw daemon that exposes an HTTP/SSE gateway, drives the on-board MCU (generate → compile → flash), and answers with cloud-first / on-board-Qwen-fallback inference. APESS proxies each board and streams its activity to participants and instructors.

What a provisioned board runs

  • zeroclaw daemon on :8080 — the gateway APESS talks to (/pair, /webhook, /ws/chat, /api/events, /health). Use daemon, not gateway start — only the daemon/agent/channel paths register the hardware tools (uno_q_flash, …).
  • llama-server on :8083 — on-board Qwen for offline/fallback inference.
  • zeroclaw-reload-watcher (loopback) — applies a team's browser config edits (e.g. enabling their Telegram bot) via /admin/reload. See modalities.
  • Four agents, selected per request via ?agent= (callers pass the alias explicitly — see sendPrompt(teamId, message, agent?) in src/lib/api.ts):
    alias provider behaviour
    default custom.claude cloud, fallback to on-board Qwen
    cloud custom.cloud cloud only
    local llamacpp.local on-board Qwen only (fully offline)
    chaos custom.dead dead endpoint → always fails over to Qwen (outage demo)

Talking to the node — the three modalities

Beyond the APESS-proxied prompt path, a team reaches its own node three ways. All bind to the default (cloud-first) agent, so responses are fast; local stays the offline fallback.

  1. Web chat by IP. The board is LAN-open during setup (config.template.toml sets host = "0.0.0.0", allow_public_bind = true, require_pairing = false), so a team opens http://<board-lan-ip>:8080/ — the embedded ZeroClaw dashboard + /ws/chat — straight from the "Open your node →" link APESS shows after they claim. No token while open.
  2. Telegram. Each board is its own agent, so each needs its own bot. A team makes one via @BotFather, then in the dashboard (Config → channels → telegram.default) pastes the bot_token, adds their username to allowed_users, and sets enabled = true. Dashboard writes only set pending_reload; the zeroclaw-reload-watcher (loopback) applies them within a few seconds — no shell. The channel is pre-bound to default.
  3. Voice (browser-mic streaming). Talk in the web chat, the agent speaks back — requires a daemon built with the voice feature (see Voice below), then uncomment [channels.voice_duplex.default] in the config.

Open → locked lifecycle

Boards boot open so setup is frictionless on the isolated workshop LAN. When a team is done, harden the node with zeroclaw-lockdown.sh (run on the board or over adb — pair-code minting + reload are loopback-only, so it can't be a LAN button):

adb -s <serial> shell '/home/arduino/zeroclaw-lockdown.sh'   # prints a pair code

It flips require_pairing = true, reloads, and mints a pair code the team enters once at http://<board-ip>:8080/pairing. After lockdown only their paired devices reach the node. (Note: locking down also cuts the shared APESS proxy's access unless you re-register a fresh token — a deliberate "graduate to a private node" transition; validate against your run's needs.)

Voice — building a voice-capable board binary

gateway-voice-duplex is a Cargo feature and the SPA is embedded at compile time, so the board binary must be built as:

cargo xtask web build      # build the SPA into web/dist first (embedded-web needs it)
cargo build --release --target aarch64-unknown-linux-gnu \
  --features "hardware,peripheral-rpi,embedded-web,zeroclaw-gateway/gateway-voice-duplex"

gateway-voice-duplex has no root-crate alias, so it's namespaced. The web chat does client-side STT (Web Speech API) and plays the streamed tts_chunk audio; the server handle_voice_event path is finished as part of this work. This is the one modality that needs an on-board build + hardware test to verify.

Default agent skills

Every node ships with a bundle of UNO Q skills (skills/) installed into every agent's workspace, so the agent knows this board's specifics instead of guessing generic Arduino. Two complementary sets:

  • arduino-uno-q — one comprehensive reference skill (SKILL.md + 7 references/*.md): the dual-brain mental model, App Lab vs IDE, remote access, the Bridge/RPC deep-dive, full pin tables, wireless, and a workshop playbook. Best for the cloud agent via read_skill → references.
  • The granular set (bridge, flashing, led-matrix, uno-q-hardware, sketch-patterns, modulino, linux-led, audio, vision, wireless, arduino-app-lab) — small, keyword-triggered skills. The fork's eager skill-inliner has rules for these names, so the on-board Qwen auto-inlines them (no read_skill round-trip). flashing + led-matrix carry the exact uno_q_flash + frame-API / ArduinoGraphics-not-installed detail.

It must live in each agent's own workspace, not a shared bundle. ZeroClaw's read_skill returns only SKILL.md; the agent reads a references/*.md on demand via the workspace-sandboxed file_read tool, so the references are only reachable when the skill sits under ~/.zeroclaw/agents/<alias>/workspace/skills/. push-skill.sh installs it into every agent workspace; provision-fleet.sh does this per board. The risk profile (config.template.toml) auto-approves read_skill + file_read so agents load it without a human approver.

./push-skill.sh <serial>                       # install the default skill into every agent
./push-skill.sh <serial> skills/<other-skill>  # any SKILL.md + references/ skill
# then restart the daemon so it re-discovers skills at agent construction

Notes: the skill NAME is the directory name (arduino-uno-q); keep description: single-line (ZeroClaw's frontmatter parser is a flat scanner, not full YAML); no symlinks/scripts inside a skill dir (the auditor rejects them). The fork's eager keyword→skill inliner has no rule for arduino-uno-q, so small local models load it via a read_skill round-trip rather than auto-inlining — fine for the cloud default; add a skill_preroute rule if you want the on-board Qwen to auto-inline it.

Prerequisites on the board

  • /home/arduino/zeroclaw — the ZeroClaw binary (aarch64), with the gateway-peripheral-registration fix (merged to osobh/zeroclaw main).
  • /home/arduino/llama/llama-server + /home/arduino/models/qwen.gguf.
  • Arduino Zephyr core arduino:zephyr:unoq (0.51.0) + OpenOCD for flashing, and ArduinoGraphics if using scroll text.

Provision (dev — over USB/adb)

export APESS_ADMIN_CODE=adm-xxxxxxxx          # instructor code
./provision-uno-q.sh <adb-serial> team-07 https://apess-api.redclaw.dev

Steps it runs: install config.template.toml (cloud endpoint substituted) → start llama-server → start zeroclaw daemon → pair for a bearer token → POST /nodes to APESS. Idempotent; re-run to re-provision.

Override the cloud endpoint (default is the local claude_shim on :8090):

CLOUD_URI=https://api.anthropic.com/v1 CLOUD_MODEL=claude-haiku-4-5 \
  APESS_ADMIN_CODE=adm-xxxx ./provision-uno-q.sh <serial> team-07

The shim on :8090 is a bring-up convenience (one Mac). A real fleet points CLOUD_URI at a shared cloud endpoint (Anthropic / OpenRouter / LiteLLM) with a key, so boards don't each need a tunnel.

Onboarding — attendee self-serve (self-register + claim)

The workshop path: boards are preloaded and, on boot, announce themselves into APESS's unclaimed pool; an attendee then claims their board to their team from the web app (kit + 6-digit code) — no operator, no admin code, no adb.

board boot ──self-register {kitId,url,token}──▶ APESS unclaimed pool
                                                        ▲
attendee: scan kit QR → team name → 6-digit code ──POST /claim──┘
             (bearer token moves pool → node bridge, never touches the browser)

Fleet prep (host, once). Mint per-kit codes + per-board env files + the QR sticker CSV:

FLEET_SECRET=<shared-secret> APESS_URL=https://apess-api.redclaw.dev \
  APESS_WEB=https://apess.redclaw.dev ./gen-kit-codes.sh 15
# → kit-codes/env/KIT-NN.env  (one per board)   kit-codes/kit-codes.csv (stickers)

The same FLEET_SECRET must be set on the APESS API (FLEET_SECRET env) — it gates /nodes/self-register so only your boards can seed the pool. Each sticker QR encodes …/workshop?kit=KIT-NN&code=NNNNNN, so scanning it pre-fills both.

All boards at once (recommended). Map each kit to its board's adb serial in fleet.csv (see fleet.csv.example), then provision the fleet in one command — per board it pushes the env + self-register script, enables the boot/refresh timer (root) or an equivalent cron (MODE=cron, the default, no root), and flashes the default boot animation (sketches/matrix_rain) onto the MCU:

cp fleet.csv.example fleet.csv    # fill in kit,serial for each board
./provision-fleet.sh fleet.csv    # MODE=systemd for root boards; FLASH_DEFAULT=0 to skip the animation

One board (manual equivalent):

adb -s <serial> push kit-codes/env/KIT-07.env /home/arduino/.zeroclaw/apess-node.env
adb -s <serial> push apess-selfregister.sh /home/arduino/
adb -s <serial> shell 'chmod +x /home/arduino/apess-selfregister.sh'
# with root: enable the boot + refresh timer
sudo cp systemd/apess-selfregister.service systemd/apess-selfregister.timer /etc/systemd/system/
sudo systemctl enable --now apess-selfregister.timer
# no root: run it once now (and let cron re-run it)
/home/arduino/apess-selfregister.sh

Print the claim stickers. Turn the sticker CSV into a self-contained, printable QR sheet (QRs baked in as inline SVG — no network needed to print):

brew install qrencode          # one-time (build-time only)
./gen-qr-sheet.sh              # kit-codes/kit-codes.csv → kit-codes/qr-sheet.html
open kit-codes/qr-sheet.html   # print A4 at 100%; dashed borders are cut lines

provision-uno-q.sh (admin-driven, binds a board straight to a known team) is still there for pre-provisioning / demo boards — the paths coexist.

Provision (production — systemd, boots on power-up)

With root on the board:

sudo cp systemd/zeroclaw-llama.service systemd/zeroclaw-daemon.service /etc/systemd/system/
sudo systemctl enable --now zeroclaw-llama zeroclaw-daemon

For a LAN fleet (participants reach the board's WiFi IP directly), the template now ships this by default (host = "0.0.0.0", allow_public_bind = true, require_pairing = false) so the dashboard/web-chat is reachable + open during setup — see modalities above. Enable the reload-watcher alongside the daemon:

sudo cp systemd/zeroclaw-reload-watcher.service /etc/systemd/system/
sudo systemctl enable --now zeroclaw-reload-watcher

Then register once so APESS has the board's { url, token } (step 5; under open-boot the token is a placeholder — the open board ignores auth).

Resilience — surviving a disconnect (no-root boards)

A sudden USB/adb drop breaks things that don't self-heal: the adb tunnels vanish (board loses the cloud shim), held-shell services die, llama can wedge (process alive but :8083 dead), and flashes silently stop landing while the tool still reports success. The MCU keeps its last sketch; the paired token survives.

The systemd units above are the clean answer when you have root. Some dev boards don't — an expired account blocks sudo and there's no user session bus, so neither system nor user units can run. For those, use the no-root pieces:

  • zeroclaw-supervisor.sh (runs on the board) — a watchdog that polls the /health endpoints (a wedged process passes pgrep but fails here) and restarts llama / the daemon when they die or wedge. Children are launched with setsid … exec so they survive the shell that started them — the property plain nohup … & inside adb shell does not give you. Install + persist:

    adb -s <serial> push zeroclaw-supervisor.sh /home/arduino/ && \
      adb -s <serial> shell 'chmod +x /home/arduino/zeroclaw-supervisor.sh; \
        setsid nohup /home/arduino/zeroclaw-supervisor.sh >/dev/null 2>&1 </dev/null &'
    # boot persistence (no root; cron must be running):
    adb -s <serial> shell '(crontab -l 2>/dev/null | grep -v zeroclaw-supervisor.sh; \
      echo "@reboot /home/arduino/zeroclaw-supervisor.sh") | crontab -'
    
  • recover-uno-q.sh (runs on the host) — after the board is physically back, re-does adb + both tunnels, ensures the supervisor is up, and health-checks every hop by endpoint:

    ./recover-uno-q.sh <serial> [cloud-shim-port]   # default shim port 8090
    

See the unoq-disconnect-recovery note for the full failure-mode list.

Verify

adb -s <serial> forward tcp:8080 tcp:8080
curl -s localhost:8080/health                       # {"status":"ok"}
# each agent alias resolves (needs a bearer token from /pair):
curl -s -X POST 'localhost:8080/webhook?agent=local'  -H "authorization: Bearer $TOK" \
  -H 'content-type: application/json' -d '{"message":"one word: local"}'

?agent=local routes to Qwen (offline), ?agent=cloud to the cloud, ?agent=default cloud-with-fallback. In APESS, the team's provider/fallback toggle picks the alias.

Files

  • config.template.toml — the node config (secrets stripped; __CLOUD_URI__ / __CLOUD_MODEL__ substituted at provision time).
  • provision-uno-q.sh — one-shot provisioner (adb-driven, binds to a known team).
  • gen-kit-codes.sh — host: mint per-kit claim codes + per-board env files + sticker CSV.
  • provision-fleet.sh — host: provision a whole fleet from fleet.csv (env + self-register + timer/cron).
  • fleet.csv.example — kit→adb-serial assignment template for provision-fleet.sh.
  • gen-qr-sheet.sh — host: render the sticker CSV into a self-contained printable QR sheet (needs qrencode).
  • flash-sketch.sh — host: compile + flash a sketch onto a board's MCU (sets the boot animation; default sketches/matrix_rain).
  • sketches/ — LED-matrix animations (matrix_rain is the board default; matrix_effects is a sampler). See sketches/README.md.
  • apess-selfregister.sh — on-board: announce into APESS's unclaimed pool on boot.
  • apess-node.env.example — per-board onboarding identity (KIT_ID / CLAIM_CODE / FLEET_SECRET / APESS_URL).
  • systemd/*.service + apess-selfregister.timer — production units (need root).
  • zeroclaw-supervisor.sh — on-board no-root watchdog (endpoint health + restart).
  • zeroclaw-reload-watcher.sh — on-board: applies browser dashboard config edits by firing the loopback /admin/reload when config.toml changes (makes Telegram-enable / lockdown self-serve without a shell). Root unit: systemd/zeroclaw-reload-watcher.service; no-root: launch via setsid nohup.
  • zeroclaw-lockdown.sh — on-board: the harden step — flips require_pairing=true, reloads, and mints a pair code (open → locked).
  • push-skill.sh — host: install a skill (or a whole skills dir) into every agent's workspace on a board. provision-fleet.sh runs it per board over skills/.
  • skills/ — the bundled UNO Q skills (comprehensive arduino-uno-q + the granular set). Installed per-agent so each node has them by default.
  • recover-uno-q.sh — host-side post-disconnect recovery (re-tunnel + health-check).