Measured on the board 2026-07-20. Offline is limited by capability, not only
by speed:
simple tool call (i2cdetect, 708 tok) -> works, 20s, real answer
write+compile+flash a sketch (1997 tok) -> never completed in 450s, two
identical requests, no tool call
So with no network the node can sense and decide, but cannot author new
code - that needs the cloud model. This is the concrete degradation boundary
students are asked to state in ADD Layer 4: what survives an outage is the
loop over already-flashed firmware, not writing new firmware.
Also corrects the cold-load figure. The documented 3-5 minutes was measured
against the 1.1GB coder model; the 409MB qwen.gguf we standardised on came
up healthy in ~5s from a cold boot.
Notes that BuildFlash routes to the cloud agent, so the student build/flash
exercise does not sit behind this boundary.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Replaces the never-executed "LiteRT-LM on UNO Q 4GB" spike plan with a
measured record of the local-fallback path we actually run.
Measured on board 65301572 (Qwen2.5-0.5B-Instruct, -c 8192, 4 threads),
using llama-server's own timings rather than wall clock:
prefill ~17-20 tok/s (linear), decode ~6-11 tok/s (degrades with KV)
warm prefix-cached tool call: 3.8s, 6/6 correct structured calls
Two findings that changed the deployment:
1. The board had drifted onto Qwen2.5-Coder-1.5B - larger and tuned for
the wrong task. Reverting to the repo's 0.5B made tool calls ~6x
faster (24s -> 3.8s) and freed ~700MB. The repo was right.
2. The harness, not the model, was the bottleneck. The default agent
profile sent a 4718-token prompt (~4.6 min prefill) and the client
cancelled before the model could answer. A lean runtime profile cuts
that to 706 tokens, lifts prefix-cache match 0.435 -> 0.966, and
completes a full agentic turn with a real tool call in 11s warm.
The ZeroClaw text parser was never at fault.
Prompt cost model for budgeting profiles: ~706 base (1 tool),
~244/additional tool, +315 for uno_q_flash (schema + flash imperative),
~53/skill in compact mode.
Also standardises context on -c 8192 across all three provisioning paths
(a 16k window costs ~16 min to fill at this speed and doubles KV for
nothing), and fixes stale references to the deleted src/lib/harness.ts.
Adds bench-prefill.sh and bench-tools.py as reproducible baselines.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
