docs(uno-q): record the offline capability boundary
Measured on the board 2026-07-20. Offline is limited by capability, not only
by speed:
simple tool call (i2cdetect, 708 tok) -> works, 20s, real answer
write+compile+flash a sketch (1997 tok) -> never completed in 450s, two
identical requests, no tool call
So with no network the node can sense and decide, but cannot author new
code - that needs the cloud model. This is the concrete degradation boundary
students are asked to state in ADD Layer 4: what survives an outage is the
loop over already-flashed firmware, not writing new firmware.
Also corrects the cold-load figure. The documented 3-5 minutes was measured
against the 1.1GB coder model; the 409MB qwen.gguf we standardised on came
up healthy in ~5s from a cold boot.
Notes that BuildFlash routes to the cloud agent, so the student build/flash
exercise does not sit behind this boundary.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
e27fad46e2
commit
37a54521e8
@@ -77,8 +77,11 @@ where systemd isn't usable. Two properties matter:
|
|||||||
|
|
||||||
- It polls **`/health` endpoints, not `pgrep`** — a wedged process passes `pgrep` but
|
- It polls **`/health` endpoints, not `pgrep`** — a wedged process passes `pgrep` but
|
||||||
fails a health check. Process liveness is not service liveness.
|
fails a health check. Process liveness is not service liveness.
|
||||||
- It respects a **warmup grace** (`LLAMA_WARMUP=300`). A cold GGUF load takes **3–5
|
- It respects a **warmup grace** (`LLAMA_WARMUP=300`); reaping mid-load produces an
|
||||||
minutes**; reaping mid-load produces an infinite restart loop that never converges.
|
infinite restart loop that never converges. The 3–5 minute cold-load figure was measured
|
||||||
|
against the 1.1 GB coder model — the 409 MB `qwen.gguf` we standardised on came up
|
||||||
|
**healthy in ~5 s** from a cold boot (2026-07-20). The 300 s grace is now generous rather
|
||||||
|
than necessary, which is harmless.
|
||||||
|
|
||||||
Children start under `setsid` so they survive the launching shell closing — plain
|
Children start under `setsid` so they survive the launching shell closing — plain
|
||||||
`nohup … &` inside an `adb shell` does **not** give you that, which is why services
|
`nohup … &` inside an `adb shell` does **not** give you that, which is why services
|
||||||
@@ -187,6 +190,32 @@ Three ways to destroy that advantage — all easy to do by accident:
|
|||||||
- **`max_tool_iterations = 6`.** Each iteration is a full model call — ~3 min per task.
|
- **`max_tool_iterations = 6`.** Each iteration is a full model call — ~3 min per task.
|
||||||
Acceptable for a background loop, not for anything interactive.
|
Acceptable for a background loop, not for anything interactive.
|
||||||
|
|
||||||
|
### The offline capability boundary (measured 2026-07-20)
|
||||||
|
|
||||||
|
Speed is not the only limit — there is a hard capability cliff. Same board, same lean
|
||||||
|
profile, same on-board 0.5B:
|
||||||
|
|
||||||
|
| task | result |
|
||||||
|
|---|---|
|
||||||
|
| Call a simple tool (`i2cdetect`, 708-token prompt) | **works — 20 s**, tool call fired, real answer |
|
||||||
|
| Write + compile + flash a sketch (1,997-token prompt) | **never completed** — 450 s, two identical requests, no tool call ever emitted |
|
||||||
|
|
||||||
|
So offline the node can **sense and decide, but it cannot author new code**. Code
|
||||||
|
generation needs the cloud model. This is the concrete degradation boundary to state in
|
||||||
|
ADD Layer 4: what still works with no network is the sensing and decision loop over
|
||||||
|
*already-flashed* firmware — not writing new firmware.
|
||||||
|
|
||||||
|
Two related prompt-shape findings from the same session:
|
||||||
|
|
||||||
|
- **Imperative, not interrogative.** `"List the I2C devices on the bus."` fires a tool call
|
||||||
|
in 20 s; `"What sensors can you find on the I2C bus?"` produced **no tool call at all**
|
||||||
|
in 200 s. Same agent, same 708-token prompt — phrasing was the only variable.
|
||||||
|
- **It does not hallucinate hardware.** Asked to list I2C devices on a board with an empty
|
||||||
|
bus, it ran the tool and reported the bus numbers rather than inventing a sensor.
|
||||||
|
|
||||||
|
Note also that `BuildFlash` in the SPA routes to the **`cloud`** agent, not `local` — so the
|
||||||
|
student build/flash exercise does not depend on the boundary above.
|
||||||
|
|
||||||
### What this means for offline work
|
### What this means for offline work
|
||||||
|
|
||||||
On the **0.5B** (measured or extrapolated from the curve above):
|
On the **0.5B** (measured or extrapolated from the curve above):
|
||||||
|
|||||||
Reference in New Issue
Block a user