clawmates/handoff/AGENT_HANDOFF.md

Clawmates Visualization Layer — Agent Setup & Real-Time Wiring Handoff

Audience: an autonomous build agent (or a human operator) provisioning a Linux node. Goal: install everything required to serve the Clawmates visualizations and feed them live from the durable runner so the World / Observe / Dashboard views update in real time as the claws actually work.

Read this file top-to-bottom once before running anything. Every command is idempotent and safe to re-run. Where a step touches the existing Clawmates platform (the Rust workspace + Next.js app), it is clearly marked [INTEGRATES WITH cm-api]; you can also run the whole layer in demo mode with synthetic events and zero backend.

---

0. What you are installing

Three browser-native visualizations plus a thin real-time bridge:

Asset	What it is	Lives in
Clawmates World	The Large World map with three formations — Hierarchy (org▸company▸team▸claw tree), Flat (peer mesh), and Live (Gource-style convergence on a <canvas>).	visualizations/Clawmates World.dc.html
Clawmates Observe	Two linked observation modes — Agent close-up (live reasoning stream + the agent's live computer screen) and System mission-control (both claws, comms, routines, telemetry).	visualizations/Clawmates Observe.dc.html
Clawmates Dashboard	The tiered admin dashboard (org→company→team→claw) with topology morph + agent anatomy.	visualizations/Clawmates Dashboard.dc.html
Live bridge	A Node service that subscribes to the runner's event stream, normalizes it to the Clawmates Event Taxonomy (§4), and re-emits one SSE feed the browsers consume.	realtime/server.mjs
Client adapter	window.ClawmatesLive — connects to the bridge, dispatches typed events, and exposes them to each visualization.	realtime/clawmates-live.js

The visualizations are .dc.html files: self-contained HTML that renders via the bundled support.js runtime. They open directly in a browser — no build step, no npm install to view them. The only thing that needs installing is (a) a static web server to serve them and (b) Node for the live bridge.

---

1. System requirements

• OS: Linux x86_64 (Debian/Ubuntu 22.04+ or any systemd distro). ARM64 works too.
• CPU/RAM: 2 vCPU / 2 GB is plenty; the canvas animation runs client-side in the browser.
• Node.js: v20 LTS or newer (for the live bridge only).
• A static file server: any of caddy, nginx, or npx serve. Caddy is recommended (auto-HTTPS, one binary).
• Outbound network: only needed to download packages during install. The visualizations and bridge run fully offline afterward — consistent with the §15 "sandboxes have no network" model.
• A modern browser to display: Chromium/Chrome 120+ or Firefox 121+ (Canvas2D + ES2020).

---

2. One-shot install

From the unzipped package root:

chmod +x install.sh
./install.sh

install.sh will:

1. Verify Node ≥ 20 (and print install instructions if missing — it will not silently install a runtime).
2. Install the bridge's dependencies (npm ci in realtime/, single dependency: none beyond Node built-ins — it's intentionally dependency-free).
3. Install Caddy if no static server is found (via the official apt repo; falls back to npx serve).
4. Write a .env from .env.example if one does not exist.
5. Print the exact next commands for demo mode and live mode.

If you prefer to do it by hand, the rest of this document is the manual path.

---

3. Run it

3a. Demo mode (no backend — synthetic events)

Best first run: proves the whole pipe lights up.

# Terminal 1 — the live bridge in demo mode (emits synthetic agent activity)
cd realtime
CLAWMATES_MODE=demo node server.mjs
# → Live bridge on http://localhost:8420/live  (SSE)   ·  mode=demo

# Terminal 2 — serve the visualizations
caddy file-server --root ../visualizations --listen :8080
#   (or:  npx serve ../visualizations -l 8080)

Open http://localhost:8080/Clawmates%20World.dc.html?live=http://localhost:8420/live

The ?live= query param tells the client adapter where the SSE feed is. Switch to the Live formation and you should see the agent particles converge on nodes driven by real events from the bridge, not the built-in synthetic loop. (Without ?live=, every visualization falls back to its self-contained demo animation, so it always looks alive even with nothing wired.)

3b. Live mode [INTEGRATES WITH cm-api]

Point the bridge at the durable runner's SSE gateway and give it a session token. The bridge swaps the session for a bearer token exactly like the Next.js /api proxy does, then streams.

cd realtime
cp ../.env.example ../.env    # then edit ../.env
CLAWMATES_MODE=live node server.mjs

Required .env values for live mode:

CLAWMATES_MODE=live
# Where the runner publishes its raw run/turn SSE stream (cm-api streaming gateway):
CM_API_SSE_URL=http://127.0.0.1:8080/v1/runs/stream
# Bearer token (or a session cookie the bridge will exchange):
CM_API_TOKEN=sk-...                 # service token scoped to read run events
# The port this bridge listens on for browsers:
LIVE_PORT=8420
# Comma-separated allowed origins for the browser SSE (CORS):
LIVE_ALLOWED_ORIGINS=http://localhost:8080

Security note (carry this through): the bridge is read-only on the runner stream. It must never hold or forward secret-broker credentials, and it must not expose any door-approval endpoint to the public browser. Approvals stay on the authenticated cm-api path. The bridge only renders that a door is pending (door.request) so the UI can badge it; the actual approve/deny call goes through your existing authenticated API, not through this bridge.

---

4. The Clawmates Event Taxonomy (the contract)

This is the heart of the real-time layer. The runner produces many internal events; the bridge normalizes them into this small, stable set. The browser only ever sees these. Full JSON Schema is in realtime/events.schema.json; this is the human summary.

Every event is a line on the SSE feed: event: <type>\ndata: <json>\n\n.

Agent lifecycle & work

Type	Payload	Drives
agent.status	{ agentId, status: "online"|"working"|"idle", role }	Status dots in every view; the list rail.
agent.task.update	{ agentId, taskId, title, elapsedMs, steps:[{label,state:"done"|"active"|"pending"}] }	Observe → "WORKING ON NOW" card.
agent.reasoning.delta	{ agentId, text } (token chunk)	Observe → REASONING STREAM console (append).
agent.computer.frame	{ agentId, app:"browser"|"terminal"|"slack", url?, lines?:[{text,kind}] }	Observe → the live computer screen tile.

Tools, doors & safety (§15)

Type	Payload	Drives
agent.tool.call	{ agentId, tool, target, doorRequired:bool }	Reasoning stream "tool" lines.
door.request	{ doorId, agentId, action, target, summary }	The door-approval toast + "doors awaiting approval" badge.
door.resolve	{ doorId, decision:"approve"|"deny", by }	Dismisses the toast; logs to history.

Collaboration & world

Type	Payload	Drives
agent.message	{ fromAgentId, toAgentId, text, ts }	Observe System → INTER-AGENT COMMS stream; World → message-in-flight on the connector.
world.touch	{ agentId, nodeId, kind:"service"|"event" }	World → Live (Gource): the agent particle retargets to nodeId and emits a beam.
node.activity	{ nodeId, label, kind, heat:0..1 }	World → node glow/pulse intensity.
topology.update	{ formation:"hierarchy"|"flat"|..., nodes:[...], edges:[...] }	World → re-layout (add/remove org units, agents, projects).

System telemetry & routines

Type	Payload	Drives
telemetry	{ tokensPerMin, costPerHr, loops, doorsPending }	Top-bar pills + Observe System telemetry strip.
routine.update	{ routineId, name, kind:"cron"|"loop", owner, schedule?, progress?, state }	Observe System → ROUTINES & LOOPS; the scheduler view.

Mapping rule of thumb: the runner already emits a checkpointed event per step on the durable runner. In the bridge's normalize() you translate each runner event into one or more taxonomy events above. Start by mapping just world.touch, agent.status, and telemetry — that alone makes the Large World view come alive — then add the rest incrementally.

---

5. How each visualization consumes the feed

The client adapter clawmates-live.js is included by each .dc.html (or injected — see §6). It:

1. Reads ?live=<url> (or window.CLAWMATES_LIVE_URL); if absent, does nothing and the view keeps its built-in synthetic animation.
2. Opens an EventSource to that URL.
3. Dispatches each event as a CustomEvent on window and calls any registered handlers:

   window.ClawmatesLive.on('world.touch', e => worldEngine.touch(e.agentId, e.nodeId));
   window.ClawmatesLive.on('agent.reasoning.delta', e => observe.appendReasoning(e.agentId, e.text));
   

4. Buffers events fired before a view registers, and replays the last value per nodeId/agentId so a late-loading view paints current state immediately (mirrors the runner's "resume at checkpoint" behavior).

Integration points inside the .dc.html files (where synthetic → live)

• World — Live canvas: in the logic class, the _startLive() engine has targets[] (the project/service/event nodes) and an agents[] array whose .target index is currently chosen at random on a timer. Replace that random retarget with world.touch: when a world.touch arrives, set the matching agent's .target to the index of nodeId and force a beam. Use topology.update to rebuild targets[]. Everything else (beams, sparks, glow) already keys off the node being "hot", so it just works.
• Observe — Agent: the WORKING-ON-NOW card, the REASONING STREAM, and the computer screen are static markup today. Bind them to agent.task.update, agent.reasoning.delta, and agent.computer.frame respectively (append for deltas; replace for task/frame).
• Observe — System & Dashboard: bind the telemetry pills to telemetry, the comms list to agent.message, routines to routine.update, and the door toast to door.request/door.resolve.

Each of these is a small ClawmatesLive.on(...) handler that calls this.setState(...). No re-layout of the markup is needed — you are feeding values into components that already exist.

---

6. Embedding into the existing Next.js app [INTEGRATES WITH cm-api]

You have two deployment shapes; pick one.

A) Standalone (fastest): serve the visualizations/ folder with Caddy as in §3 and link to it from the app, or drop it behind the same domain at /viz/*. The bridge runs as its own systemd service. Good for an ops/observability surface that lives beside the product.

B) In-app (tightest): mount each .dc.html inside the Next.js app via an <iframe src="/viz/...">, and have the app pass the live URL through: /viz/world?live=/api/live. Proxy /api/live in Next to the bridge so it inherits the same session→bearer swap and CORS as the rest of /api. This keeps auth and origin identical to the product.

Either way, the bridge is the only new long-running process. Run it under systemd:

# /etc/systemd/system/clawmates-live.service
[Unit]
Description=Clawmates Live Visualization Bridge
After=network.target

[Service]
WorkingDirectory=/opt/clawmates/handoff/realtime
EnvironmentFile=/opt/clawmates/handoff/.env
ExecStart=/usr/bin/node server.mjs
Restart=always
RestartSec=2
# Hardening — read-only, no privileged access (consistent with §15 posture):
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target

sudo cp -r . /opt/clawmates/handoff
sudo systemctl enable --now clawmates-live
sudo systemctl status clawmates-live

---

7. Verification checklist

Run these in order; each should pass before moving on.

1. node realtime/server.mjs (demo) prints Live bridge on http://localhost:8420/live · mode=demo.
2. curl -N http://localhost:8420/live streams event: ... lines that don't stop. (Ctrl-C to exit.)
3. The World view at ?live=... → Live formation shows particles retargeting in sync with the world.touch lines you see in the curl output (not the smooth random demo motion).
4. Kill the bridge mid-stream → the browser shows the reconnect state and resumes when it returns (EventSource auto-reconnects; the adapter replays last-known state).
5. [live] Point at cm-api, trigger a real run, and confirm telemetry numbers in the top bar move.

---

8. Troubleshooting

• View looks alive but ignores the feed → you didn't pass ?live= (or window.CLAWMATES_LIVE_URL isn't set); it's running the built-in synthetic animation. That's the intended fallback.
• CORS error in console → add the browser's origin to LIVE_ALLOWED_ORIGINS in .env.
• SSE connects then drops every ~30s → a proxy is buffering/timing out SSE. Disable buffering (proxy_buffering off; in nginx; Caddy is fine by default) and set read timeout high.
• Canvas blank on the Live tab → the tab was never the active formation at first paint; switch to it once. (The engine starts on mount/visibility.)
• High CPU → only the Live canvas animates; cap devicePixelRatio (already capped at 2) or pause rendering when the tab is hidden (document.hidden).

---

9. What to build next (ordered)

1. Wire world.touch + topology.update → the World Live engine. (Biggest visual payoff, smallest change.)
2. Make canvas nodes clickable → drive the right-hand PANEL and drill into an agent (switch to Observe/Agent).
3. Bind Observe Agent to agent.task.update / agent.reasoning.delta / agent.computer.frame.
4. Add a time-scrubber that replays the durable runner's checkpoint log — Gource-style playback of the world's history.
5. Promote the bridge from read-only mirror to a typed gateway with backpressure + per-tenant scoping.

---

Generated as a handoff for the Clawmates visualization layer. The .dc.html files are the source of truth for the UI; this document plus realtime/ is everything needed to serve them and make them breathe with live agent activity.