feat(voice): same-origin proxy client reusing the reliable /webhook path

The /ws/chat path builds a fresh Agent that omits the dynamically-registered
peripheral tools (matrix_pattern), so the model improvises with shell/read and the
matrix never changes. Switch the voice client to serve.py, which serves the page and
proxies POST /webhook same-origin to the node gateway (bearer token server-side).
Browser does STT+TTS (Web Speech API, no keys); the agent runs on cloud sonnet and
fires matrix_pattern reliably. Validated end-to-end server-side (checker in 6.2s).

Co-Authored-By: Claude Opus 4.8 <[email protected]>

deploy/voice-client/README.md

@@ -1,39 +1,44 @@
 # APESS Voice → Node client
 
-A self-contained, single-file voice client that lets you **talk to the on-board agent
+Talk to the on-board agent and change the LED-matrix animation **by voice** — same
-and change the LED-matrix animation by voice** — same agent, same `matrix_pattern` tool
+agent, same `matrix_pattern` tool as web chat and Telegram.
-as web chat and Telegram.
 
-It does **STT and TTS entirely in the browser** (Web Speech API), so it needs **no keys,
+- **STT + TTS run in the browser** (Web Speech API) — no ElevenLabs, no keys.
-no ElevenLabs, and no server-side TTS**. It talks to the node's `/ws/chat` WebSocket
+- **`serve.py` serves the page AND proxies `/webhook`** to the node on the same origin,
-(`?agent=demo`), which requires no auth on the workshop board (`require_pairing = false`).
+  so the browser needs no CORS and no bearer token, and we reuse the reliable
+  `/webhook` path (the `/ws/chat` path drops the peripheral matrix tool, so we avoid it).
 
-## Run it (needs a secure context for mic access)
+## Run it
 
-`getUserMedia`/`SpeechRecognition` only work on `https://` or `http://localhost`. Serve the
+The node is reached over USB via `adb forward`, or by its LAN IP on the day.
-folder from localhost on the demo laptop:
 
 ```bash
+# 1. expose the board's gateway locally (USB path)
+adb forward tcp:8080 tcp:8080
+
+# 2. serve the client + proxy (token stays server-side, never in the browser)
 cd deploy/voice-client
-python3 -m http.server 8090
+NODE_URL=http://127.0.0.1:8080 NODE_TOKEN=<gateway-bearer-token> python3 serve.py 8090
-# open http://localhost:8090 in Chrome
+#   on the day, point NODE_URL at the board's LAN IP instead:
+#   NODE_URL=http://192.168.x.x:8080 NODE_TOKEN=... python3 serve.py 8090
+
+# 3. open http://localhost:8090 in Chrome
 ```
 
-Then in the page:
+Then: the dot goes green (proxy reachable), **hold** the circle, say
-1. Set the board address (e.g. `192.168.x.x:8080`) and agent (`demo`), click **Connect**
+*"show the wave animation"*, release. The node runs the agent → `matrix_pattern` →
-   (the dot goes green).
+the matrix changes, and the reply is spoken back.
-2. **Hold** the circle, say *"show the wave animation"*, release.
-3. The node runs the lean on-board agent → `matrix_pattern` → the matrix changes, and the
-   reply is spoken back.
 
-The `ws://…` connection works fine from an `http://localhost` page (only `https` pages
+`localhost` is a secure context so Chrome grants mic access; the proxy hop is
-block insecure `ws`). Chrome is required (Web Speech API).
+server-side so there is no CORS. One process, one origin, no keys. Chrome required
+(Web Speech API).
 
 ## Upgrading to ElevenLabs (later)
 
-Browser TTS is functional but robotic. For ElevenLabs-quality speech, switch to the
+Browser TTS is functional but robotic. For ElevenLabs-quality speech, add a TTS call
-gateway's **voice-duplex** path (`gateway-voice-duplex` binary already built): configure
+in `serve.py`: after getting the node's `response`, POST it to ElevenLabs
-`[channels.voice_duplex.default]` + an ElevenLabs `tts_provider`, set the demo agent's
+`/v1/text-to-speech/<voice_id>` with your key and return the audio to the client to
-`tts_provider`, and have the client emit `speech_end {transcript}` and play the server's
+play, instead of using the browser's `speechSynthesis`. STT stays browser-side.
-`tts_chunk` frames instead of using `speechSynthesis`. The STT stays browser-side.
+(The gateway also has a native voice-duplex path, but it shares the `/ws/chat`
+peripheral-tool gap noted above and needs a fork fix first.)
 </content>

deploy/voice-client/index.html

@@ -60,10 +60,11 @@
   const ipEl=$('ip'), agentEl=$('agent'), micEl=$('mic'), logEl=$('log'),
         dot=$('dot'), connLabel=$('connlabel'), hint=$('hint'), connectBtn=$('connect');
 
-  // Persist the board address between sessions.
+  // The node address is configured on serve.py; the browser only picks the agent.
-  ipEl.value = localStorage.getItem('apess_ip') || location.hostname + ':8080';
+  ipEl.style.display = 'none';
+  agentEl.value = localStorage.getItem('apess_agent') || 'demo';
 
-  let ws=null, connected=false, nodeRow=null, nodeText='';
+  let connected=false;
 
   const SR = window.SpeechRecognition || window.webkitSpeechRecognition;
   if (!SR) { hint.textContent = 'This browser has no Web Speech API — use Chrome.'; }
@@ -71,7 +72,7 @@
   function setConn(state){ // 'on' | 'err' | ''
     dot.className = 'dot' + (state ? ' '+state : '');
     connected = state==='on';
-    connLabel.textContent = connected ? 'Connected' : (state==='err'?'Retry':'Connect');
+    connLabel.textContent = connected ? 'Ready' : (state==='err'?'Retry':'Connect');
     micEl.disabled = !connected || !SR;
     if (connected) hint.textContent = 'Hold the circle, speak, release. The node acts and talks back.';
   }
@@ -83,34 +84,14 @@
     r.scrollIntoView({behavior:'smooth',block:'end'}); return r;
   }
 
+  // Transport: POST to a SAME-ORIGIN /webhook that serve.py proxies to the node's
+  // gateway (reliable path — the WS path drops peripheral tools). No CORS, no auth
+  // in the browser (serve.py holds the bearer token).
   function connect(){
-    const ip = ipEl.value.trim(); const agent = agentEl.value.trim() || 'demo';
+    const agent = agentEl.value.trim() || 'demo';
-    localStorage.setItem('apess_ip', ip);
+    localStorage.setItem('apess_agent', agent);
-    try { ws && ws.close(); } catch(e){}
     setConn('');
-    ws = new WebSocket(`ws://${ip}/ws/chat?agent=${encodeURIComponent(agent)}`);
+    fetch('/ping').then(r => setConn(r.ok?'on':'err')).catch(()=>setConn('err'));
-    ws.onopen = () => setConn('on');
-    ws.onclose = () => setConn('');
-    ws.onerror = () => setConn('err');
-    ws.onmessage = ev => {
-      let m; try { m = JSON.parse(ev.data); } catch(e){ return; }
-      switch (m.type) {
-        case 'chunk':
-          if (!nodeRow){ nodeRow=addRow('node','Node',''); nodeText=''; }
-          nodeText += (m.content||''); nodeRow.querySelector('.txt').textContent = nodeText; break;
-        case 'tool_call':
-          addRow('tool','tool', '→ '+(m.tool||m.name||'tool')+ (m.args?(' '+JSON.stringify(m.args)):'')); break;
-        case 'agent_end':
-        case 'done': {
-          const reply = (m.full_response!=null? m.full_response : nodeText).trim();
-          if (reply && !nodeRow) addRow('node','Node',reply);
-          else if (reply && nodeRow) nodeRow.querySelector('.txt').textContent = reply;
-          speak(reply); nodeRow=null; nodeText=''; break;
-        }
-        case 'error':
-          addRow('tool','error', m.message||m.error||'error'); nodeRow=null; break;
-      }
-    };
   }
 
   function speak(text){
@@ -120,11 +101,23 @@
     u.rate = 1.02; u.pitch = 1.0; window.speechSynthesis.speak(u);
   }
 
-  function send(text){
+  async function send(text){
     if (!text) return;
     addRow('you','You',text);
-    if (ws && ws.readyState===1) ws.send(JSON.stringify({type:'message', content:text}));
+    const agent = agentEl.value.trim() || 'demo';
-    else addRow('tool','error','not connected');
+    const pending = addRow('node','Node','…');
+    try {
+      const r = await fetch('/webhook?agent='+encodeURIComponent(agent), {
+        method:'POST', headers:{'Content-Type':'application/json'},
+        body: JSON.stringify({message:text})
+      });
+      const j = await r.json();
+      const reply = (j.response || j.error || '(no response)').trim();
+      pending.querySelector('.txt').textContent = reply;
+      speak(reply);
+    } catch(e) {
+      pending.querySelector('.txt').textContent = 'error: '+e.message;
+    }
   }
 
   // --- push-to-talk: hold the circle, speak, release ---
@@ -155,6 +148,7 @@
   micEl.addEventListener('touchend', e=>{e.preventDefault();stopListen();},{passive:false});
   connectBtn.addEventListener('click', connect);
   setConn('');
+  connect(); // auto-check the proxy on load
 })();
 </script>
 </body>

deploy/voice-client/serve.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+"""APESS voice client host + gateway proxy.
+
+Serves index.html AND proxies POST /webhook to the node's ZeroClaw gateway on the
+same origin — so the browser needs no CORS and no bearer token, and we reuse the
+reliable /webhook path (the /ws/chat path drops the peripheral matrix tool).
+
+Usage:
+    NODE_URL=http://127.0.0.1:8080 NODE_TOKEN=zc_xxx python3 serve.py [port]
+
+- NODE_URL   node gateway base (default http://127.0.0.1:8080; via `adb forward
+             tcp:8080 tcp:8080` over USB, or the board's LAN IP:8080 on the day).
+- NODE_TOKEN gateway bearer token (kept server-side, never sent to the browser).
+- port       local port to serve on (default 8090). Open http://localhost:<port>.
+
+localhost is a secure context, so the browser grants mic access; the proxy hop is
+server-side, so there is no CORS. One process, one origin.
+"""
+import os, sys, json, urllib.request
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+NODE_URL = os.environ.get("NODE_URL", "http://127.0.0.1:8080").rstrip("/")
+NODE_TOKEN = os.environ.get("NODE_TOKEN", "")
+PORT = int(sys.argv[1]) if len(sys.argv) > 1 else 8090
+
+class H(BaseHTTPRequestHandler):
+    def log_message(self, *a): pass  # quiet
+
+    def _send(self, code, body, ctype="application/json"):
+        self.send_response(code)
+        self.send_header("Content-Type", ctype)
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def do_GET(self):
+        if self.path == "/ping":
+            return self._send(200, b'{"ok":true}')
+        path = "/index.html" if self.path in ("/", "") else self.path.split("?")[0]
+        fp = os.path.normpath(os.path.join(HERE, path.lstrip("/")))
+        if not fp.startswith(HERE) or not os.path.isfile(fp):
+            return self._send(404, b"not found", "text/plain")
+        ctype = "text/html" if fp.endswith(".html") else "text/plain"
+        with open(fp, "rb") as f:
+            self._send(200, f.read(), ctype)
+
+    def do_POST(self):
+        if not self.path.startswith("/webhook"):
+            return self._send(404, b'{"error":"not found"}')
+        n = int(self.headers.get("Content-Length", 0))
+        body = self.rfile.read(n)
+        q = self.path[len("/webhook"):]  # keep ?agent=...
+        req = urllib.request.Request(
+            f"{NODE_URL}/webhook{q}", data=body, method="POST",
+            headers={"Content-Type": "application/json",
+                     "Authorization": f"Bearer {NODE_TOKEN}"})
+        try:
+            with urllib.request.urlopen(req, timeout=60) as resp:
+                self._send(resp.status, resp.read())
+        except urllib.error.HTTPError as e:
+            self._send(e.code, e.read() or b'{"error":"upstream"}')
+        except Exception as e:
+            self._send(502, json.dumps({"error": str(e)}).encode())
+
+if __name__ == "__main__":
+    if not NODE_TOKEN:
+        print("WARN: NODE_TOKEN is empty — gateway calls will 401.", file=sys.stderr)
+    print(f"APESS voice client → {NODE_URL}")
+    print(f"open http://localhost:{PORT}")
+    ThreadingHTTPServer(("127.0.0.1", PORT), H).serve_forever()