Fleet terminal: daemon self-diagnosis + immediate banner + debug route
ci / gates (push) Failing after 5s
ci / rust (push) Has been skipped
ci / sandbox-k8s (push) Has been skipped
ci / frontend (push) Has been skipped
ci / e2e (push) Has been skipped

The server trace showed pty_open is sent but the daemon (morpheus, v0.2.0) emits
no pty_out — so the PTY spawn was dying silently. Instrument it:
- daemon open_pty: log open/tmux/first-read/EOF/error/total to stdout, and send
  an IMMEDIATE banner pty_out ("[clawmates] host shell on <host> — starting…") so
  the browser confirms the relay even before the shell draws. If open_pty fails,
  send the error as pty_out (was a silent pty_exit). Bump to v0.2.1.
- cm-api: temp GET /api/debug/node-pty/{id}?dbg=… opens a node terminal and reads
  ~2s of output with no browser/auth, to test the relay in isolation.
- NodeTerminalApp + ticket/ws routes already log each hop ([node-term]/[fleet-term]).

Diagnostic logic: banner shows + shell doesn't → relay ok, shell is the problem;
nothing shows → relay broken; daemon "EOF after N bytes" → shell exited.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
Omar Sobh
2026-06-25 00:06:51 -07:00
co-authored by Claude Opus 4.8
parent b840de7c3b
commit 27f5d05f96
5 changed files with 87 additions and 8 deletions
+39 -2
View File
@@ -136,6 +136,39 @@ pub async fn sandbox_check(
}
}
#[derive(Deserialize)]
pub struct DbgQuery {
pub dbg: String,
}
/// TEMP debug: open a terminal on a node and read ~2s of its output, no browser
/// or auth. Lets us test the daemon→server PTY relay in isolation.
pub async fn debug_node_pty(
State(state): State<AppState>,
Path(id): Path<Uuid>,
Query(q): Query<DbgQuery>,
) -> Json<Value> {
if q.dbg != "clawdbg1" {
return Json(json!({ "ok": false, "error": "denied" }));
}
let node_id = NodeId::from(id);
let Some((sid, mut rx)) = state.node_hub.open_terminal(node_id, 120, 40).await else {
return Json(json!({ "ok": false, "error": "open_terminal None (node not connected)" }));
};
let mut total = 0usize;
let mut preview: Vec<u8> = Vec::new();
let deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(2);
while let Ok(Some(bytes)) = tokio::time::timeout_at(deadline, rx.recv()).await {
total += bytes.len();
if preview.len() < 300 {
preview.extend_from_slice(&bytes);
preview.truncate(300);
}
}
state.node_hub.terminal_close(node_id, sid).await;
Json(json!({ "ok": true, "sid": sid, "bytes": total, "preview": String::from_utf8_lossy(&preview) }))
}
/// `DELETE /api/nodes/{id}` — deregister a node (workspace-scoped).
pub async fn remove(
State(state): State<AppState>,
@@ -179,7 +212,9 @@ pub async fn terminal_ticket(
nodes::get(&state.pool, node_id, user.workspace_id)
.await?
.ok_or(ApiError::NotFound)?;
if !state.node_hub.is_online(node_id).await {
let online = state.node_hub.is_online(node_id).await;
eprintln!("[fleet-term] ticket request node={node_id} online={online}");
if !online {
return Ok(Json(json!({ "error": "node is offline" })));
}
Ok(Json(json!({ "ticket": state.node_hub.mint_ticket(node_id).await })))
@@ -194,7 +229,9 @@ pub async fn terminal_ws(
upgrade: WebSocketUpgrade,
) -> Response {
let node_id = NodeId::from(id);
match state.node_hub.redeem_ticket(&q.token).await {
let redeemed = state.node_hub.redeem_ticket(&q.token).await;
eprintln!("[fleet-term] ws upgrade node={node_id} ticket_ok={}", redeemed == Some(node_id));
match redeemed {
Some(t) if t == node_id => {}
_ => return ApiError::Unauthorized.into_response(),
}