Under a half-open TCP (server side closed, client OS still buffering
writes), the daemon's write.send() inside the tokio::select! branch
blocks forever. tokio::select does not preempt a running future, so
the whole loop freezes — idle_tick never gets to check last_rx, no
'channel ended' log ever fires, and the daemon silently spins on a
dead socket for hours.
Observed on architect Jul 5 2026: daemon connected at 12:59:13,
heartbeats worked for ~2.5 min, then went silent. TCP session showed
ESTABLISHED on the node side, no ESTABLISHED on the gateway side.
Restarting the daemon 'fixed' it — but it re-hung within minutes.
Fix: wrap both write.send() call sites in a tokio::time::timeout of
WRITE_DEADLINE=10s. If a send stalls past that, we log and return Ok()
to trigger the main-loop reconnect. Short enough that it fires long
before the 40s read-idle would (which was our only escape hatch and
never triggered because the loop was frozen).
Omar Sobh
51d501e36e