Split deployment into event and turn nodes
coturn now runs on its own node (turn.redclaw.dev), so it owns TCP 443 for TURN-over-TLS with no conflict against Caddy's HTTPS on the event node. This removes the single-IP 443 trade-off and restores the PRD's full strict-firewall traversal by default. - README: two-node runbook (event node: Caddy + signaling; turn node: coturn). Per-node installs and ufw rules; certbot on the turn node needs no Caddy stop. - turnserver.conf, Caddyfile: drop the single-IP 443 conflict notes; coturn owns 443 on its dedicated node. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
5bb512979e
commit
48084dd0df
@@ -87,17 +87,38 @@ Web (`web/.env`, see `.env.example`):
|
|||||||
|
|
||||||
## Production deployment runbook
|
## Production deployment runbook
|
||||||
|
|
||||||
A single small VPS (Hetzner or Vultr) runs everything: Caddy (TLS + static SPA
|
LiveCast deploys across two nodes:
|
||||||
+ reverse proxy), the Node signaling server, and coturn. Commands below assume
|
|
||||||
a fresh Ubuntu 22.04/24.04 host and `sudo` access. Replace `redclaw.dev` hosts
|
|
||||||
and the certbot email if they differ.
|
|
||||||
|
|
||||||
### Phase 0, provisioning
|
- **event node** (`event.redclaw.dev`): Caddy (TLS, static SPA, reverse proxy)
|
||||||
|
and the Node signaling server.
|
||||||
|
- **turn node** (`turn.redclaw.dev`): coturn only.
|
||||||
|
|
||||||
DNS first, so cert issuance works later: point `event.redclaw.dev` and
|
Giving coturn its own node lets it own TCP 443 for TURN-over-TLS (the path that
|
||||||
`turn.redclaw.dev` A records at the host's public IP.
|
defeats firewalls allowing only 443) without colliding with Caddy's HTTPS.
|
||||||
|
Commands assume fresh Ubuntu 22.04/24.04 hosts with `sudo`. Replace the
|
||||||
|
`redclaw.dev` hosts and the certbot email if they differ.
|
||||||
|
|
||||||
Install Node LTS, Caddy, coturn, certbot, and git:
|
### Phase 0, DNS and provisioning
|
||||||
|
|
||||||
|
DNS first, so cert issuance works: `event.redclaw.dev` A record to the event
|
||||||
|
node IP, `turn.redclaw.dev` A record to the turn node IP.
|
||||||
|
|
||||||
|
**Turn node** (coturn + certbot):
|
||||||
|
|
||||||
|
```
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install -y coturn certbot git
|
||||||
|
sudo git clone https://git.redclaw.dev/clawverse/livecast.git /opt/livecast
|
||||||
|
|
||||||
|
sudo ufw allow 80/tcp # ACME cert issuance and renewal (certbot)
|
||||||
|
sudo ufw allow 443 # TURN over TLS
|
||||||
|
sudo ufw allow 3478 # STUN/TURN (UDP + TCP)
|
||||||
|
sudo ufw allow 5349 # TURN over TLS (standard port)
|
||||||
|
sudo ufw allow 49152:65535/udp # TURN relay range
|
||||||
|
sudo ufw enable
|
||||||
|
```
|
||||||
|
|
||||||
|
**Event node** (Node + Caddy):
|
||||||
|
|
||||||
```
|
```
|
||||||
# Node LTS (NodeSource)
|
# Node LTS (NodeSource)
|
||||||
@@ -111,78 +132,59 @@ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
|
|||||||
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
|
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
|
||||||
| sudo tee /etc/apt/sources.list.d/caddy-stable.list
|
| sudo tee /etc/apt/sources.list.d/caddy-stable.list
|
||||||
sudo apt-get update
|
sudo apt-get update
|
||||||
sudo apt-get install -y caddy coturn certbot git
|
sudo apt-get install -y caddy git
|
||||||
```
|
|
||||||
|
|
||||||
Get the code onto the box:
|
|
||||||
|
|
||||||
```
|
|
||||||
sudo git clone https://git.redclaw.dev/clawverse/livecast.git /opt/livecast
|
sudo git clone https://git.redclaw.dev/clawverse/livecast.git /opt/livecast
|
||||||
```
|
|
||||||
|
|
||||||
Open the firewall (`ufw`):
|
|
||||||
|
|
||||||
```
|
|
||||||
sudo ufw allow 80/tcp # HTTP (Let's Encrypt, redirect to HTTPS)
|
sudo ufw allow 80/tcp # HTTP (Let's Encrypt, redirect to HTTPS)
|
||||||
sudo ufw allow 443 # HTTPS (Caddy)
|
sudo ufw allow 443 # HTTPS (Caddy)
|
||||||
sudo ufw allow 3478 # STUN/TURN (UDP + TCP)
|
|
||||||
sudo ufw allow 5349 # TURN over TLS
|
|
||||||
sudo ufw allow 49152:65535/udp # TURN relay range
|
|
||||||
sudo ufw enable
|
sudo ufw enable
|
||||||
```
|
```
|
||||||
|
|
||||||
### Phase 1, TURN/STUN
|
### Phase 1, turn node (coturn)
|
||||||
|
|
||||||
Generate the shared secret, install the config, and inject the secret into the
|
Generate the shared secret, install the config, and inject the secret into the
|
||||||
live config (not the repo copy). **Save the printed secret** for the signaling
|
live config (not the repo copy). **Save the printed secret** for the event
|
||||||
server's `.env` in Phase 2; the two must match.
|
node's `.env` in Phase 2; the two must match.
|
||||||
|
|
||||||
```
|
```
|
||||||
SECRET=$(openssl rand -hex 32)
|
SECRET=$(openssl rand -hex 32)
|
||||||
sudo cp /opt/livecast/deploy/turnserver.conf /etc/turnserver.conf
|
sudo cp /opt/livecast/deploy/turnserver.conf /etc/turnserver.conf
|
||||||
sudo sed -i "s/REPLACE_WITH_LONG_RANDOM_SECRET/$SECRET/" /etc/turnserver.conf
|
sudo sed -i "s/REPLACE_WITH_LONG_RANDOM_SECRET/$SECRET/" /etc/turnserver.conf
|
||||||
echo "TURN_SECRET=$SECRET" # copy this value into server/.env
|
echo "TURN_SECRET=$SECRET" # copy this into the event node's server/.env
|
||||||
|
|
||||||
# coturn ships disabled on Ubuntu; enable the daemon
|
# coturn ships disabled on Ubuntu; enable the daemon
|
||||||
sudo sed -i 's/^#TURNSERVER_ENABLED/TURNSERVER_ENABLED/' /etc/default/coturn
|
sudo sed -i 's/^#TURNSERVER_ENABLED/TURNSERVER_ENABLED/' /etc/default/coturn
|
||||||
```
|
```
|
||||||
|
|
||||||
**TURN-over-TLS on 443 (decision point).** On a single public IP, Caddy owns
|
Obtain the TURN certificate. coturn owns this node (no Caddy to stop), so
|
||||||
TCP 443 for HTTPS, so coturn cannot also bind 443. Two options:
|
certbot standalone can use port 80 directly. The deploy hook copies the cert
|
||||||
|
into `/etc/coturn` so the unprivileged coturn user can read it, and restarts
|
||||||
- *Single IP (default, zero cost):* keep TURN-over-TLS on 5349 only. Comment
|
coturn now and on every future renewal:
|
||||||
the 443 listener out: `sudo sed -i 's/^alt-tls-listening-port=443/#&/' /etc/turnserver.conf`.
|
|
||||||
Slightly weaker traversal through firewalls that block everything except 443.
|
|
||||||
- *Two IPs (full PRD behavior):* attach a second IP to the host, set
|
|
||||||
`listening-ip` / `relay-ip` to it in `/etc/turnserver.conf`, and leave
|
|
||||||
`alt-tls-listening-port=443` enabled so TURN-over-TLS is reachable on 443.
|
|
||||||
|
|
||||||
Obtain the TURN certificate. The deploy hook copies it into `/etc/coturn` (so
|
|
||||||
the unprivileged coturn user can read it) and restarts coturn, now and on every
|
|
||||||
future renewal. certbot needs port 80, so stop Caddy briefly:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
sudo chmod +x /opt/livecast/deploy/coturn-cert-deploy-hook.sh
|
sudo chmod +x /opt/livecast/deploy/coturn-cert-deploy-hook.sh
|
||||||
sudo systemctl stop caddy
|
|
||||||
sudo certbot certonly --standalone -d turn.redclaw.dev \
|
sudo certbot certonly --standalone -d turn.redclaw.dev \
|
||||||
--non-interactive --agree-tos -m [email protected] \
|
--non-interactive --agree-tos -m [email protected] \
|
||||||
--deploy-hook /opt/livecast/deploy/coturn-cert-deploy-hook.sh
|
--deploy-hook /opt/livecast/deploy/coturn-cert-deploy-hook.sh
|
||||||
sudo systemctl start caddy
|
sudo systemctl enable --now coturn
|
||||||
sudo systemctl enable coturn
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`alt-tls-listening-port=443` stays enabled in `turnserver.conf`: with coturn
|
||||||
|
alone on this node there is no conflict, so TURN-over-TLS is reachable on 443.
|
||||||
|
|
||||||
Verify with a Trickle ICE test against `turn.redclaw.dev` that both `srflx`
|
Verify with a Trickle ICE test against `turn.redclaw.dev` that both `srflx`
|
||||||
(STUN) and `relay` (TURN) candidates appear (AC-10), then confirm TURN-over-TLS
|
(STUN) and `relay` (TURN) candidates appear (AC-10), then confirm TURN-over-TLS
|
||||||
works from a locked-down network.
|
on 443 works from a locked-down network.
|
||||||
|
|
||||||
### Phase 2, signaling
|
### Phase 2, event node (signaling)
|
||||||
|
|
||||||
```
|
```
|
||||||
cd /opt/livecast/server
|
cd /opt/livecast/server
|
||||||
sudo npm install --omit=dev
|
sudo npm install --omit=dev
|
||||||
sudo cp .env.example .env
|
sudo cp .env.example .env
|
||||||
sudo nano .env # set TURN_SECRET (the value from Phase 1), TURN_HOST,
|
sudo nano .env # set TURN_SECRET (the value from Phase 1),
|
||||||
# TURN_REALM = turn.redclaw.dev. Leave PORT=8080.
|
# TURN_HOST=turn.redclaw.dev, TURN_REALM=turn.redclaw.dev.
|
||||||
|
# Leave PORT=8080.
|
||||||
|
|
||||||
sudo cp /opt/livecast/deploy/livecast-signaling.service /etc/systemd/system/
|
sudo cp /opt/livecast/deploy/livecast-signaling.service /etc/systemd/system/
|
||||||
sudo systemctl daemon-reload
|
sudo systemctl daemon-reload
|
||||||
@@ -191,7 +193,7 @@ sudo systemctl enable --now livecast-signaling
|
|||||||
curl localhost:8080/health # expect 200 with status JSON (AC-7)
|
curl localhost:8080/health # expect 200 with status JSON (AC-7)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Phase 3, web app
|
### Phase 3, event node (web app + Caddy)
|
||||||
|
|
||||||
Build on the box (Node is installed) and let Caddy serve the static output.
|
Build on the box (Node is installed) and let Caddy serve the static output.
|
||||||
`VITE_SIGNAL_URL` is left blank so the client derives `wss://event.redclaw.dev/ws`
|
`VITE_SIGNAL_URL` is left blank so the client derives `wss://event.redclaw.dev/ws`
|
||||||
|
|||||||
+4
-5
@@ -21,8 +21,7 @@ event.redclaw.dev {
|
|||||||
file_server
|
file_server
|
||||||
}
|
}
|
||||||
|
|
||||||
# Note: turn.redclaw.dev is NOT served by Caddy. coturn terminates its own TLS
|
# Note: turn.redclaw.dev runs on its own node and is NOT served by Caddy. coturn
|
||||||
# using a certificate obtained by certbot (see the runbook and the certbot
|
# terminates its own TLS there using a certificate obtained by certbot (see the
|
||||||
# deploy hook in deploy/coturn-cert-deploy-hook.sh). On a single-IP host Caddy
|
# runbook and deploy/coturn-cert-deploy-hook.sh), and owns TCP 443 for
|
||||||
# owns TCP 443 for HTTPS, so coturn's TURN-over-TLS uses 5349 there; see the
|
# TURN-over-TLS on that node.
|
||||||
# runbook for the second-IP option that restores TURN-over-TLS on 443.
|
|
||||||
|
|||||||
@@ -7,10 +7,8 @@
|
|||||||
|
|
||||||
listening-port=3478
|
listening-port=3478
|
||||||
tls-listening-port=5349
|
tls-listening-port=5349
|
||||||
# Also listen on 443 for TURN-over-TLS to defeat strict firewalls.
|
# Also listen on 443 for TURN-over-TLS to defeat strict firewalls. coturn runs
|
||||||
# IMPORTANT: on a single-IP host Caddy already owns TCP 443 for HTTPS, so this
|
# on its own node (turn.redclaw.dev), so it owns 443 here with no conflict.
|
||||||
# line must be commented out there (TURN-over-TLS then uses 5349). Keep it only
|
|
||||||
# if coturn is bound to a second IP. See the runbook "TURN-over-TLS on 443".
|
|
||||||
alt-tls-listening-port=443
|
alt-tls-listening-port=443
|
||||||
|
|
||||||
fingerprint
|
fingerprint
|
||||||
|
|||||||
Reference in New Issue
Block a user