ci: build and deploy to production from a push to main

Closes the one manual step left in the pipeline. gw-04 has run
clawmates-deploy.timer every minute since July, pulling :latest and rolling on
drift — the CD half already worked. What was missing was anything that moved
:latest, since the old build host (tank) is packed for the move.

The runner lives on gw-04 because it is the only reachable x86_64 host and prod
images must be linux/amd64: web-01 is aarch64 and the fleet build boxes are
offline. Host executor, capacity 1, so builds serialize rather than competing
with production traffic.

Three details that are not obvious:
- `docker push :latest` does NOT move the tag on this registry once the manifest
  exists under another tag. The PUT-the-manifest step is what actually moves it,
  and its absence is how a "successful" deploy could leave prod on a stale image.
- The final step verifies the image prod is RUNNING, not the one we pushed. A
  green edge on the old image is the failure this pipeline exists to prevent.
- broker is built here too. It had no :latest tag at all, so gw-04's deploy loop
  logged a pull failure every single cycle since 2026-08-11.

Also ignore the local env backups: `.env` was ignored but `.env.bak.*` was not,
and those copies hold real credentials.

Co-Authored-By: Claude Opus 5 <[email protected]>

.gitea/workflows/deploy.yml

@@ -0,0 +1,152 @@
+# Local → production pipeline.
+#
+#   push to main → test → build amd64 images → push to the fleet registry
+#   → move :latest → gw-04's existing 60s rolling timer picks it up.
+#
+# The last hop is NOT in this file and does not need to be: gw-04 already runs
+# `clawmates-deploy.timer` every minute, which pulls
+# `$REGISTRY/clawmates/<svc>:latest`, compares it to the running image id, and
+# recreates on drift. This workflow's job is to make `:latest` mean the newest
+# green commit. See deploy/gw-04/clawmates-deploy.sh.
+#
+# Runs on the `gw04` runner (host executor, systemd unit act-runner). gw-04 is
+# the only reachable x86_64 host — web-01 is aarch64 and the fleet build boxes
+# are packed — and prod images must be linux/amd64, so builds are native here
+# rather than emulated.
+name: deploy
+
+on:
+  push:
+    branches: [main]
+  # Lets you re-run a deploy without an empty commit.
+  workflow_dispatch:
+
+env:
+  REGISTRY: 100.94.185.103:5000
+  NAMESPACE: clawmates
+
+jobs:
+  test:
+    runs-on: gw04
+    steps:
+      - uses: actions/checkout@v4
+
+      # A throwaway Postgres so the integration tests actually run. Without
+      # CM_TEST_DATABASE_URL, cm-testkit tries a default admin URL and the
+      # approvals_api tests die on PoolTimedOut — which looks like a failure but
+      # only means "no database here".
+      - name: Start test Postgres
+        run: |
+          docker rm -f cm-ci-pg 2>/dev/null || true
+          docker run -d --name cm-ci-pg \
+            -e POSTGRES_PASSWORD=postgres -e POSTGRES_USER=postgres \
+            -p 127.0.0.1:55432:5432 postgres:16-alpine
+          for i in $(seq 1 30); do
+            docker exec cm-ci-pg pg_isready -U postgres >/dev/null 2>&1 && break
+            sleep 2
+          done
+          docker exec cm-ci-pg pg_isready -U postgres
+
+      # Rust lives in a container because gw-04 has no cargo. The named volumes
+      # are the whole reason this is not painfully slow: without them every run
+      # recompiles the world.
+      - name: Rust tests
+        run: |
+          docker run --rm --network host \
+            -v "$PWD":/w -w /w \
+            -v cm-ci-cargo-registry:/usr/local/cargo/registry \
+            -v cm-ci-cargo-git:/usr/local/cargo/git \
+            -v cm-ci-target:/w/target \
+            -e SQLX_OFFLINE=true \
+            -e CM_TEST_DATABASE_URL=postgres://postgres:[email protected]:55432/postgres \
+            rust:1.96-slim \
+            sh -c 'apt-get update -qq && apt-get install -y -qq pkg-config libssl-dev cmake git >/dev/null && cargo test --workspace'
+
+      - name: Stop test Postgres
+        if: always()
+        run: docker rm -f cm-ci-pg 2>/dev/null || true
+
+      # node 22 is on the host, so these run directly.
+      - name: Frontend checks
+        working-directory: frontend
+        run: |
+          npm ci --no-audit --no-fund
+          npm run typecheck
+          npm run test
+          # Lint is advisory: the repo currently has pre-existing max-lines and
+          # set-state-in-effect errors that predate this pipeline. Failing the
+          # deploy on them would mean nothing could ship until they are cleared.
+          npm run lint || echo "::warning::lint reported problems (advisory)"
+
+  build:
+    runs-on: gw04
+    needs: test
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Build + push images
+        run: |
+          set -eu
+          SHA=$(git rev-parse --short HEAD)
+          echo "SHA=$SHA" >> "$GITHUB_ENV"
+          # The daemon binary the frontend serves at /dl. images/frontend.Dockerfile
+          # expects it staged; rsync-based deploys create it out of band, so build
+          # it here or the image ships without the node installer.
+          mkdir -p frontend/public/dl
+          docker run --rm \
+            -v "$PWD":/w -w /w \
+            -v cm-ci-cargo-registry:/usr/local/cargo/registry \
+            -v cm-ci-cargo-git:/usr/local/cargo/git \
+            -v cm-ci-target:/w/target \
+            -e SQLX_OFFLINE=true -e CARGO_NET_GIT_FETCH_WITH_CLI=true \
+            rust:1.96-slim \
+            sh -c 'apt-get update -qq && apt-get install -y -qq pkg-config libssl-dev cmake git >/dev/null && cargo build --release -p clawmates-node && cp target/release/clawmates-node frontend/public/dl/clawmates-node-linux-amd64'
+
+          for svc in server frontend broker; do
+            docker build -f "images/$svc.Dockerfile" \
+              -t "$REGISTRY/$NAMESPACE/$svc:main-$SHA" \
+              -t "$REGISTRY/$NAMESPACE/$svc:latest" .
+            docker push "$REGISTRY/$NAMESPACE/$svc:main-$SHA"
+            docker push "$REGISTRY/$NAMESPACE/$svc:latest"
+          done
+
+      # `docker push :latest` does NOT reliably move the tag on this registry:
+      # when the manifest already exists under another tag (it does — we just
+      # pushed main-$SHA), the push reports a digest but `:latest` keeps
+      # resolving to the OLD image. Writing the manifest to the tag over the
+      # HTTP API is what actually moves it. This is the same trick
+      # scripts/deploy.sh uses, and the reason a "successful" deploy could
+      # previously leave prod on a stale image.
+      - name: Repoint :latest
+        run: |
+          set -eu
+          for svc in server frontend broker; do
+            ct=$(curl -s -o /tmp/m.json -D- \
+              -H 'Accept: application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.oci.image.manifest.v1+json' \
+              "http://$REGISTRY/v2/$NAMESPACE/$svc/manifests/main-$SHA" \
+              | awk -F': ' '/^[Cc]ontent-[Tt]ype/{print $2}' | tr -d '\r')
+            code=$(curl -s -o /dev/null -w '%{http_code}' -X PUT \
+              -H "Content-Type: $ct" --data-binary @/tmp/m.json \
+              "http://$REGISTRY/v2/$NAMESPACE/$svc/manifests/latest")
+            echo "$svc :latest → main-$SHA (HTTP $code)"
+            case "$code" in 20*) ;; *) echo "tag write failed"; exit 1 ;; esac
+          done
+
+      # Verify the thing that actually matters: what prod is RUNNING, not what
+      # we pushed. A green edge on a stale image is the failure mode this whole
+      # pipeline exists to prevent.
+      - name: Wait for the rolling deploy
+        run: |
+          set -eu
+          want=$(docker image inspect -f '{{.Id}}' "$REGISTRY/$NAMESPACE/server:latest")
+          for i in $(seq 1 30); do
+            got=$(docker inspect -f '{{.Image}}' clawmates_server_1 2>/dev/null || echo none)
+            if [ "$got" = "$want" ]; then
+              echo "prod is running main-$SHA"
+              curl -s -o /dev/null -w "edge HTTP %{http_code}\n" -m 10 https://clawmates.work/ || true
+              exit 0
+            fi
+            sleep 10
+          done
+          echo "prod did not roll onto main-$SHA within 5m — check clawmates-deploy.timer"
+          exit 1

.gitignore

@@ -14,3 +14,18 @@ token.key
 
 # Hosted node-agent binaries (built + baked into the frontend image, not committed)
 frontend/public/dl/
+
+# Local env backups. `.env` is already ignored above, but a timestamped or
+# suffixed copy of it is not — and these hold real credentials (subscription
+# OAuth token, forge PAT, DB password). Ignore every variant, not just the
+# exact name.
+.env.bak*
+*.env.bak*
+deploy/compose/.env.*
+
+# Local-only compose override. NOT for prod or the air-gapped install: it
+# rebinds published ports to loopback, enables the login bypass, and points the
+# runtime at MacBook-specific paths. docker-compose picks this file up
+# automatically, so committing it would silently reconfigure anyone who runs
+# deploy/compose.
+deploy/compose/docker-compose.override.yml