Omar SobhandClaude Opus 5
25a3d6902a
On workflow_dispatch GITHUB_REF_NAME is the BRANCH, so the upload step created a Gitea release AND a git tag both named "main" — a tag sharing the branch name, from a run that was only meant to be a smoke test. Both have been deleted. Gated on github.ref_type == 'tag'. A dispatch now exercises build, SBOM, sign and offline verify, and stops there. Co-Authored-By: Claude Opus 5 <[email protected]>
216 lines
10 KiB
YAML
216 lines
10 KiB
YAML
# Release: build the images both deploy targets share, assemble the SIGNED
|
|
# air-gapped bundle, verify it offline, rehearse the customer's install, and
|
|
# attach everything to the Gitea release for the tag.
|
|
#
|
|
# Moved from .github/workflows/ and rewritten for this forge. The old copy could
|
|
# never have run: `runs-on: ubuntu-latest` matches no runner here, and
|
|
# `softprops/action-gh-release` talks to GitHub's API, not Gitea's.
|
|
#
|
|
# The signing key is a repo secret (BUNDLE_SIGNING_KEY, hex ed25519 from
|
|
# `clawmates-bundler keygen`). The matching PUBLIC key is published out of band
|
|
# so customers can verify a bundle before `docker load`.
|
|
name: release
|
|
|
|
on:
|
|
push:
|
|
tags: ["v*"]
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
bundle:
|
|
runs-on: gw04
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Version from tag
|
|
run: |
|
|
# workflow_dispatch has no tag; fall back to the short sha so a manual
|
|
# run produces a clearly-not-a-release version rather than an empty one.
|
|
if [ "${GITHUB_REF_TYPE:-}" = "tag" ]; then
|
|
echo "VERSION=${GITHUB_REF_NAME#v}" >> "$GITHUB_ENV"
|
|
else
|
|
echo "VERSION=0.0.0-$(git rev-parse --short HEAD)" >> "$GITHUB_ENV"
|
|
fi
|
|
|
|
- name: Build images
|
|
run: |
|
|
set -eu
|
|
docker build -t "clawmates/server:$VERSION" -f images/server.Dockerfile .
|
|
docker build -t "clawmates/frontend:$VERSION" -f images/frontend.Dockerfile .
|
|
docker build -t "clawmates/broker:$VERSION" -f images/broker.Dockerfile .
|
|
docker build -t "clawmates/agent-base:$VERSION" images/agent-base
|
|
docker build -t "clawmates/agent-browser:$VERSION" images/agent-browser
|
|
docker pull -q postgres:16-alpine
|
|
docker pull -q tecnativa/docker-socket-proxy:0.3
|
|
|
|
# syft goes in the workspace, NOT /usr/local/bin. The host executor runs
|
|
# as root on the production gateway; a release should not leave binaries
|
|
# behind on it.
|
|
- name: SBOMs for every shipped image
|
|
run: |
|
|
set -eu
|
|
mkdir -p dist/sboms .tools
|
|
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh \
|
|
| sh -s -- -b .tools
|
|
for image in server frontend broker agent-base agent-browser; do
|
|
./.tools/syft "clawmates/$image:$VERSION" -o spdx-json \
|
|
> "dist/sboms/$image.spdx.json"
|
|
done
|
|
|
|
- name: Save image tarballs
|
|
run: |
|
|
set -eu
|
|
mkdir -p dist/images
|
|
docker save "clawmates/server:$VERSION" -o dist/images/server.tar
|
|
docker save "clawmates/frontend:$VERSION" -o dist/images/frontend.tar
|
|
docker save "clawmates/broker:$VERSION" -o dist/images/broker.tar
|
|
docker save "clawmates/agent-base:$VERSION" -o dist/images/agent-base.tar
|
|
docker save "clawmates/agent-browser:$VERSION" -o dist/images/agent-browser.tar
|
|
docker save tecnativa/docker-socket-proxy:0.3 -o dist/images/socket-proxy.tar
|
|
docker save postgres:16-alpine -o dist/images/postgres.tar
|
|
du -sh dist/images
|
|
|
|
# gw-04 has no cargo, so the bundler builds in a container — same pattern
|
|
# and same cache volumes as deploy.yml. The forge credential is here
|
|
# because cargo resolves the whole workspace, which includes cm-brain's
|
|
# private clawhdf5 git dependency.
|
|
- name: Build bundler
|
|
run: |
|
|
docker run --rm \
|
|
-v "$PWD":/w -w /w \
|
|
-v cm-ci-cargo-registry:/usr/local/cargo/registry \
|
|
-v cm-ci-cargo-git:/usr/local/cargo/git \
|
|
-v cm-ci-target:/w/target \
|
|
-e SQLX_OFFLINE=true -e CARGO_NET_GIT_FETCH_WITH_CLI=true \
|
|
-e FORGE_TOKEN='${{ secrets.FORGE_TOKEN }}' \
|
|
rust:1.96-slim \
|
|
sh -c 'set -e
|
|
apt-get update -qq
|
|
apt-get install -y -qq pkg-config libssl-dev cmake git >/dev/null
|
|
git config --global url."https://oauth2:[email protected]/".insteadOf "https://git.redclaw.dev/"
|
|
cargo build --release -p clawmates-bundler
|
|
# Copy the binary OUT of the target volume and into the workspace.
|
|
# /w/target is a named docker volume, so anything left there is
|
|
# invisible to later steps running on the host — which is exactly
|
|
# how this failed the first time (exit 127, No such file).
|
|
mkdir -p /w/.tools
|
|
cp target/release/clawmates-bundler /w/.tools/clawmates-bundler'
|
|
test -x .tools/clawmates-bundler || { echo "bundler did not land in the workspace"; exit 1; }
|
|
|
|
- name: Assemble and sign the bundle
|
|
env:
|
|
BUNDLE_SIGNING_KEY: ${{ secrets.BUNDLE_SIGNING_KEY }}
|
|
run: |
|
|
set -eu
|
|
test -n "$BUNDLE_SIGNING_KEY" || { echo "BUNDLE_SIGNING_KEY is empty"; exit 1; }
|
|
umask 077
|
|
printf '%s' "$BUNDLE_SIGNING_KEY" > /tmp/release.key
|
|
BUNDLER=.tools/clawmates-bundler
|
|
ARTIFACTS=""
|
|
for tar in dist/images/*.tar; do
|
|
ARTIFACTS="$ARTIFACTS $tar=images/$(basename "$tar")"
|
|
done
|
|
for migration in migrations/*.sql; do
|
|
ARTIFACTS="$ARTIFACTS $migration=migrations/$(basename "$migration")"
|
|
done
|
|
# shellcheck disable=SC2086
|
|
"$BUNDLER" assemble dist/bundle "$VERSION" /tmp/release.key \
|
|
deploy/compose/docker-compose.yml=compose/docker-compose.yml \
|
|
deploy/compose/clawmates.toml=compose/clawmates.toml \
|
|
deploy/compose/.env.example=compose/.env.example \
|
|
deploy/e2e/scenarios.toml=compose/scenarios.toml \
|
|
images/seccomp/agent-profile.json=seccomp/agent-profile.json \
|
|
deploy/airgapped/install.sh=install.sh \
|
|
"$BUNDLER"=bin/clawmates-bundler \
|
|
dist/sboms/server.spdx.json=sboms/server.spdx.json \
|
|
dist/sboms/frontend.spdx.json=sboms/frontend.spdx.json \
|
|
dist/sboms/agent-base.spdx.json=sboms/agent-base.spdx.json \
|
|
dist/sboms/agent-browser.spdx.json=sboms/agent-browser.spdx.json \
|
|
$ARTIFACTS
|
|
chmod +x dist/bundle/bin/clawmates-bundler dist/bundle/install.sh
|
|
rm -f /tmp/release.key
|
|
|
|
- name: Verify the bundle offline (public key only)
|
|
env:
|
|
BUNDLE_SIGNING_KEY: ${{ secrets.BUNDLE_SIGNING_KEY }}
|
|
run: |
|
|
set -eu
|
|
umask 077
|
|
printf '%s' "$BUNDLE_SIGNING_KEY" > /tmp/release.key
|
|
.tools/clawmates-bundler pubkey /tmp/release.key dist/release.pub
|
|
rm -f /tmp/release.key
|
|
# The customer's exact procedure: the public half only, inside a
|
|
# NETWORK-DISABLED container, proving verification needs no internet.
|
|
docker run --rm --network none \
|
|
-v "$PWD/dist:/dist:ro" \
|
|
ubuntu:24.04 \
|
|
/dist/bundle/bin/clawmates-bundler verify /dist/bundle /dist/release.pub
|
|
|
|
- name: Tarball
|
|
run: tar -C dist -czf "clawmates-bundle-$VERSION.tgz" bundle
|
|
|
|
# The clean-room install rehearsal is DELIBERATELY NOT RUN HERE.
|
|
#
|
|
# Every other step in this job is inert with respect to production: it
|
|
# builds images, writes SBOMs, signs a bundle, and verifies it in a
|
|
# network-isolated container. The rehearsal is the one step whose entire
|
|
# purpose is to stand a full stack UP and then tear it down with
|
|
# `down -v` — on the machine serving production.
|
|
#
|
|
# On 2026-08-13 it did exactly that: the bundled compose file declares
|
|
# `name: clawmates`, which beat --project-directory, so the rehearsal
|
|
# adopted the live stack and its teardown deleted clawmates_pgdata. The
|
|
# database was lost and there were no backups.
|
|
#
|
|
# scripts/rehearse-install.sh is now isolated (`-p rehearse-$$` plus a
|
|
# guard that refuses the production project name) and its health probe is
|
|
# fixed, so it is safe to run — just not on this host. Run it on a build
|
|
# box or throwaway VM:
|
|
#
|
|
# CLAWMATES_BUNDLER=… COMPOSE=/path/to/compose-v2 ./scripts/rehearse-install.sh
|
|
#
|
|
# Restore this step here only if the release ever moves off the gateway.
|
|
|
|
# Gitea's release API, not softprops/action-gh-release (GitHub-only).
|
|
# Create-or-reuse, so a re-run of the same tag updates instead of 409ing.
|
|
# Tag pushes only. On workflow_dispatch GITHUB_REF_NAME is the BRANCH, so
|
|
# this step previously created a release — and a git tag — literally named
|
|
# "main". A smoke-test run must not be able to mint a release.
|
|
- name: Attach to the Gitea release
|
|
if: github.ref_type == 'tag'
|
|
env:
|
|
FORGE_TOKEN: ${{ secrets.FORGE_TOKEN }}
|
|
run: |
|
|
set -eu
|
|
API="https://git.redclaw.dev/api/v1/repos/$GITHUB_REPOSITORY/releases"
|
|
TAG="${GITHUB_REF_NAME}"
|
|
id=$(curl -sS -H "Authorization: token $FORGE_TOKEN" "$API/tags/$TAG" \
|
|
| sed -n 's/.*"id":[ ]*\([0-9]\+\).*/\1/p' | head -1)
|
|
if [ -z "$id" ]; then
|
|
id=$(curl -sS -X POST -H "Authorization: token $FORGE_TOKEN" \
|
|
-H 'content-type: application/json' \
|
|
-d "{\"tag_name\":\"$TAG\",\"name\":\"$TAG\",\"body\":\"Air-gapped bundle for $TAG. Verify with the published public key before docker load.\"}" \
|
|
"$API" | sed -n 's/.*"id":[ ]*\([0-9]\+\).*/\1/p' | head -1)
|
|
fi
|
|
test -n "$id" || { echo "could not create or find the release for $TAG"; exit 1; }
|
|
for f in "clawmates-bundle-$VERSION.tgz" dist/release.pub; do
|
|
code=$(curl -sS -o /dev/null -w '%{http_code}' -X POST \
|
|
-H "Authorization: token $FORGE_TOKEN" \
|
|
-F "attachment=@$f" \
|
|
"$API/$id/assets?name=$(basename "$f")")
|
|
echo " attached $(basename "$f") (HTTP $code)"
|
|
case "$code" in 20*) ;; *) echo "attach failed"; exit 1 ;; esac
|
|
done
|
|
|
|
# Release artifacts are GBs of image tarballs on the production gateway.
|
|
# Never `docker image prune -a` here: clawmates/agent-*:dev exist in no
|
|
# registry and are the source of the microVM rootfs files.
|
|
- name: Reclaim disk
|
|
if: always()
|
|
run: |
|
|
rm -rf dist .tools "clawmates-bundle-$VERSION.tgz" || true
|
|
for i in server frontend broker agent-base agent-browser; do
|
|
docker rmi "clawmates/$i:$VERSION" 2>/dev/null || true
|
|
done
|
|
df -h / | awk 'NR==2{print " disk free: "$4}'
|