ci: make a failed run readable, and stop laundering cargo's exit code
deploy / test (push) Failing after 8s
deploy / build (push) Skipped

Three runs failed and I debugged all three blind: Gitea's actions-log API
returns 403 for the token we have, so the only evidence was the word
"failure". I twice inferred a cause from that and was twice wrong — first
node, then dash — and a third theory (two runs stomping each other) was
right about a real defect but not about these failures.

Worse, my own reproduction lied. It ran `cargo test ... | tail -80`, so the
reported exit code was TAIL's. A green pipeline over a red suite is exactly
the trap this repo already documents, and I walked into it while hunting a
red build.

  - every step writes its full output to /tmp/ci-logs on the RUNNER HOST,
    which outlives the container, so a failure can be read afterwards
  - the Rust step captures cargo's status in a variable and exits with it,
    with the grep and tail in between — no pipe anywhere near the status
  - the frontend step runs npm ci / typecheck / vitest separately, keeps
    each status, prints all three tails, and fails if any is non-zero.
    Previously a `set -e` abort meant later steps produced no output at all

What is now known, verified on the runner host itself with cargo's real
exit code: `cargo test --workspace` PASSES on gw-04 in the CI container
against a CI-shaped Postgres (CARGO_RC=0), and `npm ci`, `typecheck` and
`vitest` all pass there too. So the failing step is not one of those, and
the next run will say which it is instead of leaving it to be guessed.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-20 07:17:39 -07:00
co-authored by Claude Opus 5
parent 72ba4ba523
commit a864f2ccc7
+27 -4
View File
@@ -53,6 +53,9 @@ jobs:
# only means "no database here".
- name: Start test Postgres
run: |
# Where every step leaves its full output, on the HOST, so a failed
# run can be read afterwards without the actions-log API.
mkdir -p /tmp/ci-logs && rm -f /tmp/ci-logs/*.log
# Run-scoped name. `cm-ci-pg` was shared by every run, so a second
# run removed the first one's database while it was still being used.
# The concurrency group above should prevent overlap; this makes the
@@ -114,6 +117,7 @@ jobs:
-e CARGO_NET_GIT_FETCH_WITH_CLI=true \
-e FORGE_TOKEN='${{ secrets.FORGE_TOKEN }}' \
-e CM_TEST_DATABASE_URL=postgres://postgres:[email protected]:55432/postgres \
-v /tmp/ci-logs:/cilog \
rust:1.96-slim \
sh -c 'set -e
apt-get update -qq
@@ -125,7 +129,19 @@ jobs:
# which is how this was found, on the first push that carried them.
apt-get install -y -qq pkg-config libssl-dev cmake git nodejs >/dev/null
git config --global url."https://oauth2:[email protected]/".insteadOf "https://git.redclaw.dev/"
cargo test --workspace'
# Full output to a host-mounted file, then the tail, then exit
# with CARGO's status. Piping cargo into `tail` would report
# tail's exit code — a green job over a red suite. The log
# survives the container so a failure is diagnosable at all:
# Gitea's actions-log API returns 403 for our token, and three
# failed runs were debugged blind before this existed.
set +e
cargo test --workspace > /cilog/rust.log 2>&1
rc=$?
set -e
grep -nE "test result: FAILED|^error(\[|:)|panicked at" /cilog/rust.log | head -40 || true
tail -40 /cilog/rust.log
exit $rc'
# -v, not just -f. The postgres image declares a VOLUME, so removing the
# container without it orphans an anonymous data directory EVERY run.
@@ -139,9 +155,16 @@ jobs:
- name: Frontend checks
working-directory: frontend
run: |
npm ci --no-audit --no-fund
npm run typecheck
npm run test
mkdir -p /tmp/ci-logs
set +e
npm ci --no-audit --no-fund > /tmp/ci-logs/npm-ci.log 2>&1; ci=$?
npm run typecheck > /tmp/ci-logs/typecheck.log 2>&1; tc=$?
npm run test > /tmp/ci-logs/vitest.log 2>&1; vt=$?
set -e
for f in npm-ci typecheck vitest; do
printf '=== %s ===\n' "$f"; tail -25 "/tmp/ci-logs/$f.log" || true
done
[ "$ci" -eq 0 ] && [ "$tc" -eq 0 ] && [ "$vt" -eq 0 ]
# Lint is advisory: the repo currently has pre-existing max-lines and
# set-state-in-effect errors that predate this pipeline. Failing the
# deploy on them would mean nothing could ship until they are cleared.