ci: breadcrumb which step dies
deploy / test (push) Failing after 7s
deploy / build (push) Skipped

The host-log change proved the job never reaches `cargo test` — rust.log is
absent while /tmp/ci-logs exists. But TWO steps create that directory, so
"the directory exists" does not say how far the job got, and that ambiguity
cost a debugging cycle on its own.

Each step now overwrites /tmp/ci-logs/STEP on entry, so the last value names
the step that died. The postgres step also runs under `set -x`.

Verified manually on gw-04 in the meantime: the postgres step's exact
commands succeed there (STEP_RC=0), as does the whole Rust container
command with cargo's real exit code, as do the three frontend commands. So
the failure is something the job does that running its steps by hand does
not reproduce — which is precisely what a breadcrumb answers and guessing
does not.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-20 07:19:29 -07:00
co-authored by Claude Opus 5
parent a864f2ccc7
commit 24393819bd
+13 -3
View File
@@ -55,7 +55,14 @@ jobs:
run: |
# Where every step leaves its full output, on the HOST, so a failed
# run can be read afterwards without the actions-log API.
mkdir -p /tmp/ci-logs && rm -f /tmp/ci-logs/*.log
#
# STEP is a breadcrumb: each step overwrites it on entry, so the last
# value names the step that died. Two steps used to create this
# directory, which made "the directory exists" ambiguous about how far
# the job got — and that ambiguity cost a whole debugging cycle.
mkdir -p /tmp/ci-logs && rm -f /tmp/ci-logs/*.log /tmp/ci-logs/STEP
echo "1-start-postgres" > /tmp/ci-logs/STEP
set -x
# Run-scoped name. `cm-ci-pg` was shared by every run, so a second
# run removed the first one's database while it was still being used.
# The concurrency group above should prevent overlap; this makes the
@@ -106,6 +113,7 @@ jobs:
# The token is a repo secret, so it is masked in logs and never in git.
- name: Rust tests
run: |
echo "2-rust" > /tmp/ci-logs/STEP
docker run --rm --network host \
-v "$PWD":/w -w /w \
-v cm-ci-cargo-registry:/usr/local/cargo/registry \
@@ -149,13 +157,15 @@ jobs:
# 38 GB of leaked volumes before anyone noticed.
- name: Stop test Postgres
if: always()
run: docker rm -fv "$PG" 2>/dev/null || true
run: |
echo "3-stop-postgres" >> /tmp/ci-logs/STEP
docker rm -fv "$PG" 2>/dev/null || true
# node 22 is on the host, so these run directly.
- name: Frontend checks
working-directory: frontend
run: |
mkdir -p /tmp/ci-logs
echo "4-frontend" >> /tmp/ci-logs/STEP
set +e
npm ci --no-audit --no-fund > /tmp/ci-logs/npm-ci.log 2>&1; ci=$?
npm run typecheck > /tmp/ci-logs/typecheck.log 2>&1; tc=$?