From 70dbee662da4f3bfb7be37df27cabd9b07bfda3d Mon Sep 17 00:00:00 2001 From: Omar Sobh Date: Tue, 22 Sep 2026 09:42:07 -0500 Subject: [PATCH] harness(rolepolicy): prove the DEPLOYED gate enforces the role policy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The role policy was unit-tested against the script the code generates. This probes the script the SERVER INSTALLED, inside a live mission container: the verifier's Write exits 2 with its reason, the lead's identical Write exits 0, the verifier's Read and another role's Edit exit 0, and the denial the deployed gate wrote names role-verifier-readonly and agent_type verifier. 5/5 on prod. Three of the four probes are negative controls. A gate that refused everything would pass the first and be worthless — the same trade the module's header refuses. 'Compiled in and CI-green' and 'enforced by the artifact in production' are different claims; the gap between them is this module's history. The record is matched with a shell glob on the raw JSON line, not a nested python -c: the first version could not survive quoting through bash, ssh and sh, and reported an empty record while the gate had written a correct one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz --- docs/NEXT-SESSION.md | 13 +++++ scripts/verify-mission-delivery.sh | 90 +++++++++++++++++++++++++++++- 2 files changed, 102 insertions(+), 1 deletion(-) diff --git a/docs/NEXT-SESSION.md b/docs/NEXT-SESSION.md index 8fa1d44..518e441 100644 --- a/docs/NEXT-SESSION.md +++ b/docs/NEXT-SESSION.md @@ -535,6 +535,19 @@ Rendered into the same guest script as the floor, so the shell and the Rust predicate cannot disagree; shell tests run the generated script and check that the lead's identical write is still allowed. +**Proven against the deployed artifact**, not only the unit-tested one +(`rolepolicy` scenario, 5/5): payloads piped through the gate script the +server actually installed, inside a live mission container. The verifier's +`Write` exits 2 with the reason; the lead's identical `Write` exits 0; the +verifier's `Read` and another role's `Edit` exit 0; and the denial the +deployed gate wrote names `rule: role-verifier-readonly` and +`agent_type: verifier`. Three of the four probes are negative controls on +purpose — a gate that refused everything would pass the first one and be +worthless. "Compiled in and CI-green" and "enforced by the artifact in +production" are different claims, and the gap between them is this +module's whole history (a gate installed and inert; an `--agents` list +that silently did nothing until 2.1.243 rejected it). + **The real gap the paper names, and we do not have:** `arg_provenance`. Our gate sees a command string and cannot tell a URL the operator supplied from one a fetched web page supplied — so "no outbound action derived from diff --git a/scripts/verify-mission-delivery.sh b/scripts/verify-mission-delivery.sh index 21fe4b5..f0ebef2 100755 --- a/scripts/verify-mission-delivery.sh +++ b/scripts/verify-mission-delivery.sh @@ -1365,6 +1365,90 @@ scenario_door() { ssh "$HOST" "docker exec clawmates_postgres_1 psql -U postgres -d clawmates -tAc \"delete from outbox where subject in ('harness: judge spend','Onboarding') and recipient in ('ops@redclaw.dev','contractor@freelance.example');\"" >/dev/null } +# ── Scenario: rolepolicy — the DEPLOYED gate enforces a role's limits ── +# +# The gate's role policy is unit-tested against the script the code +# generates. This checks the script the SERVER INSTALLED, inside a real +# mission container, by piping hook payloads through it — because "compiled +# in and CI-green" and "enforced by the artifact in production" are +# different claims, and this module's history is full of the gap between +# them (a gate installed and inert, an --agents list that silently did +# nothing until 2.1.243 rejected it). +# +# Three of the four probes are negative controls. A gate that refused +# everything would pass the first one and is worthless: the lead's +# identical write, the verifier's read and another role's edit must all +# still go through. + +ROLEPROBE_BODY=$(cat </dev/null + + # The container is the artifact under test; it exists only while the + # mission runs, so this waits for the gate file rather than for the + # mission, and gives up loudly rather than reporting a pass it did not earn. + container="cm-runtime-mission-$(printf '%s' "$mission" | tr -d -)" + waited=0 + while [ "$waited" -lt 300 ]; do + ssh "$HOST" "docker exec $container test -f /root/toolhooks/tool-gate.sh" 2>/dev/null && break + sleep 5; waited=$((waited + 5)) + done + if ! ssh "$HOST" "docker exec $container test -f /root/toolhooks/tool-gate.sh" 2>/dev/null; then + norun "rolepolicy: no gate script in $container after ${waited}s — nothing to probe" + return 1 + fi + + # `probe