osobh/clawmates
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(harness): the verdict check matched psql's display form, not the query's

Browse Source 
`select met || ' ' || independent` casts the booleans to `true`/`false`, but the
pattern matched `t`/`f` — psql's *column display* form. So the check reported "no
verdict recorded for the phase" while the row sat in the table saying met=true,
independent=true, glm-4.7.

A check that fails for a reason unrelated to what it checks is worse than no check:
it trains you to ignore the output. The booleans are cast explicitly now so the
shape cannot drift again, and the failure message prints what it actually got.

`verify-mission-delivery.sh microvm` now passes 5/5 against production:
  - the agent ran under guest kernel 6.1.128, not the gateway's 6.8.0-124 or the
    node's 7.0.0-28 — the one assertion that cannot pass by accident
  - the lead delegated to 1 subagent
  - the condition was met and judged INDEPENDENTLY by glm-4.7
  - the checkout has exactly one writer (uid 65532)
  - negative control: a backend no node can run is refused at launch
This commit is contained in:
Omar Sobh
2026-08-05 22:44:25 -07:00
parent 9aed20b6d0
commit b17e18aa67
1 changed files with 10 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/verify-mission-delivery.sh
+10 -5
View File
@@ -319,14 +319,19 @@ assert_microvm() { # <token> <mission> <report>
# And the verdict: judged, and by whom. `independent` is only true when the # And the verdict: judged, and by whom. `independent` is only true when the
# judge came from a different provider family than the agent. # judge came from a different provider family than the agent.
indep=$(ssh "$HOST" "docker exec clawmates_postgres_1 psql -U postgres -d clawmates -tAc \ indep=$(ssh "$HOST" "docker exec clawmates_postgres_1 psql -U postgres -d clawmates -tAc \
\"select met || ' ' || independent || ' ' || model from mission_phase_evaluations \ \"select met::text || ' ' || independent::text || ' ' || model from mission_phase_evaluations \
where phase_id in (select id from mission_phases where mission_id='$mission') \ where phase_id in (select id from mission_phases where mission_id='$mission') \
order by created_at desc limit 1;\"" | head -1 | tr -d '\r') order by created_at desc limit 1;\"" | head -1 | tr -d '\r')
# `true`/`false`, not psql's display `t`/`f`: concatenating a boolean with text
# casts it to its full spelling. The first version matched "t t " and therefore
# never matched a real verdict, reporting "no verdict recorded" while the row was
# sitting in the table — a check that failed for a reason that had nothing to do
# with what it was checking.
case "$indep" in case "$indep" in
"t t "*) pass "microvm: condition met, judged independently (${indep##* })" ;; "true true "*) pass "microvm: condition met, judged independently (${indep##* })" ;;
"t f "*) info "microvm: condition met but judged by the agent's own provider family (${indep##* }) — set CLAWMATES_VALIDATOR_MODEL" ;; "true false "*) info "microvm: condition met but judged by the agent's own provider family (${indep##* }) — set CLAWMATES_VALIDATOR_MODEL for an independent check" ;;
"f "*) fail "microvm: the judge says the condition was NOT met: $indep" ;; "false "*) fail "microvm: the judge says the condition was NOT met: $indep" ;;
*) fail "microvm: no verdict recorded for the phase (done_when was set)" ;; *) fail "microvm: no verdict recorded for the phase (done_when was set), got: ${indep:-<empty>}" ;;
esac esac
} }