docs: the judge is back and the subagent path is no longer a claim
The z.ai quota reset on schedule. glm-5.3 answers on the same key and has since passed a real done_when — phase completed on iteration 0, with the verdict naming the arXiv ids it checked rather than waving the phase through. Mission 01a07498 was the failed validation run plus one change, and it closed the honest negative the last handoff recorded: 87 tool calls, 43 from the main turn and 44 across 4 general-purpose subagents, 4 distinct subagent_ids against 4 Agent spawns. Before this the field was correct in unit tests and had never been watched writing. The one change was the finding. The earlier task invited delegation and got none; naming the tool and forbidding the single-turn shortcut produced four spawns from the same recipe and the same delivery arm. A fan-out path that is merely invited measures nothing. Also records that postgres is clawmates-postgres-1 locally and clawmates_postgres_1 on gw-04 — the wrong one reports "No such container", which reads like a down stack rather than a typo. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
co-authored by
Claude Opus 5
parent
daf8d12157
commit
2a3409ec53
+32
-42
@@ -1,39 +1,32 @@
|
|||||||
# Where this left off — 2026-08-29
|
# Where this left off — 2026-09-05
|
||||||
|
|
||||||
Read `MISSION-EGRESS.md` for what a mission container can reach,
|
`MISSION-EGRESS.md` has what a mission container can reach, `UPSTREAM-SCAN.md`
|
||||||
`UPSTREAM-SCAN.md` for the fork-vs-upstream position, and
|
the fork-vs-upstream position, `SKILL-USE-BASELINE.md` the delivery measurement.
|
||||||
`SKILL-USE-BASELINE.md` for the delivery measurement.
|
|
||||||
|
|
||||||
## Read this first — the judge is down until 2026-09-04
|
## Read this first — the judge is back, and the subagent path is proven
|
||||||
|
|
||||||
```
|
The z.ai quota reset on schedule. `glm-5.3` answers again on the same key
|
||||||
429 Too Many Requests
|
(`sha256[0:12] = 616fb5076696`), and it has since passed a real `done_when`, not
|
||||||
[1310][Weekly/Monthly Limit Exhausted. Your limit will reset at 2026-09-04 10:01:33]
|
just a ping. **Nothing is blocked on the judge any more.**
|
||||||
```
|
|
||||||
|
|
||||||
`glm-5.3` is the default validator on **both** stacks — neither sets
|
Mission `01a07498-d343-7953-9ed1-8ea9ba83b140` (2026-09-05, local, `index` arm)
|
||||||
`validator_model`, and the `ZAI_API_KEY` fingerprints are identical
|
was the previous validation run plus one change — a task that EXPLICITLY
|
||||||
(`sha256[0:12] = 616fb5076696`). So **every mission that declares a `done_when`
|
instructs `Agent` delegation — and it closed both items the last pass left open:
|
||||||
will fail its evaluation on local AND production until the quota resets.**
|
|
||||||
|
|
||||||
The code is behaving correctly and this is worth understanding before "fixing"
|
| | |
|
||||||
it. `evaluator.rs:480` deliberately refuses to fall back to the agent's own
|
|---|---|
|
||||||
provider, because a verdict produced by the same family the agent ran on would
|
| judge recovered | **CONFIRMED** — phase `completed` on iteration 0, first attempt. The verdict cites the actual arXiv ids it checked and the "what could not be established" section by name. A substantive read, not a rubber stamp. |
|
||||||
claim an independence it does not have. The phase stays unmet, says why, and the
|
| **subagent attribution** | **OBSERVED, finally.** 87 tool calls: 43 from the main turn, **44 across 4 `general-purpose` subagents**, 4 distinct `subagent_id`s matching 4 `Agent` spawns. The field had only ever been unit-tested before this. |
|
||||||
sweep retries. Three retries, three identical 429s, then `failed`.
|
|
||||||
|
|
||||||
Unlike the subscription-OAuth 429 (which was a malformed system prompt), this one is REAL: it carries
|
The delegation had to be *demanded*. The earlier task invited it and got none;
|
||||||
code 1310 and a reset date. Do not go looking for a malformed prompt.
|
naming the tool and saying "do not gather every approach yourself in a single
|
||||||
|
turn" produced four spawns. Worth remembering when designing any run that needs
|
||||||
Options, with the trade stated:
|
to exercise the fan-out path.
|
||||||
|
|
||||||
1. Top up the z.ai quota — restores the judge we actually measured.
|
|
||||||
2. Wait until 2026-09-04 — nothing else breaks; `done_when` phases just can't pass.
|
|
||||||
3. Switch to kimi — available now, and our own eval found it produces FALSE
|
|
||||||
POSITIVES including passing deleted tests. Worse than no judge. Not advised.
|
|
||||||
4. The Anthropic subscription judge — works, but same family as the agent, so
|
|
||||||
the verdict loses the independence the design exists for.
|
|
||||||
|
|
||||||
|
The 429 that blocked the last pass was real (code `1310`, reset timestamp) and
|
||||||
|
the code was right to fail closed — `evaluator.rs:480` still refuses a
|
||||||
|
same-family fallback, and should keep refusing. If a `done_when` phase fails
|
||||||
|
again with the artifacts present and correct, check the quota before the mission.
|
||||||
## What shipped this pass
|
## What shipped this pass
|
||||||
|
|
||||||
Seven commits on `main`, all deployed to gw-04 (server recreated
|
Seven commits on `main`, all deployed to gw-04 (server recreated
|
||||||
@@ -91,35 +84,28 @@ A validation mission ran locally on the `index` arm
|
|||||||
| the corrected gate | **CONFIRMED installed** — new rules present in the live container, node present, no inert marker, 0 denials against 23 body-free `Bash` calls |
|
| the corrected gate | **CONFIRMED installed** — new rules present in the live container, node present, no inert marker, 0 denials against 23 body-free `Bash` calls |
|
||||||
| attribution | 34/34, third perfect run |
|
| attribution | 34/34, third perfect run |
|
||||||
| `subagent` column | written, all null — **correct**, no subagents spawned |
|
| `subagent` column | written, all null — **correct**, no subagents spawned |
|
||||||
| **subagent path** | **STILL UNEXERCISED.** These agents did not delegate. The mechanism is proven against the real binary and unit-tested; it has never been watched populating `mission_events`. |
|
| **subagent path** | unexercised in THIS run; **closed on 2026-09-05** by `01a07498` — see the top of this file. |
|
||||||
|
|
||||||
To close it: run one mission whose task EXPLICITLY instructs delegation. That is
|
|
||||||
what forces the path; a task that merely invites it does not.
|
|
||||||
|
|
||||||
## Open, in the order I would take them
|
## Open, in the order I would take them
|
||||||
|
|
||||||
1. **The judge quota.** Everything with a `done_when` is blocked until it is
|
1. **Confirm the last-run cards in the prod UI.** They read HISTORICAL data, so
|
||||||
resolved. See the top of this file.
|
|
||||||
2. **Watch the subagent field populate.** One mission, task explicitly
|
|
||||||
instructing `Agent` use.
|
|
||||||
3. **Confirm the last-run cards in the prod UI.** They read HISTORICAL data, so
|
|
||||||
unlike everything else this pass they should work right now against the two
|
unlike everything else this pass they should work right now against the two
|
||||||
2026-08-25 missions. If blank, suspect the roster join: `agent_last_run`
|
2026-08-25 missions. If blank, suspect the roster join: `agent_last_run`
|
||||||
reaches agents via `team_members → mission_teams → missions`, so an agent no
|
reaches agents via `team_members → mission_teams → missions`, so an agent no
|
||||||
longer on the mission's team will not resolve.
|
longer on the mission's team will not resolve.
|
||||||
4. **Three fork items, all behind one runtime image rebuild.** Branch
|
2. **Three fork items, all behind one runtime image rebuild.** Branch
|
||||||
`port/config-write-lock` on the zeroclaw fork carries upstream `841f28c7f1`
|
`port/config-write-lock` on the zeroclaw fork carries upstream `841f28c7f1`
|
||||||
(cherry-picked CLEANLY despite being 331 behind; `handle_prop_put` — the
|
(cherry-picked CLEANLY despite being 331 behind; `handle_prop_put` — the
|
||||||
endpoint ClawMates writes through — now takes the lock). Not merged, not
|
endpoint ClawMates writes through — now takes the lock). Not merged, not
|
||||||
deployed. Still to take: credential-fragment redaction (`eadaee0b62`) and
|
deployed. Still to take: credential-fragment redaction (`eadaee0b62`) and
|
||||||
`/api/pair` lockout hardening (`47adb9863e`). Tank is offline, so the rebuild
|
`/api/pair` lockout hardening (`47adb9863e`). Tank is offline, so the rebuild
|
||||||
has to happen on gw-04 and push to the web-01 registry.
|
has to happen on gw-04 and push to the web-01 registry.
|
||||||
5. **Mission egress remediation** — written up in `MISSION-EGRESS.md`,
|
3. **Mission egress remediation** — written up in `MISSION-EGRESS.md`,
|
||||||
deliberately NOT applied on the operator's call. A mission container reaches
|
deliberately NOT applied on the operator's call. A mission container reaches
|
||||||
the entire tailnet and host SSH.
|
the entire tailnet and host SSH.
|
||||||
6. **The 331-commit upstream merge.** 18 overlapping files; `claude_cli.rs`
|
4. **The 331-commit upstream merge.** 18 overlapping files; `claude_cli.rs`
|
||||||
cannot conflict (zero upstream files).
|
cannot conflict (zero upstream files).
|
||||||
7. **Two silent-discard defects found by sweep, not yet fixed.**
|
5. **Two silent-discard defects found by sweep, not yet fixed.**
|
||||||
`container_tool_hooks::install`'s outcome is discarded at both call sites and
|
`container_tool_hooks::install`'s outcome is discarded at both call sites and
|
||||||
recorded nowhere, so "did this mission run gated?" is unanswerable once the
|
recorded nowhere, so "did this mission run gated?" is unanswerable once the
|
||||||
container is reaped — and `PreToolUse` is now a security control, not just
|
container is reaped — and `PreToolUse` is now a security control, not just
|
||||||
@@ -136,6 +122,10 @@ what forces the path; a task that merely invites it does not.
|
|||||||
apart from the last-run cards, unexercised in prod.
|
apart from the last-run cards, unexercised in prod.
|
||||||
- **Fleet offline 17 days** — `architect`, `morpheus`, `tank`; `nodes` table is
|
- **Fleet offline 17 days** — `architect`, `morpheus`, `tank`; `nodes` table is
|
||||||
empty. microVM tier stays blocked.
|
empty. microVM tier stays blocked.
|
||||||
|
- **Postgres is named differently on each stack.** Locally it is
|
||||||
|
`clawmates-postgres-1` (dashes); on gw-04 it is `clawmates_postgres_1`
|
||||||
|
(underscores). Same for `server`/`frontend`. Using the wrong one gives
|
||||||
|
"No such container", which reads like a down stack rather than a typo.
|
||||||
- **`target/` is a symlink to `/Volumes/NVMeRAID`.** If it detaches mid-run,
|
- **`target/` is a symlink to `/Volumes/NVMeRAID`.** If it detaches mid-run,
|
||||||
cargo dies with SIGKILL and then `Not a directory`. It is the drive, not a
|
cargo dies with SIGKILL and then `Not a directory`. It is the drive, not a
|
||||||
flaky test.
|
flaky test.
|
||||||
|
|||||||
Reference in New Issue
Block a user