docs: the open list as of 2026-09-20, and addendum 3
Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
co-authored by
Claude Opus 5
parent
1a44405308
commit
ef1f21024d
+56
-21
@@ -100,28 +100,36 @@ guessed family.
|
|||||||
`worker`, names that promised providers they never used); map routes
|
`worker`, names that promised providers they never used); map routes
|
||||||
`researcher`/`analyst` to `worker` directly.
|
`researcher`/`analyst` to `worker` directly.
|
||||||
|
|
||||||
## Open, in the order I would take them
|
## Open, in the order I would take them (rewritten 2026-09-20)
|
||||||
|
|
||||||
1. **The judge runs to its check cap almost every time.** Compaction made
|
Everything on the 09-14 list is closed or explained; see the addenda. What is
|
||||||
that cheaper; it did not ask why a research verdict needs 12 commands.
|
genuinely left:
|
||||||
Watch `requests` per verdict on the next few missions. If it stays at the
|
|
||||||
cap, the lever is the judge prompt, not the budget.
|
1. **The judge costs ~9 requests / ~20 K input per verdict, and that is now
|
||||||
2. ~~**microVM tier is unexercised since v0.8.5.**~~ Closed 2026-09-18: three
|
legitimate work.** Three measured passes (09-19): the 12 KB output window
|
||||||
VM missions on Claude Code 2.1.276 — claude `01a0b75e`, glm `01a0b761`,
|
truncated an 18 KB deliverable → 64 KB; then my compaction erased the read
|
||||||
kimi `01a0b768` — 28/28 checks including provider egress proof. See the
|
before the judge could use it → the latest round stays whole; then the
|
||||||
addendum. architect rebuilt to 2.1.276 the same evening (claude +
|
prompt says a cat is complete, decide first, read once. Result: four cats
|
||||||
local-ornith, selftest real turn PASS), so placement no longer decides the
|
of the deliverables plus cheap greps that VERIFY (placeholders, URL count,
|
||||||
CLI version.
|
sources, structure), zero re-reads. The only remaining lever is getting
|
||||||
3. **`files` is n=3.** 7 of 9 is a signal. `structured-paper-summary` went
|
the model to batch independent greps in one turn — a behaviour bet.
|
||||||
unread in 2 of the last 3 runs; the skills section sits 87–90% into the
|
2. **`files` arm: 8 of 12 across four runs; the same skill is skipped every
|
||||||
prompt. Position is the untested lever.
|
time.** With the section moved to 2% of the prompt (a50c41a) the evidence
|
||||||
4. **Compliance checks exist for 6 of 53 skills.** The rest score
|
checker still does not open `structured-paper-summary`, whose
|
||||||
`not_applicable` on that axis forever.
|
`when_to_use` is "summarising a research paper". That is triage, not a
|
||||||
5. **The persistent `clawmates-runtime` container's own config** still names
|
miss. Nothing to fix; the scorer already says `not_applicable`.
|
||||||
`worker_glm`/`worker_kimi` (3 mentions). Not on the mission path; editing
|
3. **Compliance checks: 11 of 53.** The remaining 42 are content judgement.
|
||||||
it restarts the paired runtime.
|
Adding a check for one means finding a rule that leaves a mark in tool
|
||||||
6. **Prod holds our 7 test missions** with 90-day retention. They are the
|
arguments or delivered files — the module's own bar, and the right one.
|
||||||
evidence for everything above; wipe them when they stop being.
|
4. **Follow-ups deliberately not taken:** `Spend{provider}` for VM turns
|
||||||
|
(requested-not-answered; `charge()` bills a credit for zero tokens — the
|
||||||
|
node egress log is the honest source and the harness asserts it); the
|
||||||
|
persistent runtime's own config still naming `worker_glm`/`worker_kimi`
|
||||||
|
(cosmetic; editing it restarts the paired runtime).
|
||||||
|
5. **Product/scaling work that was always scoped later:** missions-as-workflows
|
||||||
|
#14 (plan viewer + planner mode) and #15 (scheduling + loop-progress
|
||||||
|
events); scaling Phase 2/3 (node bring-up automation, Postgres HA,
|
||||||
|
autoscale); node-placed terminal cross-container writes.
|
||||||
|
|
||||||
## State you should know about
|
## State you should know about
|
||||||
|
|
||||||
@@ -220,3 +228,30 @@ all ours.
|
|||||||
Rebuild trap: rsync copies the Mac's dead `target` symlink to the node —
|
Rebuild trap: rsync copies the Mac's dead `target` symlink to the node —
|
||||||
`rm -f ~/clawmates/target` before `cargo build`.
|
`rm -f ~/clawmates/target` before `cargo build`.
|
||||||
|
|
||||||
|
## Addendum 3 — 2026-09-19/20
|
||||||
|
|
||||||
|
- `--agents` `tools` must be a JSON array (cbc9c2d); every CLI before 2.1.243
|
||||||
|
dropped the string form silently, so the verifier's tool restriction had
|
||||||
|
plausibly never applied on a VM. Caught by the canary.
|
||||||
|
- `evaluator::implementer_family(missions.backend)` — a glm mission is judged
|
||||||
|
by the subscription, honestly independent (794f212).
|
||||||
|
- `checkpoint.vm` on `topology_runs`: rootfs + guest `claude --version` per
|
||||||
|
VM run; `verify-mission-delivery.sh glm|kimi` with `assert_provider_egress`.
|
||||||
|
- VM tool-gate `denied.jsonl`/`inert` → `gate.denied`/`gate.inert` (9fc904a).
|
||||||
|
- Judge: 64 KB window (4e342d1), latest round whole + budget prompt (1a44405).
|
||||||
|
- Skills section second in the prompt (a50c41a); 4 more compliance checks
|
||||||
|
(507d744).
|
||||||
|
- `cm-files` MinIO test pinned to quay.io — Docker Hub deleted `minio/minio`;
|
||||||
|
CI passed only on gw-04's year-old cache (5d9edd6).
|
||||||
|
- **CI trap:** a newer push cancels the in-progress run (Gitea concurrency).
|
||||||
|
Four commits in ten minutes looked like three failures and one real one;
|
||||||
|
the one real one was environmental (gw-04 at 85% disk, 51 GB build cache —
|
||||||
|
reclaimed 44 GB, re-run green). Push, then wait.
|
||||||
|
- Both stacks wiped to zero 09-19 via the API; `_outputs` came with the
|
||||||
|
missions this time. Judge ledger kept (15 rows prod, 3 local).
|
||||||
|
- Fleet: tank claude/glm/kimi/local-ornith and architect claude/local-ornith
|
||||||
|
all on 2.1.276; backups deleted after sign-off.
|
||||||
|
- **The Mac's Tailscale switched to another tailnet** (`taila5f63e.ts.net`,
|
||||||
|
`noc@alucio.net.id`) at some point on 09-19/20 — zero fleet peers from the
|
||||||
|
laptop. `ssh gw-04-pub` etc. are the route until it is switched back.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user