docs: the open list as of 2026-09-20, and addendum 3
Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz
This commit is contained in:
co-authored by
Claude Opus 5
parent
1a44405308
commit
ef1f21024d
+56
-21
@@ -100,28 +100,36 @@ guessed family.
|
||||
`worker`, names that promised providers they never used); map routes
|
||||
`researcher`/`analyst` to `worker` directly.
|
||||
|
||||
## Open, in the order I would take them
|
||||
## Open, in the order I would take them (rewritten 2026-09-20)
|
||||
|
||||
1. **The judge runs to its check cap almost every time.** Compaction made
|
||||
that cheaper; it did not ask why a research verdict needs 12 commands.
|
||||
Watch `requests` per verdict on the next few missions. If it stays at the
|
||||
cap, the lever is the judge prompt, not the budget.
|
||||
2. ~~**microVM tier is unexercised since v0.8.5.**~~ Closed 2026-09-18: three
|
||||
VM missions on Claude Code 2.1.276 — claude `01a0b75e`, glm `01a0b761`,
|
||||
kimi `01a0b768` — 28/28 checks including provider egress proof. See the
|
||||
addendum. architect rebuilt to 2.1.276 the same evening (claude +
|
||||
local-ornith, selftest real turn PASS), so placement no longer decides the
|
||||
CLI version.
|
||||
3. **`files` is n=3.** 7 of 9 is a signal. `structured-paper-summary` went
|
||||
unread in 2 of the last 3 runs; the skills section sits 87–90% into the
|
||||
prompt. Position is the untested lever.
|
||||
4. **Compliance checks exist for 6 of 53 skills.** The rest score
|
||||
`not_applicable` on that axis forever.
|
||||
5. **The persistent `clawmates-runtime` container's own config** still names
|
||||
`worker_glm`/`worker_kimi` (3 mentions). Not on the mission path; editing
|
||||
it restarts the paired runtime.
|
||||
6. **Prod holds our 7 test missions** with 90-day retention. They are the
|
||||
evidence for everything above; wipe them when they stop being.
|
||||
Everything on the 09-14 list is closed or explained; see the addenda. What is
|
||||
genuinely left:
|
||||
|
||||
1. **The judge costs ~9 requests / ~20 K input per verdict, and that is now
|
||||
legitimate work.** Three measured passes (09-19): the 12 KB output window
|
||||
truncated an 18 KB deliverable → 64 KB; then my compaction erased the read
|
||||
before the judge could use it → the latest round stays whole; then the
|
||||
prompt says a cat is complete, decide first, read once. Result: four cats
|
||||
of the deliverables plus cheap greps that VERIFY (placeholders, URL count,
|
||||
sources, structure), zero re-reads. The only remaining lever is getting
|
||||
the model to batch independent greps in one turn — a behaviour bet.
|
||||
2. **`files` arm: 8 of 12 across four runs; the same skill is skipped every
|
||||
time.** With the section moved to 2% of the prompt (a50c41a) the evidence
|
||||
checker still does not open `structured-paper-summary`, whose
|
||||
`when_to_use` is "summarising a research paper". That is triage, not a
|
||||
miss. Nothing to fix; the scorer already says `not_applicable`.
|
||||
3. **Compliance checks: 11 of 53.** The remaining 42 are content judgement.
|
||||
Adding a check for one means finding a rule that leaves a mark in tool
|
||||
arguments or delivered files — the module's own bar, and the right one.
|
||||
4. **Follow-ups deliberately not taken:** `Spend{provider}` for VM turns
|
||||
(requested-not-answered; `charge()` bills a credit for zero tokens — the
|
||||
node egress log is the honest source and the harness asserts it); the
|
||||
persistent runtime's own config still naming `worker_glm`/`worker_kimi`
|
||||
(cosmetic; editing it restarts the paired runtime).
|
||||
5. **Product/scaling work that was always scoped later:** missions-as-workflows
|
||||
#14 (plan viewer + planner mode) and #15 (scheduling + loop-progress
|
||||
events); scaling Phase 2/3 (node bring-up automation, Postgres HA,
|
||||
autoscale); node-placed terminal cross-container writes.
|
||||
|
||||
## State you should know about
|
||||
|
||||
@@ -220,3 +228,30 @@ all ours.
|
||||
Rebuild trap: rsync copies the Mac's dead `target` symlink to the node —
|
||||
`rm -f ~/clawmates/target` before `cargo build`.
|
||||
|
||||
## Addendum 3 — 2026-09-19/20
|
||||
|
||||
- `--agents` `tools` must be a JSON array (cbc9c2d); every CLI before 2.1.243
|
||||
dropped the string form silently, so the verifier's tool restriction had
|
||||
plausibly never applied on a VM. Caught by the canary.
|
||||
- `evaluator::implementer_family(missions.backend)` — a glm mission is judged
|
||||
by the subscription, honestly independent (794f212).
|
||||
- `checkpoint.vm` on `topology_runs`: rootfs + guest `claude --version` per
|
||||
VM run; `verify-mission-delivery.sh glm|kimi` with `assert_provider_egress`.
|
||||
- VM tool-gate `denied.jsonl`/`inert` → `gate.denied`/`gate.inert` (9fc904a).
|
||||
- Judge: 64 KB window (4e342d1), latest round whole + budget prompt (1a44405).
|
||||
- Skills section second in the prompt (a50c41a); 4 more compliance checks
|
||||
(507d744).
|
||||
- `cm-files` MinIO test pinned to quay.io — Docker Hub deleted `minio/minio`;
|
||||
CI passed only on gw-04's year-old cache (5d9edd6).
|
||||
- **CI trap:** a newer push cancels the in-progress run (Gitea concurrency).
|
||||
Four commits in ten minutes looked like three failures and one real one;
|
||||
the one real one was environmental (gw-04 at 85% disk, 51 GB build cache —
|
||||
reclaimed 44 GB, re-run green). Push, then wait.
|
||||
- Both stacks wiped to zero 09-19 via the API; `_outputs` came with the
|
||||
missions this time. Judge ledger kept (15 rows prod, 3 local).
|
||||
- Fleet: tank claude/glm/kimi/local-ornith and architect claude/local-ornith
|
||||
all on 2.1.276; backups deleted after sign-off.
|
||||
- **The Mac's Tailscale switched to another tailnet** (`taila5f63e.ts.net`,
|
||||
`noc@alucio.net.id`) at some point on 09-19/20 — zero fleet peers from the
|
||||
laptop. `ssh gw-04-pub` etc. are the route until it is switched back.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user