From ef1f21024d62c0b3eb0fc282fa8966758bf7d62c Mon Sep 17 00:00:00 2001 From: Omar Sobh Date: Sun, 20 Sep 2026 20:24:51 -0500 Subject: [PATCH] docs: the open list as of 2026-09-20, and addendum 3 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz --- docs/NEXT-SESSION.md | 77 ++++++++++++++++++++++++++++++++------------ 1 file changed, 56 insertions(+), 21 deletions(-) diff --git a/docs/NEXT-SESSION.md b/docs/NEXT-SESSION.md index 46105a6..d06fdf4 100644 --- a/docs/NEXT-SESSION.md +++ b/docs/NEXT-SESSION.md @@ -100,28 +100,36 @@ guessed family. `worker`, names that promised providers they never used); map routes `researcher`/`analyst` to `worker` directly. -## Open, in the order I would take them +## Open, in the order I would take them (rewritten 2026-09-20) -1. **The judge runs to its check cap almost every time.** Compaction made - that cheaper; it did not ask why a research verdict needs 12 commands. - Watch `requests` per verdict on the next few missions. If it stays at the - cap, the lever is the judge prompt, not the budget. -2. ~~**microVM tier is unexercised since v0.8.5.**~~ Closed 2026-09-18: three - VM missions on Claude Code 2.1.276 — claude `01a0b75e`, glm `01a0b761`, - kimi `01a0b768` — 28/28 checks including provider egress proof. See the - addendum. architect rebuilt to 2.1.276 the same evening (claude + - local-ornith, selftest real turn PASS), so placement no longer decides the - CLI version. -3. **`files` is n=3.** 7 of 9 is a signal. `structured-paper-summary` went - unread in 2 of the last 3 runs; the skills section sits 87–90% into the - prompt. Position is the untested lever. -4. **Compliance checks exist for 6 of 53 skills.** The rest score - `not_applicable` on that axis forever. -5. **The persistent `clawmates-runtime` container's own config** still names - `worker_glm`/`worker_kimi` (3 mentions). Not on the mission path; editing - it restarts the paired runtime. -6. **Prod holds our 7 test missions** with 90-day retention. They are the - evidence for everything above; wipe them when they stop being. +Everything on the 09-14 list is closed or explained; see the addenda. What is +genuinely left: + +1. **The judge costs ~9 requests / ~20 K input per verdict, and that is now + legitimate work.** Three measured passes (09-19): the 12 KB output window + truncated an 18 KB deliverable → 64 KB; then my compaction erased the read + before the judge could use it → the latest round stays whole; then the + prompt says a cat is complete, decide first, read once. Result: four cats + of the deliverables plus cheap greps that VERIFY (placeholders, URL count, + sources, structure), zero re-reads. The only remaining lever is getting + the model to batch independent greps in one turn — a behaviour bet. +2. **`files` arm: 8 of 12 across four runs; the same skill is skipped every + time.** With the section moved to 2% of the prompt (a50c41a) the evidence + checker still does not open `structured-paper-summary`, whose + `when_to_use` is "summarising a research paper". That is triage, not a + miss. Nothing to fix; the scorer already says `not_applicable`. +3. **Compliance checks: 11 of 53.** The remaining 42 are content judgement. + Adding a check for one means finding a rule that leaves a mark in tool + arguments or delivered files — the module's own bar, and the right one. +4. **Follow-ups deliberately not taken:** `Spend{provider}` for VM turns + (requested-not-answered; `charge()` bills a credit for zero tokens — the + node egress log is the honest source and the harness asserts it); the + persistent runtime's own config still naming `worker_glm`/`worker_kimi` + (cosmetic; editing it restarts the paired runtime). +5. **Product/scaling work that was always scoped later:** missions-as-workflows + #14 (plan viewer + planner mode) and #15 (scheduling + loop-progress + events); scaling Phase 2/3 (node bring-up automation, Postgres HA, + autoscale); node-placed terminal cross-container writes. ## State you should know about @@ -220,3 +228,30 @@ all ours. Rebuild trap: rsync copies the Mac's dead `target` symlink to the node — `rm -f ~/clawmates/target` before `cargo build`. +## Addendum 3 — 2026-09-19/20 + +- `--agents` `tools` must be a JSON array (cbc9c2d); every CLI before 2.1.243 + dropped the string form silently, so the verifier's tool restriction had + plausibly never applied on a VM. Caught by the canary. +- `evaluator::implementer_family(missions.backend)` — a glm mission is judged + by the subscription, honestly independent (794f212). +- `checkpoint.vm` on `topology_runs`: rootfs + guest `claude --version` per + VM run; `verify-mission-delivery.sh glm|kimi` with `assert_provider_egress`. +- VM tool-gate `denied.jsonl`/`inert` → `gate.denied`/`gate.inert` (9fc904a). +- Judge: 64 KB window (4e342d1), latest round whole + budget prompt (1a44405). +- Skills section second in the prompt (a50c41a); 4 more compliance checks + (507d744). +- `cm-files` MinIO test pinned to quay.io — Docker Hub deleted `minio/minio`; + CI passed only on gw-04's year-old cache (5d9edd6). +- **CI trap:** a newer push cancels the in-progress run (Gitea concurrency). + Four commits in ten minutes looked like three failures and one real one; + the one real one was environmental (gw-04 at 85% disk, 51 GB build cache — + reclaimed 44 GB, re-run green). Push, then wait. +- Both stacks wiped to zero 09-19 via the API; `_outputs` came with the + missions this time. Judge ledger kept (15 rows prod, 3 local). +- Fleet: tank claude/glm/kimi/local-ornith and architect claude/local-ornith + all on 2.1.276; backups deleted after sign-off. +- **The Mac's Tailscale switched to another tailnet** (`taila5f63e.ts.net`, + `noc@alucio.net.id`) at some point on 09-19/20 — zero fleet peers from the + laptop. `ssh gw-04-pub` etc. are the route until it is switched back. +