TOOL-CALL-ARCHITECTURE.md said "switch claude_cli to stream-json" as the cheapest fix. That was wrong and is now marked so, with what actually happened: zero tool.call events with the parser working perfectly, because TurnEvent::ToolCall only fires for tools ZeroClaw itself executes. Hooks sidestep that entirely, and the doc now leads with the resolution rather than the theory. A fresh session is pointed at this file, so leaving the wrong recommendation on top would have sent it down the same path. NEXT-SESSION.md: state header, and the ordered list rewritten — items 1-3 are done or superseded. "Give the direct-session tier a tap" is dropped with its reason: that tier is dormant (CLAWMATES_MISSION_EXECUTOR unset), and checking before building saved the work. New top item is watching the first production mission, since the gate and tap are proven locally and unproven in prod. Added an operational section for the things that cost the most time: the 403 actions-log API, gw-04's legacy docker-compose, the socket proxy, disk contention between manual builds and CI, and Clerk-only prod auth. Also flagged that SKILL-USE-BASELINE.md's Trigger column is now stale in a good way — tool calls are observable on the container tier, so Trigger can be scored from behaviour instead of prose. That is the highest-value follow-up. Co-Authored-By: Claude Opus 5 <[email protected]>
178 lines
8.9 KiB
Markdown
178 lines
8.9 KiB
Markdown
# Where this left off — 2026-08-21
|
||
|
||
Read `CAPABILITY-REVIEW.md` for the system picture,
|
||
`TOOL-CALL-ARCHITECTURE.md` for how mission tools actually work (and the two
|
||
wrong theories that preceded it), and `SKILL-USE-BASELINE.md` for the
|
||
measurement — noting that its Trigger column is now out of date in a good way
|
||
(see "Next, in order" §2).
|
||
|
||
## State of the tree
|
||
|
||
Everything is pushed. `main` is at `5a11fae`; the fork's
|
||
`merge/upstream-v0.8.4` is at `be9c34b1c`. Local suite green: **107 test
|
||
binaries, 412 lib tests**, frontend builds.
|
||
|
||
**CI is green and production is current.** Run 507 passed `test` and `build`,
|
||
and gw-04 rolled to it. Production also runs `clawmates-runtime:hooks` for
|
||
per-mission containers (see below).
|
||
|
||
## Container-tier tool gate + telemetry — SHIPPED 2026-08-21
|
||
|
||
The tier that actually runs missions now has both, verified end to end on a
|
||
real mission (locally first, then deployed):
|
||
|
||
```
|
||
tool.call 10 Bash 6, Read 3, Write 1
|
||
file.touch 4 research/tapproof.md
|
||
```
|
||
|
||
That is the first time the container tier has ever been observable.
|
||
|
||
**How, after `stream-json` failed.** Claude Code runs its tools inside its own
|
||
subprocess, so they never reach ZeroClaw's executor and never become a
|
||
`TurnEvent::ToolCall`. Hooks bypass that entirely: `claude -p --settings <doc>`
|
||
honours `PreToolUse` and `PostToolUse`, so the gate blocks and the tap records
|
||
without ZeroClaw being involved at all.
|
||
|
||
Pieces: `--settings` on `claude_cli` (fork `be9c34b1c`),
|
||
`container_tool_hooks` writes both hook scripts and one settings document into
|
||
the mission container, `set_claude_cli_settings` points the provider at it, and
|
||
`phase_runner::drain_finished_container_phases` collects the tap into
|
||
`mission_events` (idempotent by truncation — no cursor column).
|
||
|
||
**Production state:** server on `f6e6037`;
|
||
`CLAWMATES_RUNTIME_IMAGE=clawmates-runtime:hooks` in `/opt/clawmates/.env`
|
||
(backup at `.env.bak.prehooks`; rollback = restore it and recreate). Note that
|
||
host uses **legacy `docker-compose`**, not the v2 plugin.
|
||
|
||
**Not yet observed in production** — no prod mission has run since the flip.
|
||
Prod auth is Clerk, so a mission cannot be launched from here by password. The
|
||
check when one runs:
|
||
|
||
```
|
||
ssh gw-04 'docker logs clawmates_server_1 2>&1 | grep -E "per-mission runtime image|drained .* tool call"'
|
||
```
|
||
|
||
### Three bugs the live test found, all the same shape
|
||
|
||
Each left every other link looking correct:
|
||
|
||
1. The settings document pointed `PostToolUse` at a path the installer never
|
||
wrote. Claude Code does not complain about a missing hook command — it
|
||
records nothing. A test now compares the document's commands against the
|
||
files the installer creates.
|
||
2. The mission container uses `CLAWMATES_RUNTIME_IMAGE`, not the shared
|
||
`clawmates-runtime` container — it was on an older image whose daemon schema
|
||
had no `settings` field, so the prop write returned `404 path_not_found`.
|
||
3. The drain used `connect_with_local_defaults()`; the server reaches Docker
|
||
through a **socket proxy**, so it failed — and returned `Ok(())` silently.
|
||
|
||
### CI: build failures were disk, not code
|
||
|
||
Runs 503–506 failed at `build` with an unreadable log, and the first casualty
|
||
was a **docs-only** commit. Cause: building runtime images by hand on gw-04
|
||
competes with CI for the same 150G volume; the frontend image build lost.
|
||
`docker builder prune` reclaimed 34GB (22G → 57G free) and the next run went
|
||
green. The build job now writes breadcrumbs, a `df -h` snapshot, and which
|
||
services actually pushed — the failing runs had pushed `server`, aborted on
|
||
`frontend`, and left `:latest` unmoved, which surfaced three steps later as
|
||
"the deploy did not happen".
|
||
|
||
**Operational note:** do not build images by hand on gw-04 while CI may run.
|
||
|
||
## Next, in order
|
||
|
||
1. **Watch the first production mission.** Nothing has run since the runtime
|
||
flip, so the container-tier gate and tap are proven locally and unproven in
|
||
prod. Prod auth is Clerk, so a mission cannot be launched from here.
|
||
```
|
||
ssh gw-04 'docker logs clawmates_server_1 2>&1 | grep -E "per-mission runtime image|drained .* tool call"'
|
||
ssh gw-04 'docker exec <cm-runtime-mission-…> cat /root/toolhooks/tap/tools.jsonl | head'
|
||
```
|
||
If tools appear in `mission_events`, the loop is closed. If not, check the
|
||
three failure shapes listed above — each looked correct from every other
|
||
angle.
|
||
|
||
2. **Now that tool calls are observable, redo the Skill-Use measurement.**
|
||
`docs/SKILL-USE-BASELINE.md` reports Trigger as `not_observable` on the
|
||
container tier because there was no tool evidence. There is now.
|
||
`mission_events` carries `tool.call` and `file.touch` per phase, so
|
||
Trigger can be scored from behaviour instead of prose — which is what the
|
||
paper actually measures. This is the single highest-value follow-up: it
|
||
turns the baseline from "the first honest number" into a real one.
|
||
|
||
3. **Give the microVM tier the same treatment, or retire the difference.** It
|
||
has the gate and a tap already, but by a different route (`vm_tool_tap`
|
||
installs into the guest, `microvm_executor` drains inside the turn). Two
|
||
mechanisms for one job is how they drift. Worth folding onto
|
||
`container_tool_hooks` once the fleet is back — tank and morpheus have been
|
||
offline for over a week, so the microVM tier cannot be tested at all today.
|
||
|
||
4. **Deploy the door** (`docs/TOOL-CALL-ARCHITECTURE.md` §3). Config, not code:
|
||
`/zeroclaw-data/clawmates-mcp.json` plus a door-shaped provider alias. Now
|
||
less urgent than it looked — the gate no longer depends on it — but it is
|
||
still the precondition for `clawmates_skills` being reachable, and therefore
|
||
for skills moving from inlined bodies to progressive disclosure.
|
||
|
||
5. **Pull upstream's egress policy** — `0db7d999a feat(plugins): add shared
|
||
egress policy foundation (#9137)`. We are ~220 commits behind; this is the
|
||
one item identified as worth taking, and it is defence for a problem we have
|
||
not solved.
|
||
|
||
**Dropped from this list:** "give the direct-session tier a tap". That tier is
|
||
dormant — `CLAWMATES_MISSION_EXECUTOR` is unset in production, so it never
|
||
runs. Checking that before building for it saved the work.
|
||
|
||
## Open decisions that are yours
|
||
|
||
- **Self-authoring scope.** Agents now apply their own `skill_candidate` items
|
||
with no human click (`CLAWMATES_SKILL_SELF_AUTHORING=0` restores the gate).
|
||
`identity_refinement` and `brain_consolidation` still wait for a human,
|
||
because they change what an agent IS rather than adding a procedure it can
|
||
consult. Say if you want those autonomous too.
|
||
- **Skill-Use Compliance coverage.** Most skills still score `not_applicable` —
|
||
we cannot tell whether they changed anything. `small-focused-commits` and
|
||
`tdd-red-green-refactor` are the next candidates and both need the repository
|
||
diff rather than the turn text.
|
||
|
||
## Deliberately not done
|
||
|
||
- **The mission executor swap** (running turns through `ProviderExecutor` or the
|
||
chat `Runtime`). The blockers are structural, not wiring: `cm-runtime`'s
|
||
`files` tool rejects absolute paths *by construction*, `shell` runs in a
|
||
per-agent sandbox with no mission mount, `ToolContext` carries no path or VM
|
||
handle, and approvals key on `(session_id, message_id)`. The cheap fixes
|
||
deliver what it was wanted for.
|
||
- **`cm-brain` offline tests** — 6 of 9 need live `clawbrainhub.com`. Stubbing
|
||
means reproducing an external registry protocol we have no spec for.
|
||
- **Graph memory / `clawhdf5-agent`** — in the workspace manifest, used by no
|
||
crate. Measure against a baseline before migrating.
|
||
|
||
## Operational facts that cost time to learn
|
||
|
||
- The Gitea **actions-log API returns 403** for the token in
|
||
`deploy/compose/.env`. Every CI failure this week was debugged blind because
|
||
of it. Steps now write to `/tmp/ci-logs` on the runner host as a workaround;
|
||
**a token with the `actions` scope** remains the highest-value thing to
|
||
obtain.
|
||
- **gw-04 uses legacy `docker-compose`**, not the v2 plugin. `docker compose`
|
||
fails there.
|
||
- **Do not build images by hand on gw-04 while CI may run** — same 150G volume,
|
||
and the frontend image build is what loses.
|
||
- The server reaches Docker through a **socket proxy** (`DOCKER_HOST`). Use
|
||
`container_exec::connect()`, never `connect_with_local_defaults()`.
|
||
- Prod auth is **Clerk**; the bootstrap password in `deploy/compose/.env` works
|
||
only against the local stack.
|
||
|
||
## Two corrections made this session, worth remembering
|
||
|
||
- **"Missions can't call tools at all" was wrong.** They call `Bash` and `Write`
|
||
with permissions pre-accepted. The gap was observing and gating, not having.
|
||
- **Raw test counts are a bad coverage metric.** They pointed at `cm-safety`,
|
||
whose seven tests already covered its critical paths, and missed a Slack
|
||
replay hole that let one captured request authenticate forever.
|
||
|
||
The recurring shape, now seven times over: **a claim in a comment or a doc,
|
||
believed and never checked.** Every significant finding this session came from
|
||
running the thing rather than reading about it.
|