Docker sandboxes run `sleep infinity` as PID 1. `sleep` never wait()s
on re-parented children, so Chromium's short-lived helper/crashpad
processes accumulate as zombies in long-lived browser sandboxes. With
`pids_limit` set, this eventually exhausts the slot and the container
can no longer fork — the browser tool starts failing before it looks
"unhealthy" anywhere else.
Setting HostConfig.init = Some(true) makes Docker inject tini, which
reaps re-parented children.
Note: existing sandbox containers keep their old spec (init is set at
create time). Cycle them post-deploy to pick up tini.
CI no longer references k8s; delete the dead k8s surface:
- deploy/helm/ (the chart), ci/check-helm.sh, scripts/netpol-cluster.sh
- cm-sandbox: the feature-gated K8sDriver (src/k8s.rs) + k8s_security test +
the `k8s`/`k8s-tests` features + the optional kube/k8s-openapi/rustls deps
(Cargo.lock drops the kube-rs tree). Nothing outside cm-sandbox referenced it.
Docker (bollard) DockerDriver is the sole sandbox driver. cm-sandbox + cm-runtime
compile, fmt + clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Survey + fixes so the pipeline passes at the Docker level (no k8s).
- Remove k8s: drop the `sandbox-k8s` job (kind/Calico/--features k8s-tests) and the
"Helm chart lints" gate step. release.yml was already k8s-clean.
- Rust job:
- `cargo fmt --all` — fix pre-existing formatting drift (fmt --check was failing).
- clippy -D warnings: fix 3 lib warnings (cm-brain sort_by_key→Reverse, cm-api
fleet.rs doc list indentation, node_rules map_or→is_none_or).
- Regenerate the .sqlx offline cache (was missing the cm-runtime run_loop test
query → offline compile failed). DB-backed tests use testcontainers at runtime.
- Set SQLX_OFFLINE=true on the rust + e2e jobs so query! macros compile against
the committed cache deterministically (no DB needed at compile time).
- Frontend job:
- Fix the 1 ESLint error (useAgentTelemetry: no setState-synchronously-in-effect;
tag the slice with agentId + derive null on mismatch).
- Fix 2 stale panel-params tests (`terminal` is a valid app id now; assert the
current APP_IDS + use a genuinely-unknown id for the reject case).
Verified locally: fmt clean, clippy --all-targets -D warnings clean (offline),
frontend lint 0 errors, tsc clean, 86/86 frontend tests pass, build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Completes "agent on a node" (single-node): when an agent's placement points at a
fleet node, its terminal container runs there and the browser reaches it over a
direct WebRTC DataChannel (LAN speed), sharing a node-local volume with the
sandbox. gw-04-local agents are byte-identical to before.
- cm-sandbox/docker.rs: empty drive subpath → mount the whole volume at the target
(volume_options None), so a per-agent node-local volume auto-creates at ~/drives.
- cm-api/fleet.rs: NodeHub.open_pty/webrtc_offer carry optional container+session
(injected only when Some); node-terminal caller passes None (host shell unchanged).
- cm-runtime/terminals.rs: TerminalManager gains node_provider + placement
(mirrors SandboxManager, draining-aware); node_local_drive_mount(agent) =
clawmates_agent_<id> at ~/drives; placement_for() ensures + locates the container;
attach uses driver_for(node) (local byte-identical).
- cm-runtime/sandboxes.rs: a node-placed agent sandbox mounts the same per-agent
volume → shares files with the terminal on that node.
- cm-api/routes/terminal.rs: ticket response gains `node`; ws() bridges node-placed
agents through the NodeHub relay (WebRTC + fallback) execing into the container;
local path unchanged. server main wires with_node_provider.
- frontend: agentTerminalConnector mints the ticket then picks WebRTC (node-placed,
⚡ direct / relayed badge) vs WS (local); webrtcConnector generalized to be
endpoint-agnostic (node terminal reuses it).
Known follow-up: terminal (uid 65532) and sandbox (uid 10001) share the volume but
differ in uid — cross-container writes need an aligned uid/gid (group-writable).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Agents can now provision their sandbox on a connected fleet node instead of the
gateway host. Local stays the strict default, so existing agents are byte-for-
byte unaffected until explicitly placed elsewhere.
Security parity: the daemon links the REAL cm-sandbox DockerDriver and runs the
typed container ops (sb_provision/sb_exec/sb_destroy/sb_health/sb_list) through
it — identical hardening (cap-drop ALL, seccomp, no-net, read-only, non-root) to
local sandboxes. cm-sandbox spec types are now Serialize/Deserialize so the spec
crosses the channel.
- cm-api: RemoteDriver (impl SandboxDriver over the node channel) + HubDriverProvider
(impl cm_runtime::NodeDriverProvider, hands out a driver only for connected
nodes via a sync online set) + NodeHub.call/is_connected. AppState.with_node_hub
so the hub is shared with the placement provider.
- cm-runtime SandboxManager: driver_for(node_id) routes by the recorded
agent_containers.node_id (local default = existing driver, identical path);
placement_node() reads the workspace setting and falls back to local if the
node is offline; exec/release route accordingly. NodeDriverProvider trait.
- DB: 0020_workspace_placement + repo (for_agent/get/set/clear).
- main.rs: build the NodeHub first; inject HubDriverProvider into the agent
manager + share the hub with AppState.
- API+UI: GET/PUT /api/fleet/placement + a "Run agents on: Local / <node>"
selector in the Fleet overview.
Note: a node must be able to pull the agent image (the daemon docker-pulls it);
interactive PTY for agent containers on remote nodes is not wired (Terminal app
stays local) — the in-dashboard node shell already covers host access.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Terminal app (xterm ⇄ WebSocket ⇄ per-agent themed container):
- zsh + oh-my-zsh + powerlevel10k image (agent-terminal), runs as uid 65532 to
share read-write ownership of the file-drive volume with the server.
- Interactive PTY in cm-sandbox (bollard exec tty/attach + resize) + a
TerminalManager; ticket-authed WS bridge routed straight to the backend via a
Traefik PathRegexp(/ws) rule. MOTD greets the user by name.
- tmux resumable sessions; multi-tab (one tmux session per tab, same container),
drag-to-reorder, rename, and a Save that persists named tabs to the server
(terminal_tabs, migration 0014) so they survive logout / a new device.
- Files drives mounted per-agent (subpath) at ~/drives/{documents,received,
shared}; a reconciler keeps the Files app's index in sync with terminal writes.
Storage moved to a shared `filedata` volume (CLAWMATES_STORAGE__DATA_DIR).
Obsidian vault (a markdown "second brain" per agent):
- New `vault` FileDrive (migration 0015) mounted into the terminal at ~/obsidian;
a file-content read route; a purple Obsidian tile + a vault viewer app.
Computer UI:
- Draggable computer-panel width (min = phone preset) keeping the size presets.
- Green Terminal glyph, "Claw Chat" → "Chat", colored gradient-outline app icons.
- Agent page: avatar↔activity-grid spacing + larger, uniform section fonts with
colored section-tinted tag chips.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
- scripts/netpol-cluster.sh: a kind cluster with Calico (default CNI
disabled) — the only way to PROVE the §15 default-deny NetworkPolicy,
since kindnet accepts the object but never enforces it. New live test
on that cluster: outbound connect to 1.1.1.1 dropped, DNS egress
dropped, while API-server exec keeps working (not pod network).
Kernel-level enforcement of the sandbox egress claim, demonstrated
- K8sDriver::connect_with_context: pin a kubeconfig context instead of
ambient. The whole k8s suite now pins its cluster explicitly — the
netpol cluster's creation had silently switched the current context
and stranded the seccomp test on the wrong cluster (fixed and made
impossible to recur)
- CI: netpol-cluster up + calico egress test in the sandbox-k8s job
- Visual-regression lock (plan P6): @visual Playwright spec with
animation-disabled, masked-dynamic-region screenshots of login,
workspace home, chat welcome, computer panel, credits; darwin
baselines committed (5 PNGs); CI excludes @visual until linux
baselines are generated there. Full local suite: 33 journeys
165 Rust tests + 5 live kind tests (2 clusters) + 33 journeys.
Co-Authored-By: Claude Fable 5 <[email protected]>
Full-depth rename per the approved plan; the 'claw' product vocabulary
(claws, /claws routes, clawId, Claw Chat) stays — it is now the brand.
- Display brand: Clawmates (manifest, titles, hero, login/rail logo
'clawmates'); default host app.clawmates.work; registry
ghcr.io/clawmates
- Crates tc-* -> cm-* (16 crates + all imports); binaries
clawmates-server/broker/bundler; images clawmates/*; env prefix
CLAWMATES_* (+ CM_TEST_DATABASE_URL / CM_LIVE_LLM); config
clawmates.toml; helm chart deploy/helm/clawmates with clawmates-*
resources; db names clawmates*; sockets /run/clawmates; cookie
cm_session; kind cluster clawmates-test; seccomp node profile
clawmates-agent-profile.json
- All 9 Playwright brand assertions updated in lockstep; historical
spec document left untouched as the only remaining 'TeamClaw'
- Local env migrated: dev pg clawmates-dev-pg/clawmates_dev, shared
test server clawmates-test-pg, kind cluster recreated with image +
profile, compose images rebuilt under clawmates/*
Verified end to end: 161 Rust + 68 frontend tests, 29 Playwright
journeys, 4 live kind tests, helm/install/LOC/placeholder gates, and
the clean-room install rehearsal serving the clawmates login page from
a signed bundle of the rebuilt images.
Co-Authored-By: Claude Fable 5 <[email protected]>
Omar Sobh