b89606fcf1dc7ca17988561fde3d69b14f2f1d13
107
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 4.7
|
85a97dffca |
slice 3.5d: agent_template_link + brain seed helper + skills merge
Ships the lineage layer that ties agents back to their team template
and wires the MCP skills server to actually merge template default
skills with per-agent overrides.
Migration 0050 adds `agent_template_link` (agent_id PK, template_id,
template_version, role_slot, seeded_at, created_at + indexes for
template/role lookups). Populated at agent-materialization time by
Slice 4's mission-launch orchestrator; read here by the skills MCP
server and by future level-up (Slice 8.5).
New Rust surface:
- cm_db::repo::agent_template_link (upsert / get / mark_seeded /
agents_for_template — the last is what level-up's "prompt upgrade
on template N+1" query needs)
- cm_api::brain_seed::ingest(claw_id, seed_md, identity_prompt)
opens cm_brain::ClawBrain on spawn_blocking, sets system_prompt
on first touch, writes seed as agent_md, commits. Idempotent —
skips when agent_md already populated.
- cm_api::mcp_skills::mcp_skills tools/call now resolves the caller
agent's template + role via agent_template_link and merges
template default skills with per-agent overrides (was overrides-
only in Slice 3.5b).
- cm_api::team_template_loader now binds template_role_skills after
upserting each template — looks up each declared skill by name,
attaches with pin_in_context=true for foundation skills and the
first two role skills. Missing skills log + skip.
- Boot ordering: skills load BEFORE team templates so the binding
lookup resolves.
Follow-up (Slice 4): mission-launch orchestrator calls brain_seed::ingest
+ agent_template_link::upsert when minting a team from a template.
Until that lands, the link is populated only by manual writes; the
MCP merge is silent-no-op for agents without a link (falls through
to overrides-only), which matches the pre-3.5d behavior.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
7b23f61632 |
slice 3.5c: seed 15 built-in skills across the 6 stacks
Hand-authored skill catalog anchored to real 2026-07 versions:
- Rust 1.97.1 (stable), edition 2024
- React 19.2.7, Server Components + Actions
- TailwindCSS 4.3.3 (CSS-first config, Oxide engine)
- three.js r185 (WebGPURenderer stable, BatchedMesh matured)
- React Native 0.86 / Expo SDK 54+ (New Architecture default)
- cargo-nextest 0.9.140, gitleaks 8.20+, cargo-audit 0.21+
- Postgres 17 (18 in beta, don't rely on)
- CUDA Blackwell, Metal Apple7+, ROCm CDNA3
Ships 15 skills across the categories:
foundation/ workspace-repo-commit-protocol
small-focused-commits
tdd-red-green-refactor
code-review-checklist
int-xx-marker-protocol
decompose-int-items
rust/ write-rust-current-edition
rust-error-handling
cargo-test-driven-development
rust-async-tokio-idioms
backend/ postgres-migrations-forward-only
postgres-index-selection
api-pagination-day-1
frontend/ react-19-server-components
tailwind-v4-idioms
component-4-state-model
mobile/ expo-managed-vs-bare
rn-flashlist-perf
gpu/ gpu-coalescing-and-occupancy
roofline-model
threejs/ threejs-perf-and-teardown
security/ cargo-audit-workflow
secret-scanning-gitleaks
skills_loader.rs walks skills/**/*.md, parses YAML frontmatter
(name, description, when_to_use, tags), upserts via
skills_catalog::upsert_builtin. Idempotent per boot — bumps version
+ appends skill_versions row ONLY when body changes. Deterministic
sha256-derived ids so builtins are stable across boots.
Dockerfile copies skills/ to /etc/clawmates/skills. Server boot
task spawns loader alongside team_template_loader.
Follow-ups (Slice 3.5c continuation, future PRs):
- 20-30 more skills (duckdb, shadcn composition, a11y, WebGPU
migration, metal frame capture, rocprof, deep gitea forge
integration, semgrep rulepacks)
- Bind skills to team template roles (add [role.skills] refs to
templates/teams/*.toml + wire template_role_skills population
in team_template_loader)
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
9ba5c06a1a |
slice 3: 6 team templates seeded from TOML recipes
Team templates are the canonical rosters + tool bundles that mint
concrete teams for a mission. Every builtin ships as a TOML recipe
under templates/teams/*.toml, loaded into the DB at server boot.
Migration 0048 adds:
- team_templates (id, key, name, stack, default_topology,
risk_profile, mcp_bundles, version, source,
workspace_id)
- template_roles (m2m: template_id + slot; system_prompt,
skills[], brain_seed)
- teams gets template_id + template_version for level-up lineage
Ships 6 builtins:
- rust_sdlc — planner/coder/tester/reviewer/committer for Rust
- backend — api_designer/db_engineer/coder/tester/committer
(Postgres, DuckDB, graph DBs, wire protocols)
- frontend — designer/coder/tester/committer (React + Tailwind + ShadCN)
- mobile — designer/coder/tester/committer (Expo, RN, iOS, Android)
- gpu — arch_analyst/kernel_author/bench_engineer/coder/committer
(CUDA, Metal, ROCm from Rust)
- threejs — scene_designer/coder/shader_author/perf_engineer/
committer (three.js, WebGL, WebGPU)
Each role has a versioned system_prompt + skill list + brain_seed
markdown. Skills column is a name array today; Slice 3.5a promotes it
to a typed m2m join with the real skills catalog.
Server boot:
- team_template_loader::load_builtins reads TOML from
/etc/clawmates/templates/teams (container) or templates/teams (dev),
upserts idempotently. Deterministic uuid per template key (sha256
of a fixed namespace + key) so ids are stable across boots.
- Dockerfile copies templates/ to /etc/clawmates/templates.
Read API:
- GET /api/team-templates — list all
- GET /api/team-templates/{id} — detail with roles
Wizard:
- Step 3 rewired from a raw team_id text field to a template picker
with "LLM auto-provision" as the default option + one card per
builtin, showing stack, topology, risk profile, and description.
- Mission create now passes team_template_id (not team_id) so phase
execution knows which template to mint from.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar Sobh
|
4b48c521eb |
loops: backend routes + repo + cron scheduler + HMAC webhook
Third commit of the Research + Loops arc. Lights up loops as durable
recurring topology executions:
GET /api/loops list workspace's loops
POST /api/loops create — returns webhook_token +
signing_key ONCE when webhook trigger
is enabled; never exposed again
GET /api/loops/:id detail
PATCH /api/loops/:id update definition
DELETE /api/loops/:id delete
POST /api/loops/:id/run trigger one iteration NOW
POST /api/loops/:id/enable set enabled=true
POST /api/loops/:id/disable set enabled=false
POST /webhooks/loops/:token public; HMAC-SHA256-verified
Scheduler (cm_runtime::spawn_loop_scheduler) wakes every 10s, queries the
partial index on (next_fire_at) for due loops, enqueues one topology_runs
row per fire with loop_id + iteration + parent_run_id chained back to the
previous iteration. Uses croner via the existing scheduling::next_occurrence
helper. Missed windows fire ONCE and skip the backlog — next_fire_at is
always computed strictly AFTER now(), so a late scheduler doesn't drain a
buildup.
Webhook signatures follow the same pattern as the Stripe billing webhook
(HMAC-SHA256 with constant-time hex compare). Token + signing key are
24-byte OS-RNG values; the URL uses base64-url for the token, and the
signing key is base64-std. Both surface exactly once at create time.
All three fire paths (scheduler, immediate-run, webhook) funnel through
`cm_db::repo::loops::enqueue_iteration` so the invariants stay in one
place. `iters` repeat policy is enforced by the scheduler tick; `until`
and `on_completion` land with the orchestrator hook in commit 4.
Adds cm-llm as a direct cm-api dep, getrandom for the webhook material
generator, and wires the scheduler spawn into the server binary alongside
the resume sweeper and outbox drainer.
|
||
|
Omar Sobh
|
3b382d659a |
broker: make Postgres pool size configurable, bump default 5 → 8
Under concurrent door executions the broker was queueing on its 5-connection pool, adding latency to tool calls that fanned out from the same run. Bump the default to 8 (one connection per concurrent door before queueing) and expose CLAWMATES_BROKER_POOL_SIZE so the fleet can tune it up as the load grows without a rebuild. |
||
|
Omar Sobh
|
51d501e36e |
clawmates-node: bound write.send with 10s timeout so half-open TCP can't wedge daemon
Under a half-open TCP (server side closed, client OS still buffering writes), the daemon's write.send() inside the tokio::select! branch blocks forever. tokio::select does not preempt a running future, so the whole loop freezes — idle_tick never gets to check last_rx, no 'channel ended' log ever fires, and the daemon silently spins on a dead socket for hours. Observed on architect Jul 5 2026: daemon connected at 12:59:13, heartbeats worked for ~2.5 min, then went silent. TCP session showed ESTABLISHED on the node side, no ESTABLISHED on the gateway side. Restarting the daemon 'fixed' it — but it re-hung within minutes. Fix: wrap both write.send() call sites in a tokio::time::timeout of WRITE_DEADLINE=10s. If a send stalls past that, we log and return Ok() to trigger the main-loop reconnect. Short enough that it fires long before the 40s read-idle would (which was our only escape hatch and never triggered because the loop was frozen). |
||
|
Omar SobhandClaude Opus 4.8
|
1a531e91cd |
Fleet tools: add Rust version card (probe rustc + nightly latest + rustup update)
Adds Rust to the per-node dev-tool cards: daemon probes rustc → reports 'rust'; nightly checker fetches latest stable from GitHub rust-lang/rust; GET endpoint maps rust→Rust (after docker); one-click update runs 'rustup update stable'. Frontend is data-driven (no change). $HOME/.cargo/bin added to probe candidate dirs. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
osobhandClaude Opus 4.8
|
96d1409a23 |
style: cargo fmt --all
Apply rustfmt (toolchain 1.96.0) to satisfy the CI Format check. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
b82a71d4ea |
Fleet tools: 8s probe cap + post-update refresh retries (updates now reflect)
After 'claude update' replaces the binary, the daemon's re-probe ran the fresh binary which macOS Gatekeeper re-verifies (>2s) — the 2s probe cap missed the new version, so the UI didn't refresh (update worked but looked stale). Bump the cap to 8s; frontend polls the tools endpoint a few times post-update to catch the re-probe. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
ed339c2121 |
Fleet tools: one-click per-node update (Phase 2)
The ↑ badge on each tool card is now a button: confirm → POST
/api/nodes/{id}/tools/{tool}/update → daemon runs the tool's own updater + re-probes.
- daemon: tool_update op (spawned task so the 170s update can't stall the WS loop;
re-probes + re-sends node_tools after). Fixed command allow-list (no arbitrary
shell): claude/glm → `claude update`; kimi → `uv tool upgrade kimi-cli`; ollama →
brew upgrade (mac) / install.sh (linux); else unsupported. 4KB output cap.
- cm-api: call_timeout/request_timeout (long ops); POST .../tools/{tool}/update
(workspace-scoped, allow-list) → {ok,output}.
- frontend: ↑latest becomes an Update button → confirm → spinner → refresh/err.
Note: claude/kimi/glm are user-space (no sudo); ollama on Linux uses install.sh
(needs sudo — works on passwordless nodes, returns an error otherwise; surfaced in UI).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
9263418fcb |
Fleet: per-node dev-tool version cards + nightly latest-check (Phase 1, read-only)
Each node card now shows installed versions of Docker / Claude Code / Kimi / GLM /
Ollama (conditional per node) under the ssh card, with an "update available" badge.
- daemon: probe_tools() finds docker/claude/kimi-cli/ollama across candidate bin dirs,
extracts semver from --version, reports {"t":"node_tools",...} on connect + every 15m.
- migration node_tools + tool_latest; cm-db repo node_tools (upsert/list/latest).
- cm-api: fleet.rs NodeTools uplink → upsert; tool_versions.rs spawn_latest_checker
(24h, npm/pypi/github; docker display-only); GET /api/nodes/{id}/tools (glm mirrors
claude). Spawned in clawmates-server.
- frontend: NodeTools cards on each HostCard with the ↑latest badge.
Phase 2 (one-click update execution) intentionally deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
3554a3aaf2 |
CI: remove k8s stages, fix the Docker-level pipeline green
Survey + fixes so the pipeline passes at the Docker level (no k8s).
- Remove k8s: drop the `sandbox-k8s` job (kind/Calico/--features k8s-tests) and the
"Helm chart lints" gate step. release.yml was already k8s-clean.
- Rust job:
- `cargo fmt --all` — fix pre-existing formatting drift (fmt --check was failing).
- clippy -D warnings: fix 3 lib warnings (cm-brain sort_by_key→Reverse, cm-api
fleet.rs doc list indentation, node_rules map_or→is_none_or).
- Regenerate the .sqlx offline cache (was missing the cm-runtime run_loop test
query → offline compile failed). DB-backed tests use testcontainers at runtime.
- Set SQLX_OFFLINE=true on the rust + e2e jobs so query! macros compile against
the committed cache deterministically (no DB needed at compile time).
- Frontend job:
- Fix the 1 ESLint error (useAgentTelemetry: no setState-synchronously-in-effect;
tag the slice with agentId + derive null on mismatch).
- Fix 2 stale panel-params tests (`terminal` is a valid app id now; assert the
current APP_IDS + use a genuinely-unknown id for the reject case).
Verified locally: fmt clean, clippy --all-targets -D warnings clean (offline),
frontend lint 0 errors, tsc clean, 86/86 frontend tests pass, build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
10c89f5157 |
Node-placed agent terminal: container PTY on the agent's node + WebRTC, shared node-local drives
Completes "agent on a node" (single-node): when an agent's placement points at a fleet node, its terminal container runs there and the browser reaches it over a direct WebRTC DataChannel (LAN speed), sharing a node-local volume with the sandbox. gw-04-local agents are byte-identical to before. - cm-sandbox/docker.rs: empty drive subpath → mount the whole volume at the target (volume_options None), so a per-agent node-local volume auto-creates at ~/drives. - cm-api/fleet.rs: NodeHub.open_pty/webrtc_offer carry optional container+session (injected only when Some); node-terminal caller passes None (host shell unchanged). - cm-runtime/terminals.rs: TerminalManager gains node_provider + placement (mirrors SandboxManager, draining-aware); node_local_drive_mount(agent) = clawmates_agent_<id> at ~/drives; placement_for() ensures + locates the container; attach uses driver_for(node) (local byte-identical). - cm-runtime/sandboxes.rs: a node-placed agent sandbox mounts the same per-agent volume → shares files with the terminal on that node. - cm-api/routes/terminal.rs: ticket response gains `node`; ws() bridges node-placed agents through the NodeHub relay (WebRTC + fallback) execing into the container; local path unchanged. server main wires with_node_provider. - frontend: agentTerminalConnector mints the ticket then picks WebRTC (node-placed, ⚡ direct / relayed badge) vs WS (local); webrtcConnector generalized to be endpoint-agnostic (node terminal reuses it). Known follow-up: terminal (uid 65532) and sandbox (uid 10001) share the volume but differ in uid — cross-container writes need an aligned uid/gid (group-writable). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
12212c72ac |
Daemon: PtyTarget — exec a terminal PTY into an agent container (node-placed terminal foundation)
Generalize the daemon's PTY spawn so a session can target either the node's host shell (today) or `docker exec -it <container> tmux …` (the node-placed agent terminal, which shares the container's node-local ~/drives). A `container` (+ optional `session`) field on pty_open/webrtc_offer selects the container path; absent it, the host shell path is byte-identical to before — so the Infra node terminal is unaffected. Threads PtyTarget through open_pty + rtc handle_offer → build_peer → bridge_pty. Additive; nothing emits `container` yet. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
c94784bab2 |
Fleet: actionable executions — rules engine + metrics-aware placement (Phase 2)
Turn the Beszel-tapped metrics into a self-managing loop. - migration node_rules (workspace/node-scoped: metric op threshold, for_seconds, action JSONB, last_fired). - cm-db: repo/node_rules.rs (CRUD + list_enabled); node_metrics::eval_all merges Beszel + heartbeat scalars per node + a headroom() heuristic; nodes::status_of; heartbeat now PRESERVES a `draining` status across heartbeats (so a cordon sticks). - cm-api: node_rules.rs evaluator (spawn_evaluator, 20s) — when a metric condition holds for the rule's window it fires drain / undrain / alert (in-memory sustained + cooldown tracking, modeled on the node sweeper); routes/beszel.rs rules CRUD (GET/POST/PATCH/DELETE /api/fleet/rules); spawned in clawmates-server. - cm-runtime: placement_node() is metrics-aware — a `draining` node stops receiving new agent sandboxes (falls back to local), so the drain rule is actionable. - frontend: FleetRules section in the Local view — build rules (node · metric · op · threshold · duration → action), toggle/delete, with fired-history. The loop: hot/overloaded node → rule drains it → placement avoids it → recovers → undrain rule brings it back. Deployed; node_rules migration applied. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
36a227566b |
Fleet: Beszel hub integration — rich per-node metrics + per-node monitor (Phase 1)
Tap each node's Beszel metrics (GPU/temps/disk-IO/network/per-container — beyond
our basic heartbeat) by reading the workspace's Beszel hub. The agents run in
WS-only mode with no locally-readable socket, so (per the de-risk) the server taps
the hub's PocketBase API instead of the daemon reading agents — no daemon changes.
- migrations: workspace_beszel (BYO hub URL + login, server-side only, mirrors the
Tailscale BYO pattern) + node_metrics (latest scalar columns + JSONB blob).
- cm-db: repo/fleet_beszel.rs, repo/node_metrics.rs; nodes SELECT joins node_metrics
(gpu_pct/temp_max surfaced on node_json for the live cards).
- cm-api: beszel.rs client (auth-with-password, poll `systems`, map to nodes by
hostname, upsert metrics) + a 15s spawn_poller; routes/beszel.rs (connect/status/
disconnect + GET /api/nodes/{id}/metrics with history proxied live from the hub).
- frontend: HostCard gains a GPU/temp readout + a Monitor button; NodeMonitor is a
full-width per-node page (current panel + CPU/mem/GPU/temp/net/disk charts from the
hub's 1m history); a "Beszel monitoring" connect form in the Local view.
Reachability confirmed: gw-04 → the hub over the tailnet (100.123.224.84:8090). Needs
the user to connect their hub login to activate the poller.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
4de2f31b50 |
Fleet terminal: WebRTC DataChannel direct path (low-latency) + WS fallback
Terminal keystrokes were ~400ms because every byte relayed browser→Cloudflare→
gw-04 (Europe)→tailscale→node, even when the node is on the user's own LAN. Add a
direct browser↔node WebRTC DataChannel so co-located terminals run at LAN speed;
the gateway is reduced to signaling; the WebSocket relay stays as the automatic
fallback (graceful degradation — never worse than before).
Daemon (clawmates-node v0.4.0, new src/rtc.rs):
- Add the `webrtc` crate (reuses the ring crypto provider we already install — no
conflict). Browser is the offerer; we answer, trickle ICE back over the control
channel, and on DataChannel open spawn a host PTY (tmux) bridged DIRECTLY to the
channel. Refactor open_pty → spawn_terminal_pty shared by both transports.
iceServers: STUN + auto host/tailnet candidates (direct, no relay, for LAN/tailnet).
Server (cm-api):
- NodeConn.signal_sinks; Uplink WebRtcAnswer/WebRtcIce/WebRtcFailed routed to the
browser; NodeHub webrtc_offer/ice/close + open_session/open_pty (open_terminal
split so the PTY opens only once the transport is chosen). bridge_terminal relays
signaling over the existing ticket-authed WS and opens the relay PTY on
{type:"fallback"}.
Browser (NodeTerminalApp):
- RTCPeerConnection + reliable/ordered DataChannel; offer/answer/ICE over the WS;
2.5s race → use the DataChannel if it opens, else fall back to the WS relay.
Reconnect wraps both. A direct⚡/relayed indicator shows the live transport.
Deployed; both nodes (morpheus, tank) updated to v0.4.0 and steady online. Direct-
path proof is a browser action (the ⚡ indicator + latency); confirmable from the
daemon's [rtc] logs.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
94828ed887 |
Fleet: robust real-time connectivity + mosh-inspired reconnecting terminal
Nodes flapped online/offline and the terminal died on the first blip. WebSockets are the right transport (outbound, NAT-friendly); the fixes harden around it. Server (cm-api): - Anti-clobber connection epoch: a reconnecting daemon gets a fresh epoch; a stale run_channel's teardown only clears the hub + sets offline if it still owns the slot — so a lingering old channel can't flip a live reconnection offline (the main false-offline cause). - WS keepalive: run_channel now pings every 15s and tears down if no inbound frame (incl. pong) for 35s — dead links detected in seconds, not minutes. - Staleness sweeper backstop: spawn_node_sweeper (8s tick / 20s window) wired in clawmates-server, so a vanished node goes offline within ~28s even if its channel hangs (mark_stale_offline was defined but never called). Daemon (clawmates-node v0.3.0): - Heartbeats off the select thread (dedicated thread owns System + blocking docker/tailscale/disk CLIs) so a slow op never starves heartbeats/pongs. - Each handle_frame runs on its own task; added a 40s inbound idle deadline so a half-open socket triggers a reconnect. Frontend: - useNodes streams /api/nodes/live (SSE push) instead of a 3s poll; isLive() derives online from lastSeen freshness (<15s) so a transient column flip never shows a healthy node down. - Node terminal: clean auto-reconnect loop (re-mint ticket -> reconnect -> tmux re-attaches and redraws the live screen = mosh-style snap-to-state over TCP), replacing the [disconnected] dead-end. Mosh evaluated: harvest principles (session/transport decoupling, snap-to-state, already given by tmux), don't adopt — UDP is incompatible with our browser+CF+NAT topology and it's GPLv3. Removed temporary terminal debug traces + /api/debug route. Verified: node holds steadily online (heartbeat 1-3s, no flap) and goes cleanly offline when the daemon stops. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
95bd022d07 |
Fleet terminal: fix tmux nesting-refusal + frontend resize storm
Root cause (from the server trace: 69 banner bytes then immediate "to_browser
ended" = pty_exit): tmux spawned and exited instantly, writing its "sessions
should be nested with care" warning to stderr (not the PTY). The operator runs
the daemon inside their own tmux, so $TMUX was inherited and the spawned tmux
refused to nest → browser saw nothing.
- daemon (v0.2.2): spawn tmux on a DEDICATED socket (`tmux -L clawmates
new-session -A -s main`) and `env_remove("TMUX")`, so it can never collide with
or be refused by the operator's tmux.
- NodeTerminalApp: debounce the ResizeObserver (150ms). The pull-out animates
open, firing the observer on every pixel — previously ~80 resize frames per
open, each fit()+SIGWINCH. Now one resize after layout settles.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
27f5d05f96 |
Fleet terminal: daemon self-diagnosis + immediate banner + debug route
The server trace showed pty_open is sent but the daemon (morpheus, v0.2.0) emits
no pty_out — so the PTY spawn was dying silently. Instrument it:
- daemon open_pty: log open/tmux/first-read/EOF/error/total to stdout, and send
an IMMEDIATE banner pty_out ("[clawmates] host shell on <host> — starting…") so
the browser confirms the relay even before the shell draws. If open_pty fails,
send the error as pty_out (was a silent pty_exit). Bump to v0.2.1.
- cm-api: temp GET /api/debug/node-pty/{id}?dbg=… opens a node terminal and reads
~2s of output with no browser/auth, to test the relay in isolation.
- NodeTerminalApp + ticket/ws routes already log each hop ([node-term]/[fleet-term]).
Diagnostic logic: banner shows + shell doesn't → relay ok, shell is the problem;
nothing shows → relay broken; daemon "EOF after N bytes" → shell exited.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
8e71c8403b |
clawmates-node: bump to v0.2.0 (selftest + tmux re-hosted)
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
c2a0309ad7 |
clawmates-node --selftest + server terminal tracing (diagnose blank terminal)
- Daemon: factor the terminal command into terminal_command(); add `--selftest` which opens the host terminal PTY locally and prints ~2.5s of raw output, so you can confirm tmux/zsh actually draws on a given node without the browser. (Verified locally: tmux spawns zsh + draws its status bar.) - cm-api: temporary [fleet-term] eprintln tracing in open_terminal, the pty_out router, and the browser bridge (byte counts + sink presence) to locate where output stops between daemon→server→browser. To be removed once diagnosed. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
6e81eaa52b |
clawmates-node: host terminal uses tmux (resumable, redraw-on-attach)
The node terminal opened a bare bash PTY — bash prints its prompt once, so a freshly-attached xterm showed nothing. Switch to `tmux new-session -A -s clawmates` on the HOST (mirrors the agent terminal, but on the node itself, not in a container): tmux redraws the whole screen on attach (no blank), and the session is resumable across reopens. Starts in $HOME. Falls back to a login shell if tmux is unavailable. ensure_tmux() best-effort installs tmux via the host package manager when the daemon runs as root (systemd); otherwise logs a hint to `apt install tmux`. Rebuilt + re-hosted both binaries. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
cf6c331b02 |
Fleet: node hostname/IP on register + node terminal in the infra computer
Hostname/IP: - Daemon reports the machine's hostname (sysinfo) + primary outbound IPv4 on each heartbeat. migrations/0021 adds nodes.hostname/local_ip; cm-db heartbeat stores them; node JSON exposes them. Cards now title on the real hostname (falling back to name) + show the IP, instead of the "New node" placeholder. `name` stays user-overridable (rename). Terminal moved into the pull-out computer (no more per-card modal): - New infra computer app NodeTerminalApp (computer/apps/infra) — xterm bridged to a node's host shell over the node control channel, filling the app window (mirrors the agent Terminal's layout + ResizeObserver). Added "terminal" to the INFRA_CATALOG grid; a ?node= panel param targets a specific node (picker when unset). Clicking Terminal on a node card now opens the infra computer to that node's shell instead of a separate full-screen window. Deleted NodeTerminal.tsx. Rebuilt + re-hosted both daemon binaries (hostname change). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
cfa16751c5 |
clawmates-node: fix rustls CryptoProvider panic + non-intrusive Tailscale
Two bugs surfaced running the daemon on a real node:
- Linking cm-sandbox (bollard) brought a second rustls provider into the graph,
so rustls couldn't auto-pick one and panicked at the WSS TLS handshake. Install
the ring provider explicitly at startup (rustls dep + install_default()).
- The daemon auto-ran `tailscale set --ssh`, which tries to reroute the user's
live SSH session and aborts ("will result in your session disconnecting"). Now
Tailscale is only touched when an auth key is explicitly passed (opt-in), with
--accept-risk=lose-ssh to avoid the interactive abort.
Rebuilt + re-hosted both binaries (linux-amd64, darwin-arm64).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
fb59378aa2 |
Fleet P2b: run agent sandboxes on connected nodes (RemoteDriver + placement)
Agents can now provision their sandbox on a connected fleet node instead of the gateway host. Local stays the strict default, so existing agents are byte-for- byte unaffected until explicitly placed elsewhere. Security parity: the daemon links the REAL cm-sandbox DockerDriver and runs the typed container ops (sb_provision/sb_exec/sb_destroy/sb_health/sb_list) through it — identical hardening (cap-drop ALL, seccomp, no-net, read-only, non-root) to local sandboxes. cm-sandbox spec types are now Serialize/Deserialize so the spec crosses the channel. - cm-api: RemoteDriver (impl SandboxDriver over the node channel) + HubDriverProvider (impl cm_runtime::NodeDriverProvider, hands out a driver only for connected nodes via a sync online set) + NodeHub.call/is_connected. AppState.with_node_hub so the hub is shared with the placement provider. - cm-runtime SandboxManager: driver_for(node_id) routes by the recorded agent_containers.node_id (local default = existing driver, identical path); placement_node() reads the workspace setting and falls back to local if the node is offline; exec/release route accordingly. NodeDriverProvider trait. - DB: 0020_workspace_placement + repo (for_agent/get/set/clear). - main.rs: build the NodeHub first; inject HubDriverProvider into the agent manager + share the hub with AppState. - API+UI: GET/PUT /api/fleet/placement + a "Run agents on: Local / <node>" selector in the Fleet overview. Note: a node must be able to pull the agent image (the daemon docker-pulls it); interactive PTY for agent containers on remote nodes is not wired (Terminal app stays local) — the in-dashboard node shell already covers host access. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
33aa9c0693 |
Fleet P2b: node sandbox-readiness check (hardened workload on a node)
Proves a connected node can host hardened agent workloads end-to-end, without
touching the agent run loop (zero blast radius on existing agents).
- Daemon: typed `sb_check` op — pulls a tiny image and runs it fully locked down
(cap-drop ALL, no-new-privileges, no network, read-only rootfs, non-root,
memory/pids caps), then tears it down. Fixed command; nothing caller-supplied
runs (preserves the exec-hardening invariant).
- cm-api: NodeHub.sandbox_check + POST /api/nodes/{id}/sandbox-check.
- UI: a shield "sandbox check" button on each online node card streams the
result (✓ SANDBOX READY + container id/uname).
This validates the full provision→run→destroy mechanism on nodes. The remaining
P2 work — wiring real agent deploys to auto-place onto nodes — is its own
subsystem (a RemoteDriver reusing the local DockerDriver for security parity,
agent-image distribution to nodes, and node-routing in SandboxManager) and is
best done as a focused pass; it is intentionally NOT bundled here to keep the
core agent path untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
f5f96508eb |
Fleet P2a: in-dashboard remote terminal (PTY over the WSS channel)
You can now open a real shell on any connected node from the dashboard — the
daemon spawns a host PTY and streams it over the existing outbound control
channel (no inbound port, no Tailscale brokering needed).
Daemon:
- portable-pty host shell sessions: pty_open/pty_in/pty_resize/pty_close ops; a
reader thread streams base64 pty_out frames. Outbound frames now funnel through
one mpsc channel so PTY output and heartbeats interleave.
cm-api NodeHub:
- per-connection pty_sinks + sid multiplexing; open_terminal/terminal_input/
terminal_resize/terminal_close; in-memory single-use terminal tickets (the
browser WS can't carry a bearer, and the session is instance-local anyway).
- routes/nodes.rs: POST /api/nodes/{id}/terminal/ticket + GET .../terminal/ws
(bridges browser xterm <-> node PTY: binary = keystrokes, text = resize).
Frontend:
- NodeTerminal xterm modal (reuses the agent Terminal's xterm setup); a Terminal
button on each online node card opens a shell.
This proves the bidirectional streaming-over-channel mechanism the RemoteDriver
will reuse. Remaining P2: RemoteDriver + placement (run agent workloads on nodes).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
7332d69f8a |
Fleet P1: BYO Tailscale + network metrics, Tailscale SSH, exec hardening
Security hardening: - The gateway no longer sends arbitrary shell to nodes. The WSS exec op is replaced by a typed `verify` op the daemon runs itself (fixed host+docker check); future container ops are typed too. cm-api NodeHub.verify() + the daemon's handle_command only dispatches vetted ops. BYO Tailscale: - migrations/0019_workspace_tailscale.sql + cm-db fleet_tailscale repo (store the user's Tailscale API key + tailnet, server-side only). - cm-api routes/tailscale.rs: POST/GET/DELETE /api/fleet/tailscale + GET /api/fleet/tailscale/devices (proxies api.tailscale.com device list). - Daemon: --tailscale-authkey → `tailscale up --authkey … --ssh` (enables Tailscale SSH for keyless user access); else `tailscale set --ssh=true`. Reports its tailscale IP (already). UI: - Fleet overview gains a Tailscale section: connect (key+tailnet) + live tailnet device status (online/last-seen/IP/os). Node cards show a copyable Tailscale SSH target (ssh <ip>). Remaining: P2 — RemoteDriver + placement (run agents on nodes) and the in-UI remote terminal (PTY proxied over the WSS channel). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
2bdd0a23e8 |
Fleet P0: node registry + daemon + health + connect-host wizard
Users can connect their own local-hardware nodes into a fleet. Each node runs a
new Rust daemon that dials home over an outbound WebSocket, reports host health,
and runs commands we send.
Backend:
- migrations/0018_fleet_nodes.sql: nodes + node_health tables + agent_containers
(node_id, workspace_id) index. cm-domain NodeId.
- cm-db repo/nodes.rs: create/auth/list+health/get/heartbeat/set_status/delete
(unchecked sqlx, no .sqlx regen).
- cm-api fleet.rs NodeHub: live daemon channels (node_id→sender) + the WS channel
runner (heartbeat→DB upsert, exec request/response framing). routes/nodes.rs:
POST /pair, GET /nodes, SSE /nodes/live, POST /{id}/exec-test, DELETE /{id},
WS /nodes/agent (token-auth). Wired into AppState + router.
Daemon (new crate crates/bins/clawmates-node):
- sysinfo host metrics (cpu/mem/pressure/swap/disk/load/containers), outbound WSS
dial + reconnect, heartbeat loop, exec command handling, tailscale-ip probe.
install.sh convenience installer.
Frontend:
- Fleet sidebar item + FleetOverview + LocalHardware node-health cards (live via
/api/nodes, 3s poll) + ConnectHostWizard (install → verify connection →
exec-test). InfraStage dispatches fleet/local; default selection = fleet.
Deferred: P1 (BYO Tailscale + network metrics), P2 (RemoteDriver + placement so
agents actually run on connected nodes).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
e9ce368ec1 |
Scaling Phase 1: multi-tenant onboarding + replica-safe coordination
Decouples "many users" + "many server replicas" from "many machines" so the platform is tenant-isolated and horizontally safe on the current single node. - Per-signup workspaces (cm-auth): a new hosted-identity sign-in provisions and owns its own workspace instead of joining the first. Config-gated by auth.per_signup_workspace (default off); concurrent first-logins serialized by a per-subject advisory lock so no duplicate workspaces. - Terminal tickets in Postgres (migration 0016, hashed, single-use): any replica can redeem a ticket minted by another. Drops the in-process ticket map. - Container registry in Postgres (migration 0017, agent_containers): Terminal and Sandbox managers resolve an agent's container through a shared registry, so a 2nd replica reuses it instead of spawning a duplicate. node_id recorded as 'local' (Phase 2 hook). Boot reconcile removes only true orphans, so terminals now survive a redeploy (tmux sessions resume). - Per-workspace quotas (cm-api/quota.rs): plan-tier caps on agents + live containers, enforced at agent create + terminal spin-up (reconnects allowed), returned as HTTP 402. New GET /api/quota surfaces usage vs limits. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
e61724ff82 |
Agent computer: terminal (tmux + drives + tabs), Obsidian vault, UI polish
Terminal app (xterm ⇄ WebSocket ⇄ per-agent themed container):
- zsh + oh-my-zsh + powerlevel10k image (agent-terminal), runs as uid 65532 to
share read-write ownership of the file-drive volume with the server.
- Interactive PTY in cm-sandbox (bollard exec tty/attach + resize) + a
TerminalManager; ticket-authed WS bridge routed straight to the backend via a
Traefik PathRegexp(/ws) rule. MOTD greets the user by name.
- tmux resumable sessions; multi-tab (one tmux session per tab, same container),
drag-to-reorder, rename, and a Save that persists named tabs to the server
(terminal_tabs, migration 0014) so they survive logout / a new device.
- Files drives mounted per-agent (subpath) at ~/drives/{documents,received,
shared}; a reconciler keeps the Files app's index in sync with terminal writes.
Storage moved to a shared `filedata` volume (CLAWMATES_STORAGE__DATA_DIR).
Obsidian vault (a markdown "second brain" per agent):
- New `vault` FileDrive (migration 0015) mounted into the terminal at ~/obsidian;
a file-content read route; a purple Obsidian tile + a vault viewer app.
Computer UI:
- Draggable computer-panel width (min = phone preset) keeping the size presets.
- Green Terminal glyph, "Claw Chat" → "Chat", colored gradient-outline app icons.
- Agent page: avatar↔activity-grid spacing + larger, uniform section fonts with
colored section-tinted tag chips.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
34f744734b |
Large World graph, agent platform, brain stack & dashboard rebuild
Frontend - Large World: collapse org/company/team tiers into one expandable React Flow hierarchy (WorldFlow) with per-click expand, persisted node positions, a compact tree sidebar, wrench multi-select delete across levels, and a sized right slide-out (phone/tablet/full) showing an agent summary + drill button. - Agent page: GitHub-style animated contribution grid (VitalsCard), collapsible System Prompt + Personality cards, restructured anatomy cards, bigger avatar with name/title header row, Markdown/JSON-aware rendering, brain registry + history, avatar generate/upload. - User-icon menu (Infrastructure/Brains/Tools/Profile/Credits) + ToolPanel; Master Planner deploy wizard (Specialists/Swarm/Scheduled/Triggered); Team Runs view; reap-progress modal; dashboard is the single live interface. Backend - cm-brain crate (.brain as the agent definition) + brain apply/history. - Hard-purge reap (FK-ordered) + sandbox release + SSE batch-delete. - Swarm self-verifying loop, mode-aware planner, web.search tool, webhooks (migration 0013), org/company/team delete endpoints, scheduler sweeps. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
148705769f |
Reaping: sandbox orphan reaper + DB expiry/retention sweeper + volume-init retry
Closes the cleanup gaps found in review (latent today; bites under load/crashes).
Sandbox containers (cm-sandbox / cm-runtime):
- label every sandbox `clawmates.sandbox={agent|browser}` at create
- SandboxDriver::list_managed(kind) (Docker label filter + K8s label selector)
- SandboxManager::reconcile_orphans(ttl) + spawn_reaper: removes engine
containers no live handle owns (ZERO = all)
- boot reconciliation (every pre-existing sandbox is an orphan from a dead
process) + periodic reaper (5m interval / 10m TTL)
- SIGTERM graceful drain: serve().with_graceful_shutdown → shutdown() both
managers so a redeploy can't leak; destroy errors now logged not swallowed
DB expiry/retention (cm-db cleanup.rs + cm-api cleanup_sweeper, hourly):
- expire auth_sessions + oauth_states past expires_at (security)
- prune sent outbox(7d), run_events(14d), routine_runs(30d), terminal
topology_runs(90d), consumed execution_grants(7d)
Compose: volume-init restart "no" → on-failure:5 (retry instead of wedging boot).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
e8486ee57c |
Door delivery: outbox drainer + SMTP transport (inert until configured)
The §15 door's email_send queues to `outbox` but nothing delivered it. Add a real transport: cm-db outbox repo (list_queued/mark_sent/mark_failed) + a cm-runtime drainer — an EmailSender trait (testable), a lettre STARTTLS LettreSender, drain_once (queued -> sent/failed), and spawn_drainer wired into the server beside the scheduler/sweeper/topology-worker. Config-gated: inert (logs "outbox delivery DISABLED") until CLAWMATES_SMTP_* is set, so it ships safely before credentials exist. The agent never holds the SMTP credential — it only writes to outbox through the gated door; the server owns the transport. NOTE: live delivery is still credential-blocked — Migadu's API can't send (SMTP-only) and the admin token is invalid; no SMTP creds exist. The transport is built + tested (drain_marks_sent_and_failed via a mock sender); set CLAWMATES_SMTP_* to go live with zero further code. clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
6e87433c66 |
Close gaps: GLM-judge (anthropic registry) + run cancel + deep-link
#2 GLM judge (closes #114): registry NamedProvider gains a `format` field; build_provider_registry builds an AnthropicProvider for format="anthropic". GLM's coding/OpenAI endpoint is ToS-throttled for raw SDK, but its Anthropic endpoint (api.z.ai/api/anthropic) accepts raw API calls (verified x-api-key -> glm-4.7), so CLAWMATES_JUDGE_MODEL=glm:glm-4.7 routes the door governor / topology judge through GLM with no runtime-routing. (Kimi-as-judge still needs a Platform key — coding key is agent-only.) #3 run-control: POST /api/topology-runs/{id}/cancel (workspace-scoped, queued/running only); the worker honors it at the step boundary (checks current_status in the checkpoint callback) and won't clobber a cancel with `failed`. Frontend Run tab gains a Cancel button and clickable recent runs that deep-link into a live/replayed stream (SSE replays from checkpoint). cm-* tests (incl. new cancel test) + clippy + frontend lint/typecheck green. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
272669e1f5 |
Durable topology jobs (3+4/4): background worker + async run API
POST /api/topologies/run now ENQUEUES a durable job and returns 202
{run_id, status:queued} instead of executing inside the HTTP request — the
prerequisite for long-horizon runs (no client/proxy/LB timeout, survives
restarts).
topology_worker: a spawned loop that requeues stale running jobs, claims the
next queued one (CAS via FOR UPDATE SKIP LOCKED), drives it through
execute_resumable, and checkpoints RunProgress after every step; on crash the
stale sweep requeues it and the next claim resumes from the last checkpoint.
Wired into server startup beside the scheduler + resume sweeper.
GET /api/topology-runs/{id} now reports lifecycle status/kind/error/checkpoint
+ the result blob (kept the `comparison` field name for back-compat with the
compare UI; null until completed). list_runs includes status + kind.
Tests: durable lifecycle (enqueue→claim→checkpoint→complete) + stale-requeue
resume, both green; p0 endpoints (compare path) unchanged. 13 + 2 tests pass,
clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
2ec8832ef6 |
llm: named-provider registry — judges & topology nodes on GLM/Kimi
Adds a multi-provider registry so judges and topology execution can run on
providers beyond the default. cm-config gains [[llm.providers]] (name, base_url,
api_key_env) — all OpenAI-compatible (GLM, Kimi/Moonshot). The server builds an
Arc<dyn LlmProvider> per entry (OpenAiCompatProvider) keyed by name; a missing
key is skipped with a warning, not a boot failure. cm-runtime RuntimeConfig
carries a ProviderRegistry; Runtime::resolve_provider("<name>:<model>") selects a
registry provider (else the default). The door governor (Runtime::judge) and the
topology compare endpoint both resolve through it, so CLAWMATES_JUDGE_MODEL and
CLAWMATES_TOPOLOGY_EXEC_MODEL accept "glm:glm-4.6" / "kimi:kimi-k2".
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Fable 5
|
f3f08a8edd |
R4 backend: team org-chart + leaderboard, Stripe credits, workspace apps
- GET /api/team/orgchart — each member grouped with the claws they manage (agents.managed_by); GET /api/team/leaderboard — every claw ranked by its real usage_events rollup (credits/tokens/runs, zeros included). Both tested against real Postgres. - Stripe Buy-credits (the Slack/Clerk integration pattern): [billing] config (stripe keys + price + webhook secret + credits_per_pack); POST /api/credits/checkout opens a real Checkout Session; POST /api/billing/stripe verifies Stripe's t=,v1= HMAC (constant-time) and grants one credit lot, idempotent on the session id; GET /api/billing/config gates the button (honest degradation when unset). Offline tests: signed grant + replay no-double-grant + forged-sig 400 + config flag. Live checkout creation deferred to a CM_LIVE_STRIPE test. - /apps global page support: clawId now optional on connect + directory; absent => workspace-wide connection (app_connections.agent_id NULL) via new connections::list_for_workspace. - ApiError gains a From<sqlx::Error> so inline queries use ? cleanly. cm-api 10 test files incl. team_tabs (2) + stripe_billing (3); clippy clean. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
447f7039d8 |
Compose: production README, env knobs, first-owner bootstrap — deployed & live
Made the Docker Compose route turn-key for a real self-host, then stood the whole stack up and drove a live chat through it. - First-owner bootstrap (cm-auth::bootstrap_owner): a fresh local-auth install has no users and no signup route, so the initial Owner + workspace are provisioned ONCE from CLAWMATES_BOOTSTRAP_* env on first boot — idempotent, never clobbers an existing install (keys on 'any workspace exists'). Two real-Postgres tests (creates + signs in; second call is a no-op). Wired into server boot, guarded on a non-empty password - deploy/compose/README.md: full production bring-up — services, the security topology, every config knob, Anthropic vs local-LLM, the broker-key backup, ops, and TLS/SSE proxy notes - .env.example fleshed out (bootstrap, LLM, auth mode, OTLP); compose uses optional env_file so only the knobs you set are injected (unset options never override clawmates.toml with empty strings) - volume-init one-shot chowns the broker's named volumes so the non-root scratch broker can write its socket + generated master key Deployed locally and verified end to end: all 5 containers healthy, broker generated its key, server bootstrapped owner@…, login + /api/user/me work, and a real message streamed a live Anthropic response through the gateway. Captured screenshots of login, workspace home, chat, and the Computer panel. 166 Rust tests (+2 bootstrap). Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
add4f79fed |
Rebrand: TeamClaw -> Clawmates (clawmates.work)
Full-depth rename per the approved plan; the 'claw' product vocabulary (claws, /claws routes, clawId, Claw Chat) stays — it is now the brand. - Display brand: Clawmates (manifest, titles, hero, login/rail logo 'clawmates'); default host app.clawmates.work; registry ghcr.io/clawmates - Crates tc-* -> cm-* (16 crates + all imports); binaries clawmates-server/broker/bundler; images clawmates/*; env prefix CLAWMATES_* (+ CM_TEST_DATABASE_URL / CM_LIVE_LLM); config clawmates.toml; helm chart deploy/helm/clawmates with clawmates-* resources; db names clawmates*; sockets /run/clawmates; cookie cm_session; kind cluster clawmates-test; seccomp node profile clawmates-agent-profile.json - All 9 Playwright brand assertions updated in lockstep; historical spec document left untouched as the only remaining 'TeamClaw' - Local env migrated: dev pg clawmates-dev-pg/clawmates_dev, shared test server clawmates-test-pg, kind cluster recreated with image + profile, compose images rebuilt under clawmates/* Verified end to end: 161 Rust + 68 frontend tests, 29 Playwright journeys, 4 live kind tests, helm/install/LOC/placeholder gates, and the clean-room install rehearsal serving the clawmates login page from a signed bundle of the rebuilt images. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
8046853feb |
Post-1.0: OTLP tracing, broker image + compose service, install rehearsal
- tc-telemetry: fmt subscriber always; with [telemetry] otlp_endpoint set, spans batch-export over OTLP/HTTP. Tested against a REAL OTLP receiver decoding the actual protobuf (official proto types): the emitted span and service.name arrive on the wire. No endpoint = no export = no network (air-gap stance). tower-http TraceLayer gives every API request a span - The broker finally has its own image (images/broker.Dockerfile, 9.5MB from scratch) — the Helm chart referenced one that never existed — and the compose deployment now RUNS the broker, sharing a socket volume with the server (the unix-socket equivalent of the K8s sidecar). Compose secret flows were silently dead before this - server.Dockerfile fixes surfaced by the rehearsal: the workspace build needs tools/ (bundler joined the workspace) and images/seccomp/ (include_str! profile) in the build context - scripts/rehearse-install.sh (plan: clean-VM rehearsal): assembles a REAL signed bundle from the built images (server/frontend/broker/ postgres/socket-proxy), runs the customer path — offline verify, docker load, compose up — and asserts /healthz plus the served login page before teardown. Passing locally; wired as a release.yml step, which also builds/ships the broker + socket-proxy images now 161 Rust tests + 29 journeys; clean-room rehearsal green. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
05e9612688 |
Post-1.0: Localhost seccomp on K8s, warm sandbox pool, server HPA
- K8sDriver.with_localhost_seccomp(profile): sandbox pods run under the STRICT allowlist instead of the runtime default. Proven live on kind: the harness installs the profile onto the node, a pod runs ordinary work as uid 10001, and unshare is kernel-denied inside the pod — the same probe the Docker suite uses, now passing on both targets - Helm: sandbox.seccomp=localhost renders a DaemonSet that installs the chart-shipped profile into /var/lib/kubelet/seccomp on every node (ConfigMap + hostPath); ci/check-helm.sh enforces the chart copy stays byte-identical to images/seccomp/agent-profile.json and asserts the hardened render (DaemonSet + profile + HPA) - server HPA (autoscaling/v2, CPU target) behind server.autoscaling.enabled - SandboxManager.warm(n): a background warmer keeps n pre-provisioned sandboxes ready so an agent's first exec skips container startup; unhealthy pool entries are discarded, reuse never drains the pool, shutdown destroys assigned AND pooled. [sandbox] warm_pool config (default 0). Real-Docker test: prefill -> assign -> refill -> reuse -> clean shutdown 160 Rust tests + 4 live kind tests. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
cbc8d35a2e |
Clerk authentication: hosted-identity session JWTs as a first-class mode
- tc-auth JwtVerifier: OIDC discovery -> JWKS, RS256 with the issuer pinned, 5s leeway (the crate's default 60s would double the life of Clerk's 60s session tokens), key cache with one refresh on unknown kid (Clerk rotates). Serves auth.mode = clerk AND generic oidc — a Clerk instance IS an OIDC issuer, so one verifier covers both - AuthService.authenticate dispatches: JWT-shaped bearers take the hosted-identity path, everything else stays a local opaque session. External users JIT-provision keyed by the stable sub claim (users.auth_subject, unique partial index in migration 0007); an existing local account with the same email is LINKED, not duplicated; role tracks the issuer claim every request (org:admin -> Owner) - Config auth.mode = "clerk" (requires issuer_url; validated), server pins the issuer at boot, Helm values/configmap accept mode=clerk - Tests with REAL crypto, no mocks: fresh RSA keypairs, a live local issuer publishing real discovery + JWKS docs, Clerk-shaped tokens — JIT + role mapping, repeat-subject no-dup, expired refused (leeway regression), wrong-key forgery refused, foreign issuer refused, and the full router round trip with Authorization: Bearer <session JWT> - docs/clerk.md: dashboard session-token customization (email + org role claims), config, @clerk/nextjs getToken() wiring, what CI proves 157 Rust + 63 frontend tests + 29 journeys. Air-gapped installs keep local auth — Clerk is a cloud-only alternative, not a replacement. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
4f253bec93 |
P6: browser.goto — real Chromium browsing with live web taint
- SandboxSpec gains an egress flag (default false — the kernel suite
still proves zero-network for agent sandboxes). Egress-enabled
containers exist ONLY for the browser: no credentials, no broker
route, bridge network with host-gateway alias for local test pages
- images/agent-browser: Alpine Chromium, uid 10001, setuid bits
stripped — same non-root hardening as agent-base
- browser.goto tool: headless chromium --dump-dom in the agent's
browser container; HTML stripped to readable text (4k cap) and
returned with output_taint=web; viewport screenshot captured,
base64'd out of the container, stored in the blob store
- Taint semantics tightened: the step that PRODUCED untrusted output
now carries its own taint (recorded before the step row), not just
later steps — chat.inbox test updated to the stricter §15 reading
- GET /api/claws/{id}/browser/viewport.png serves the latest capture;
BrowserApp polls it and renders the live viewport (spec §7.1),
keeping the empty state until the agent has browsed
- Proven end to end with REAL Chromium against a REAL local page:
content 'Revenue up 14 percent' returned tainted web; the gated
email.send that follows carries 'web' in its approval taint_sources
(untrusted content can never quietly reach outward); screenshot
verified by PNG magic bytes
152 Rust tests + 63 frontend + 27 Playwright journeys.
Co-Authored-By: Claude Fable 5 <[email protected]>
|
||
|
Omar SobhandClaude Fable 5
|
7392ce1d08 |
P6: shell.exec — agents execute code in their real hardened sandbox
- SandboxManager (tc-runtime): one container per agent, provisioned
lazily on first use, reused for the manager's lifetime, replaced
transparently if dead, destroyed on shutdown
- shell.exec tool: sh -lc inside the agent's sandbox; stdout/stderr/
exit_code return to the model as the step output. No external effects
declared — the sandbox boundary (uid 10001, no caps, seccomp
allowlist, read-only rootfs, zero egress) is the §15 control here,
not an approval gate
- RuntimeConfig.sandboxes (+ with_sandboxes builder); [sandbox] config
{image, enabled}; the server connects the Docker driver at boot and
tolerates an absent engine (shell.exec reports it per-call)
- Tests with the REAL DockerDriver: a scripted run executes two
commands — output proves uid 10001 from inside, and /home/agent state
written by the first call is read by the second (same sandbox); a
deployment without a sandbox runtime records honest error steps and
the run still completes
151 Rust tests + 27 Playwright journeys.
Co-Authored-By: Claude Fable 5 <[email protected]>
|
||
|
Omar SobhandClaude Fable 5
|
70ec39f696 |
P6: S3 blob store, Helm chart, air-gapped installer verify loop
- S3BlobStore (object_store, path-style) behind the same BlobStore trait, tested against a REAL MinIO container (round trip, overwrite, NotFound on get and delete, nested keys); [storage] backend=local|s3 config with validation + server-side selection (S3 creds via env overlay) - Helm chart: server pod with the secret broker as a SIDECAR sharing a private emptyDir unix socket (no network hop carries credentials), frontend, optional local PVC vs S3, OIDC/oauth values, unbuffered-SSE ingress annotations, NetworkPolicies (frontend->server only), hardened securityContexts; ci/check-helm.sh lints AND asserts the rendered topology properties - deploy/airgapped/install.sh: offline signature+checksum verification via the bundled teamclaw-bundler BEFORE any docker load; --verify-only mode; ci/test-install.sh rehearses clean/tampered/wrong-key paths with the real binary - CI: helm gate + installer rehearsal wired in 149 Rust tests; helm lint + rendered assertions green; installer verify-path rehearsal green. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
a8efada690 |
P5 exit: usage metering, credit billing, promo codes, 3-step wizard
- LlmEvent::Usage across all three providers (Scripted deterministic word-count accounting; Anthropic message_start/delta usage; OpenAI-compat stream_options include_usage) - tc-billing: ceil(tokens/1000) min 1 credit; lots drain oldest-first under FOR UPDATE; balance clamps at zero while the usage ledger records the full obligation; promo codes redeem exactly once via CAS (migration 0006) - Runtime charges every completed run (billing failure never fails a run); proven: 1 token in + 3 out -> 1 credit deducted - API: GET /api/team/usage, POST /api/credits/redeem (409 on reuse, audited) - Credits page: balance, 7-day usage meter with runway estimate, PromoRedeem - /claws/new is the full §9 wizard: ?step=identity|access|slack deep-linked progress, accent swatches + name randomizer, access toggles, optional Slack step, explicit review-and-confirm (creation = live agent), animated provisioning state -> straight into chat - E2E: chat decrements the visible balance and fills the usage meter; WELCOME500 adds exactly 500 once then refuses; wizard round trip 140 Rust + 63 frontend tests + 23 Playwright journeys. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
91327e3618 |
P4 complete: OAuth authorization-code flow, MCP-OAuth, AddApps connects
- migration 0005 oauth_states: one-time states (10-min TTL), consumed by a CAS DELETE on callback — replays and forgeries both 404 - POST /api/apps/oauth/start: OIDC discovery on the configured issuer (or the custom MCP issuer for authType=mcp_oauth), state row, authorize URL - GET /api/apps/oauth/callback: code exchanged at the REAL token endpoint (client id+secret form POST); the access token goes straight to the broker (test proves it never appears unencrypted in Postgres); connection row + audit; redirects to the claw's Add Apps panel - [oauth] config (issuer/client/redirect_base) wired through AppState - Tests against a real local IdP server (discovery + validating token endpoint): full round trip, broker-held token, replay/forged state refused, bad code fails exchange, mcp_oauth uses the custom issuer while plain oauth refuses without a configured IdP - AddAppsApp: live connection badges + inline API-key connect per app (E2E: connect Notion by key from the directory) 136 Rust + 63 frontend tests + 21 Playwright journeys. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
000b9b3a4b |
P4 core: broker-held app connections + gated, broker-executed Slack posting
- app_connections repo; POST /api/apps/connect (keys/basic): the credential goes to the secret broker over its socket and only the encrypted ref lands in the row; disconnect endpoint; /api/apps directory merged with live connection status; audit rows for connect/disconnect - Broker protocol: InvokeHttp carries a JSON body - slack.post tool (SendsExternally -> gated): marked broker_executed — the runtime skips its own grant consumption and the BROKER independently verifies + consumes the single-use grant, then calls Slack with the bot token injected; the runtime never sees the credential - Config: [broker] socket_path + [slack] base_url; e2e harness spawns the real teamclaw-broker daemon and the server hosts an e2e-only /__slack sink - SlackApp: Connection tab stores the token via the broker; connected state - Integration test: blocked while pending -> approved -> sink received exactly one post with 'Bearer xoxb-test-token' -> grant replay refused - E2E journey: connect Slack in the panel -> gated post card with preview -> sink empty while pending -> approve -> exactly one post, queue clear 133 Rust + 63 frontend tests + 21 Playwright journeys. Co-Authored-By: Claude Fable 5 <[email protected]> |