Commit Graph
11 Commits
Author SHA1 Message Date
Omar SobhandClaude Opus 4.8 e9ce368ec1 Scaling Phase 1: multi-tenant onboarding + replica-safe coordination
Decouples "many users" + "many server replicas" from "many machines" so the
platform is tenant-isolated and horizontally safe on the current single node.

- Per-signup workspaces (cm-auth): a new hosted-identity sign-in provisions and
  owns its own workspace instead of joining the first. Config-gated by
  auth.per_signup_workspace (default off); concurrent first-logins serialized by
  a per-subject advisory lock so no duplicate workspaces.
- Terminal tickets in Postgres (migration 0016, hashed, single-use): any replica
  can redeem a ticket minted by another. Drops the in-process ticket map.
- Container registry in Postgres (migration 0017, agent_containers): Terminal
  and Sandbox managers resolve an agent's container through a shared registry,
  so a 2nd replica reuses it instead of spawning a duplicate. node_id recorded
  as 'local' (Phase 2 hook). Boot reconcile removes only true orphans, so
  terminals now survive a redeploy (tmux sessions resume).
- Per-workspace quotas (cm-api/quota.rs): plan-tier caps on agents + live
  containers, enforced at agent create + terminal spin-up (reconnects allowed),
  returned as HTTP 402. New GET /api/quota surfaces usage vs limits.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-23 18:24:51 -07:00
Omar SobhandClaude Opus 4.8 e61724ff82 Agent computer: terminal (tmux + drives + tabs), Obsidian vault, UI polish
Terminal app (xterm ⇄ WebSocket ⇄ per-agent themed container):
- zsh + oh-my-zsh + powerlevel10k image (agent-terminal), runs as uid 65532 to
  share read-write ownership of the file-drive volume with the server.
- Interactive PTY in cm-sandbox (bollard exec tty/attach + resize) + a
  TerminalManager; ticket-authed WS bridge routed straight to the backend via a
  Traefik PathRegexp(/ws) rule. MOTD greets the user by name.
- tmux resumable sessions; multi-tab (one tmux session per tab, same container),
  drag-to-reorder, rename, and a Save that persists named tabs to the server
  (terminal_tabs, migration 0014) so they survive logout / a new device.
- Files drives mounted per-agent (subpath) at ~/drives/{documents,received,
  shared}; a reconciler keeps the Files app's index in sync with terminal writes.
  Storage moved to a shared `filedata` volume (CLAWMATES_STORAGE__DATA_DIR).

Obsidian vault (a markdown "second brain" per agent):
- New `vault` FileDrive (migration 0015) mounted into the terminal at ~/obsidian;
  a file-content read route; a purple Obsidian tile + a vault viewer app.

Computer UI:
- Draggable computer-panel width (min = phone preset) keeping the size presets.
- Green Terminal glyph, "Claw Chat" → "Chat", colored gradient-outline app icons.
- Agent page: avatar↔activity-grid spacing + larger, uniform section fonts with
  colored section-tinted tag chips.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-23 16:52:35 -07:00
Omar SobhandClaude Opus 4.8 34f744734b Large World graph, agent platform, brain stack & dashboard rebuild
Frontend
- Large World: collapse org/company/team tiers into one expandable React Flow
  hierarchy (WorldFlow) with per-click expand, persisted node positions, a
  compact tree sidebar, wrench multi-select delete across levels, and a sized
  right slide-out (phone/tablet/full) showing an agent summary + drill button.
- Agent page: GitHub-style animated contribution grid (VitalsCard), collapsible
  System Prompt + Personality cards, restructured anatomy cards, bigger avatar
  with name/title header row, Markdown/JSON-aware rendering, brain registry +
  history, avatar generate/upload.
- User-icon menu (Infrastructure/Brains/Tools/Profile/Credits) + ToolPanel;
  Master Planner deploy wizard (Specialists/Swarm/Scheduled/Triggered);
  Team Runs view; reap-progress modal; dashboard is the single live interface.

Backend
- cm-brain crate (.brain as the agent definition) + brain apply/history.
- Hard-purge reap (FK-ordered) + sandbox release + SSE batch-delete.
- Swarm self-verifying loop, mode-aware planner, web.search tool, webhooks
  (migration 0013), org/company/team delete endpoints, scheduler sweeps.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-22 23:21:54 -07:00
Omar SobhandClaude Opus 4.8 148705769f Reaping: sandbox orphan reaper + DB expiry/retention sweeper + volume-init retry
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
Closes the cleanup gaps found in review (latent today; bites under load/crashes).

Sandbox containers (cm-sandbox / cm-runtime):
- label every sandbox `clawmates.sandbox={agent|browser}` at create
- SandboxDriver::list_managed(kind) (Docker label filter + K8s label selector)
- SandboxManager::reconcile_orphans(ttl) + spawn_reaper: removes engine
  containers no live handle owns (ZERO = all)
- boot reconciliation (every pre-existing sandbox is an orphan from a dead
  process) + periodic reaper (5m interval / 10m TTL)
- SIGTERM graceful drain: serve().with_graceful_shutdown → shutdown() both
  managers so a redeploy can't leak; destroy errors now logged not swallowed

DB expiry/retention (cm-db cleanup.rs + cm-api cleanup_sweeper, hourly):
- expire auth_sessions + oauth_states past expires_at (security)
- prune sent outbox(7d), run_events(14d), routine_runs(30d), terminal
  topology_runs(90d), consumed execution_grants(7d)

Compose: volume-init restart "no" → on-failure:5 (retry instead of wedging boot).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-19 04:55:27 -07:00
Omar SobhandClaude Opus 4.8 e8486ee57c Door delivery: outbox drainer + SMTP transport (inert until configured)
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
The §15 door's email_send queues to `outbox` but nothing delivered it. Add a
real transport: cm-db outbox repo (list_queued/mark_sent/mark_failed) + a
cm-runtime drainer — an EmailSender trait (testable), a lettre STARTTLS
LettreSender, drain_once (queued -> sent/failed), and spawn_drainer wired into
the server beside the scheduler/sweeper/topology-worker.

Config-gated: inert (logs "outbox delivery DISABLED") until CLAWMATES_SMTP_*
is set, so it ships safely before credentials exist. The agent never holds the
SMTP credential — it only writes to outbox through the gated door; the server
owns the transport.

NOTE: live delivery is still credential-blocked — Migadu's API can't send
(SMTP-only) and the admin token is invalid; no SMTP creds exist. The transport
is built + tested (drain_marks_sent_and_failed via a mock sender); set
CLAWMATES_SMTP_* to go live with zero further code. clippy clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-18 03:22:27 -07:00
Omar SobhandClaude Opus 4.8 6e87433c66 Close gaps: GLM-judge (anthropic registry) + run cancel + deep-link
ci / frontend (push) Has been cancelled
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / e2e (push) Has been cancelled
#2 GLM judge (closes #114): registry NamedProvider gains a `format` field;
build_provider_registry builds an AnthropicProvider for format="anthropic".
GLM's coding/OpenAI endpoint is ToS-throttled for raw SDK, but its Anthropic
endpoint (api.z.ai/api/anthropic) accepts raw API calls (verified x-api-key
-> glm-4.7), so CLAWMATES_JUDGE_MODEL=glm:glm-4.7 routes the door governor /
topology judge through GLM with no runtime-routing. (Kimi-as-judge still needs
a Platform key — coding key is agent-only.)

#3 run-control: POST /api/topology-runs/{id}/cancel (workspace-scoped,
queued/running only); the worker honors it at the step boundary (checks
current_status in the checkpoint callback) and won't clobber a cancel with
`failed`. Frontend Run tab gains a Cancel button and clickable recent runs
that deep-link into a live/replayed stream (SSE replays from checkpoint).

cm-* tests (incl. new cancel test) + clippy + frontend lint/typecheck green.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-18 03:07:41 -07:00
Omar SobhandClaude Opus 4.8 272669e1f5 Durable topology jobs (3+4/4): background worker + async run API
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
POST /api/topologies/run now ENQUEUES a durable job and returns 202
{run_id, status:queued} instead of executing inside the HTTP request — the
prerequisite for long-horizon runs (no client/proxy/LB timeout, survives
restarts).

topology_worker: a spawned loop that requeues stale running jobs, claims the
next queued one (CAS via FOR UPDATE SKIP LOCKED), drives it through
execute_resumable, and checkpoints RunProgress after every step; on crash the
stale sweep requeues it and the next claim resumes from the last checkpoint.
Wired into server startup beside the scheduler + resume sweeper.

GET /api/topology-runs/{id} now reports lifecycle status/kind/error/checkpoint
+ the result blob (kept the `comparison` field name for back-compat with the
compare UI; null until completed). list_runs includes status + kind.

Tests: durable lifecycle (enqueue→claim→checkpoint→complete) + stale-requeue
resume, both green; p0 endpoints (compare path) unchanged. 13 + 2 tests pass,
clippy clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-17 19:12:54 -07:00
Omar SobhandClaude Opus 4.8 2ec8832ef6 llm: named-provider registry — judges & topology nodes on GLM/Kimi
Adds a multi-provider registry so judges and topology execution can run on
providers beyond the default. cm-config gains [[llm.providers]] (name, base_url,
api_key_env) — all OpenAI-compatible (GLM, Kimi/Moonshot). The server builds an
Arc<dyn LlmProvider> per entry (OpenAiCompatProvider) keyed by name; a missing
key is skipped with a warning, not a boot failure. cm-runtime RuntimeConfig
carries a ProviderRegistry; Runtime::resolve_provider("<name>:<model>") selects a
registry provider (else the default). The door governor (Runtime::judge) and the
topology compare endpoint both resolve through it, so CLAWMATES_JUDGE_MODEL and
CLAWMATES_TOPOLOGY_EXEC_MODEL accept "glm:glm-4.6" / "kimi:kimi-k2".

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-17 04:17:14 -07:00
Omar SobhandClaude Fable 5 f3f08a8edd R4 backend: team org-chart + leaderboard, Stripe credits, workspace apps
- GET /api/team/orgchart — each member grouped with the claws they manage
  (agents.managed_by); GET /api/team/leaderboard — every claw ranked by
  its real usage_events rollup (credits/tokens/runs, zeros included).
  Both tested against real Postgres.
- Stripe Buy-credits (the Slack/Clerk integration pattern): [billing]
  config (stripe keys + price + webhook secret + credits_per_pack);
  POST /api/credits/checkout opens a real Checkout Session; POST
  /api/billing/stripe verifies Stripe's t=,v1= HMAC (constant-time) and
  grants one credit lot, idempotent on the session id; GET
  /api/billing/config gates the button (honest degradation when unset).
  Offline tests: signed grant + replay no-double-grant + forged-sig 400 +
  config flag. Live checkout creation deferred to a CM_LIVE_STRIPE test.
- /apps global page support: clawId now optional on connect + directory;
  absent => workspace-wide connection (app_connections.agent_id NULL) via
  new connections::list_for_workspace.
- ApiError gains a From<sqlx::Error> so inline queries use ? cleanly.

cm-api 10 test files incl. team_tabs (2) + stripe_billing (3); clippy clean.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 19:58:09 -05:00
Omar SobhandClaude Fable 5 447f7039d8 Compose: production README, env knobs, first-owner bootstrap — deployed & live
Made the Docker Compose route turn-key for a real self-host, then stood
the whole stack up and drove a live chat through it.

- First-owner bootstrap (cm-auth::bootstrap_owner): a fresh local-auth
  install has no users and no signup route, so the initial Owner +
  workspace are provisioned ONCE from CLAWMATES_BOOTSTRAP_* env on first
  boot — idempotent, never clobbers an existing install (keys on 'any
  workspace exists'). Two real-Postgres tests (creates + signs in;
  second call is a no-op). Wired into server boot, guarded on a
  non-empty password
- deploy/compose/README.md: full production bring-up — services, the
  security topology, every config knob, Anthropic vs local-LLM, the
  broker-key backup, ops, and TLS/SSE proxy notes
- .env.example fleshed out (bootstrap, LLM, auth mode, OTLP); compose
  uses optional env_file so only the knobs you set are injected (unset
  options never override clawmates.toml with empty strings)
- volume-init one-shot chowns the broker's named volumes so the non-root
  scratch broker can write its socket + generated master key

Deployed locally and verified end to end: all 5 containers healthy,
broker generated its key, server bootstrapped owner@…, login + /api/user/me
work, and a real message streamed a live Anthropic response through the
gateway. Captured screenshots of login, workspace home, chat, and the
Computer panel.

166 Rust tests (+2 bootstrap).

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 14:06:15 -05:00
Omar SobhandClaude Fable 5 add4f79fed Rebrand: TeamClaw -> Clawmates (clawmates.work)
Full-depth rename per the approved plan; the 'claw' product vocabulary
(claws, /claws routes, clawId, Claw Chat) stays — it is now the brand.

- Display brand: Clawmates (manifest, titles, hero, login/rail logo
  'clawmates'); default host app.clawmates.work; registry
  ghcr.io/clawmates
- Crates tc-* -> cm-* (16 crates + all imports); binaries
  clawmates-server/broker/bundler; images clawmates/*; env prefix
  CLAWMATES_* (+ CM_TEST_DATABASE_URL / CM_LIVE_LLM); config
  clawmates.toml; helm chart deploy/helm/clawmates with clawmates-*
  resources; db names clawmates*; sockets /run/clawmates; cookie
  cm_session; kind cluster clawmates-test; seccomp node profile
  clawmates-agent-profile.json
- All 9 Playwright brand assertions updated in lockstep; historical
  spec document left untouched as the only remaining 'TeamClaw'
- Local env migrated: dev pg clawmates-dev-pg/clawmates_dev, shared
  test server clawmates-test-pg, kind cluster recreated with image +
  profile, compose images rebuilt under clawmates/*

Verified end to end: 161 Rust + 68 frontend tests, 29 Playwright
journeys, 4 live kind tests, helm/install/LOC/placeholder gates, and
the clean-room install rehearsal serving the clawmates login page from
a signed bundle of the rebuilt images.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 12:31:25 -05:00