Commit Graph
14 Commits
Author SHA1 Message Date
Omar SobhandClaude Opus 5 9c4b0722e8 feat(judge): an independent GLM judge, on the newest model z.ai publishes
deploy / test (push) Successful in 3m58s
deploy / build (push) Successful in 5m29s
Every phase verdict this session was Anthropic grading Anthropic, and the boot
log said so on each start:

    validator_preflight: no CLAWMATES_VALIDATOR_MODEL — phase verdicts are
    judged by the house model, which is NOT an independent check

`evaluator.rs` already preferred a cross-provider judge and refused to call a
same-family one `independent`; the local stack simply had no non-Anthropic
credential. It now carries the same `glm` provider gw-04 has had all along —
`format = "anthropic"` is load-bearing, since z.ai's OpenAI-compatible endpoint
is ToS-throttled for raw SDK access while its Anthropic-format one is not.

Model: glm-5.3, the newest z.ai lists (4.5, 4.5-air, 4.6, 4.7, 5, 5-turbo,
5.1, 5.2, 5.3 as of 2026-08-17). gw-04 still runs glm-4.7.

glm-5.3 is a REASONING model: it emits a `thinking` block before its JSON. Our
SSE parser ignores `thinking_delta` and keeps the text, so the wire shape is
compatible — but on a realistic phase-evidence prompt it spent 819 of the
evaluator's 1024 output tokens. A longer phase would truncate the verdict, and
a truncated verdict parses as empty and FAILS CLOSED, burning one of the
phase's passes on a judge that never answered — precisely how mission 01a00bbb
lost a pass. max_tokens raised to 2048.

Measured before wiring: asked to judge 25 commits claiming INT-01..INT-25 with
tests passing, glm-5.3 returned met=false because the evidence never
established what the brief actually required. That skepticism is the point of
an independent judge.

Boot now reports: `validator_preflight: independent validator glm:glm-5.3
answered`.

The key lives in .env (gitignored), never in this file.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-16 21:41:45 -07:00
Omar Sobh 69a6e4e7f2 missions: sweeper + socket-proxy NETWORKS grant + mount ordering (C3 slice 4-5)
ci / gates (push) Successful in 6s
ci / rust (push) Failing after 10s
ci / frontend (push) Successful in 27s
ci / e2e (push) Skipped
ci / publish (push) Skipped
- mission_runtime::spawn_sweeper: force-removes runtime containers
  for missions terminal for >=30 min, clears runtime_endpoint. Wired
  into clawmates-server main().
- docker-compose socket-proxy: NETWORKS=1 so bollard.connect_network
  can attach containers to clawmates_edge for provider egress.
- phase_runner ordering: ensure_checkout BEFORE ensure_container so
  the mission dir exists before docker mounts it.
- provisioner: mkdir_p the mission dir defensively for research-only
  missions that skip checkout entirely.
2026-07-21 22:33:44 -07:00
Omar Sobh 1cbbbbd3e5 refine: switch from Gemini to Claude Opus 4.8
ci / gates (push) Successful in 7s
ci / rust (push) Successful in 4m25s
ci / e2e (push) Skipped
ci / frontend (push) Successful in 27s
ci / publish (push) Successful in 4m16s
Prod's Gemini prepayment credits are depleted (429 on every refine
attempt). Switching to Anthropic Claude Opus 4.8 for the mission
Refine flow — ANTHROPIC_API_KEY is already set in prod for ZeroClaw's
provider config, so no new secret plumbing.

  - crates/cm-api/src/mission_refiner.rs:
    * DEFAULT_MODEL: gemini-2.5-flash → claude-opus-4-8
    * call_gemini → call_anthropic against
      https://api.anthropic.com/v1/messages with the standard
      x-api-key + anthropic-version headers
    * Response parser reads content[type='text'].text (Messages API
      block shape) instead of Gemini's candidates path
    * Timeout raised 60s → 90s (Opus can be slower than Flash on
      long briefs; still bounded so a stuck call fails fast)
  - deploy/compose/.env.example: doc block rewritten. Refine now
    reuses ANTHROPIC_API_KEY; Level-Up keeps GEMINI_API_KEY because
    it needs JSON-mode structured output.

Level-Up is NOT switched in this commit — it uses Gemini's JSON mode
which has no drop-in Anthropic equivalent (needs tool-use rewrite).
Filed as a separate concern; Refine is what was actively broken.

Verified: SQLX_OFFLINE=true cargo check -p cm-api clean;
cargo fmt --all clean.
2026-07-20 14:04:43 -07:00
Omar Sobh c0a4fbc943 compose: switch missions_workspaces to host bind-mount
Named docker volume required the separately-managed clawmates-runtime
container to know the volume's on-disk path (varies by docker root).
A predictable host path (/var/lib/clawmates-missions) means the
runtime's spawn command can bind-mount the same path directly.
2026-07-20 04:15:08 -07:00
Omar Sobh 214d0c5e9f task #25: per-mission repo checkout on mission launch
Closes the follow-up gap flagged when task #23 landed. security_scan
and benchmark_runner now exec against $CLAWMATES_MISSIONS_ROOT/
{mission_id}/repo — this commit is what actually puts a repo there.

  - crates/cm-api/src/mission_workspace.rs — new module.
    ensure_checkout(pool, workspace_id, mission_id):
      * mission with no repo_id → Ok(None), no-op
      * repo cloned into $ROOT/{id}/repo (--depth 1)
      * dir already a git repo → fetch + reset --hard origin/{branch}
        (idempotent — every launch brings the tree in sync with the
        remote default_branch)
    Auth uses the process's ambient git credential setup (SSH agent /
    .netrc / helper). Tokens deliberately not embedded in URLs.

  - crates/cm-api/src/mission_orchestrator.rs — on_launch calls
    ensure_checkout after team materialization + team_id bind.
    Non-fatal: clone failures log and continue so research_only
    missions (no repo needed) don't get blocked.

  - deploy/compose/docker-compose.yml — new named volume
    missions_workspaces mounted at /var/lib/clawmates-missions on
    both the server (writer) and where the clawmates-runtime
    container will mount it (reader for docker exec). CLAWMATES_
    MISSIONS_ROOT + CLAWMATES_RUNTIME_CONTAINER env vars set on
    the server so mission_workspace + exec_target read the same
    canonical values.

The scan/bench trigger buttons now actually produce findings once
you (a) run a mission whose repo_id is set, (b) have the
clawmates-runtime container bind-mounting missions_workspaces at
/var/lib/clawmates-missions.

Verified: SQLX_OFFLINE=true cargo check -p cm-api +
cargo test -p cm-api --test mission_orchestrator both green.
2026-07-20 04:04:36 -07:00
Omar Sobh 5b7cb55d21 mount LevelUpInbox + document Gemini env vars
1. Dashboard.tsx — mount LevelUpInbox as a bottom panel on the
   MISSIONS tier sidebar (below MissionsList, above the tier rail).
   Capped at 38% height so it never crowds the mission list; scrolls
   independently when the proposal count grows.

2. deploy/compose/.env.example — document GEMINI_API_KEY +
   CLAWMATES_REFINER_MODEL + CLAWMATES_LEVEL_UP_MODEL. Refine + Level-
   Up silently 500 without the key; the model overrides default to
   gemini-2.5-flash so setting the key is the only required step.

Closes the two loose ends I called out last turn (inbox not mounted,
env vars undocumented). The full flow — Refine → mission launch →
security scan / benchmark → level-up review — is now wireable
end-to-end on a fresh deploy just by copying .env.example → .env
and setting POSTGRES_PASSWORD + GEMINI_API_KEY.
2026-07-19 19:14:25 -07:00
Omar Sobh b431d00f1a deploy: put the broker on both core + edge networks
ci / gates (push) Successful in 7s
ci / frontend (push) Successful in 25s
ci / rust (push) Successful in 4m5s
ci / e2e (push) Has been skipped
ci / publish (push) Successful in 21s
Broker acts as the §14 door for outbound provider fetches (Gitea +
GitHub + GitLab repo lists, Slack sends, OAuth token exchange). The
prior 'internal: true'-only core network gave the container no route
to the internet, so BrokerClient::fetch_authorized() failed at DNS
before it could reach api.github.com or a self-hosted Gitea.

Adding edge keeps the broker inbound-tight (still only listens on the
unix socket inside broker_run — no exposed TCP port) while granting
outbound. Postgres access via core is preserved.

Applied by hand to /opt/clawmates/docker-compose.yml on gw-04 to
unblock the current repo-connect flow; committing here so the next
re-provision doesn't regress it. The auto-deploy timer only pulls
images — compose file drift lives with us until a future
compose-sync step is added.
2026-07-07 17:03:53 -07:00
Omar SobhandClaude Opus 4.8 148705769f Reaping: sandbox orphan reaper + DB expiry/retention sweeper + volume-init retry
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
Closes the cleanup gaps found in review (latent today; bites under load/crashes).

Sandbox containers (cm-sandbox / cm-runtime):
- label every sandbox `clawmates.sandbox={agent|browser}` at create
- SandboxDriver::list_managed(kind) (Docker label filter + K8s label selector)
- SandboxManager::reconcile_orphans(ttl) + spawn_reaper: removes engine
  containers no live handle owns (ZERO = all)
- boot reconciliation (every pre-existing sandbox is an orphan from a dead
  process) + periodic reaper (5m interval / 10m TTL)
- SIGTERM graceful drain: serve().with_graceful_shutdown → shutdown() both
  managers so a redeploy can't leak; destroy errors now logged not swallowed

DB expiry/retention (cm-db cleanup.rs + cm-api cleanup_sweeper, hourly):
- expire auth_sessions + oauth_states past expires_at (security)
- prune sent outbox(7d), run_events(14d), routine_runs(30d), terminal
  topology_runs(90d), consumed execution_grants(7d)

Compose: volume-init restart "no" → on-failure:5 (retry instead of wedging boot).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-19 04:55:27 -07:00
Omar SobhandClaude Opus 4.8 e8486ee57c Door delivery: outbox drainer + SMTP transport (inert until configured)
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
The §15 door's email_send queues to `outbox` but nothing delivered it. Add a
real transport: cm-db outbox repo (list_queued/mark_sent/mark_failed) + a
cm-runtime drainer — an EmailSender trait (testable), a lettre STARTTLS
LettreSender, drain_once (queued -> sent/failed), and spawn_drainer wired into
the server beside the scheduler/sweeper/topology-worker.

Config-gated: inert (logs "outbox delivery DISABLED") until CLAWMATES_SMTP_*
is set, so it ships safely before credentials exist. The agent never holds the
SMTP credential — it only writes to outbox through the gated door; the server
owns the transport.

NOTE: live delivery is still credential-blocked — Migadu's API can't send
(SMTP-only) and the admin token is invalid; no SMTP creds exist. The transport
is built + tested (drain_marks_sent_and_failed via a mock sender); set
CLAWMATES_SMTP_* to go live with zero further code. clippy clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-18 03:22:27 -07:00
Omar SobhandClaude Fable 5 447f7039d8 Compose: production README, env knobs, first-owner bootstrap — deployed & live
Made the Docker Compose route turn-key for a real self-host, then stood
the whole stack up and drove a live chat through it.

- First-owner bootstrap (cm-auth::bootstrap_owner): a fresh local-auth
  install has no users and no signup route, so the initial Owner +
  workspace are provisioned ONCE from CLAWMATES_BOOTSTRAP_* env on first
  boot — idempotent, never clobbers an existing install (keys on 'any
  workspace exists'). Two real-Postgres tests (creates + signs in;
  second call is a no-op). Wired into server boot, guarded on a
  non-empty password
- deploy/compose/README.md: full production bring-up — services, the
  security topology, every config knob, Anthropic vs local-LLM, the
  broker-key backup, ops, and TLS/SSE proxy notes
- .env.example fleshed out (bootstrap, LLM, auth mode, OTLP); compose
  uses optional env_file so only the knobs you set are injected (unset
  options never override clawmates.toml with empty strings)
- volume-init one-shot chowns the broker's named volumes so the non-root
  scratch broker can write its socket + generated master key

Deployed locally and verified end to end: all 5 containers healthy,
broker generated its key, server bootstrapped owner@…, login + /api/user/me
work, and a real message streamed a live Anthropic response through the
gateway. Captured screenshots of login, workspace home, chat, and the
Computer panel.

166 Rust tests (+2 bootstrap).

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 14:06:15 -05:00
Omar SobhandClaude Fable 5 add4f79fed Rebrand: TeamClaw -> Clawmates (clawmates.work)
Full-depth rename per the approved plan; the 'claw' product vocabulary
(claws, /claws routes, clawId, Claw Chat) stays — it is now the brand.

- Display brand: Clawmates (manifest, titles, hero, login/rail logo
  'clawmates'); default host app.clawmates.work; registry
  ghcr.io/clawmates
- Crates tc-* -> cm-* (16 crates + all imports); binaries
  clawmates-server/broker/bundler; images clawmates/*; env prefix
  CLAWMATES_* (+ CM_TEST_DATABASE_URL / CM_LIVE_LLM); config
  clawmates.toml; helm chart deploy/helm/clawmates with clawmates-*
  resources; db names clawmates*; sockets /run/clawmates; cookie
  cm_session; kind cluster clawmates-test; seccomp node profile
  clawmates-agent-profile.json
- All 9 Playwright brand assertions updated in lockstep; historical
  spec document left untouched as the only remaining 'TeamClaw'
- Local env migrated: dev pg clawmates-dev-pg/clawmates_dev, shared
  test server clawmates-test-pg, kind cluster recreated with image +
  profile, compose images rebuilt under clawmates/*

Verified end to end: 161 Rust + 68 frontend tests, 29 Playwright
journeys, 4 live kind tests, helm/install/LOC/placeholder gates, and
the clean-room install rehearsal serving the clawmates login page from
a signed bundle of the rebuilt images.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 12:31:25 -05:00
Omar SobhandClaude Fable 5 8046853feb Post-1.0: OTLP tracing, broker image + compose service, install rehearsal
- tc-telemetry: fmt subscriber always; with [telemetry] otlp_endpoint
  set, spans batch-export over OTLP/HTTP. Tested against a REAL OTLP
  receiver decoding the actual protobuf (official proto types): the
  emitted span and service.name arrive on the wire. No endpoint = no
  export = no network (air-gap stance). tower-http TraceLayer gives
  every API request a span
- The broker finally has its own image (images/broker.Dockerfile,
  9.5MB from scratch) — the Helm chart referenced one that never
  existed — and the compose deployment now RUNS the broker, sharing a
  socket volume with the server (the unix-socket equivalent of the K8s
  sidecar). Compose secret flows were silently dead before this
- server.Dockerfile fixes surfaced by the rehearsal: the workspace
  build needs tools/ (bundler joined the workspace) and
  images/seccomp/ (include_str! profile) in the build context
- scripts/rehearse-install.sh (plan: clean-VM rehearsal): assembles a
  REAL signed bundle from the built images (server/frontend/broker/
  postgres/socket-proxy), runs the customer path — offline verify,
  docker load, compose up — and asserts /healthz plus the served login
  page before teardown. Passing locally; wired as a release.yml step,
  which also builds/ships the broker + socket-proxy images now

161 Rust tests + 29 journeys; clean-room rehearsal green.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 11:35:32 -05:00
Omar SobhandClaude Fable 5 a26939d7bc Plan hardening: socket-proxy allowlist, gateway load test, offline verify
- The compose server NEVER sees the raw Docker socket (plan risk #5):
  tecnativa/docker-socket-proxy on an isolated engine_net with exactly
  CONTAINERS/POST/EXEC/DELETE/VERSION allowed; server reaches it via
  DOCKER_HOST. DockerDriver honors DOCKER_HOST (connect_to). Proven by a
  REAL proxy test: full sandbox lifecycle works through the allowlist
  while /networks, /secrets, and /images all 403 — the blast-radius cap
  if the server is ever owned. (This also fixes compose deployments,
  where sandbox provisioning previously had no engine access at all.)
- Gateway load test (plan P6): 40 concurrent SSE streams against one
  server — every run completes with the full §13 event vocabulary,
  every journal strictly monotonic, every resumeFrom=0 replay byte-equal
  to its live stream
- release.yml: SBOMs (syft, spdx-json) for all four images shipped
  INSIDE the signed bundle; final verification now runs in a
  --network none container — proving the customer's verify path needs
  no internet, not just claiming it

159 Rust tests.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 10:50:55 -05:00
Omar SobhandClaude Fable 5 fc173f170d P0 exit: e2e harness, Playwright shell journeys green, deploy skeleton
- teamclaw-server e2e mode (TEAMCLAW_MODE=e2e): idempotent deterministic
  seed through the real registration paths
- Playwright suite (6 journeys) against the real backend + prod Next build:
  login redirect, bad-password error, shell/roster/online-dot, team members,
  seeded credits, sign-out revocation — P0 exit criterion met
- Dockerfiles: musl-static server -> distroless, Next standalone -> distroless
  node (multi-arch via TARGETARCH)
- Air-gapped compose topology with edge/core/sandbox_net/secrets_net
  segmentation (engine-validated in CI), config-file + env-overlay pattern
- CI: compose validation + e2e job with trace upload

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-09 22:48:28 -05:00