Commit Graph
66 Commits
Author SHA1 Message Date
Omar SobhandClaude Fable 5 8046853feb Post-1.0: OTLP tracing, broker image + compose service, install rehearsal
- tc-telemetry: fmt subscriber always; with [telemetry] otlp_endpoint
  set, spans batch-export over OTLP/HTTP. Tested against a REAL OTLP
  receiver decoding the actual protobuf (official proto types): the
  emitted span and service.name arrive on the wire. No endpoint = no
  export = no network (air-gap stance). tower-http TraceLayer gives
  every API request a span
- The broker finally has its own image (images/broker.Dockerfile,
  9.5MB from scratch) — the Helm chart referenced one that never
  existed — and the compose deployment now RUNS the broker, sharing a
  socket volume with the server (the unix-socket equivalent of the K8s
  sidecar). Compose secret flows were silently dead before this
- server.Dockerfile fixes surfaced by the rehearsal: the workspace
  build needs tools/ (bundler joined the workspace) and
  images/seccomp/ (include_str! profile) in the build context
- scripts/rehearse-install.sh (plan: clean-VM rehearsal): assembles a
  REAL signed bundle from the built images (server/frontend/broker/
  postgres/socket-proxy), runs the customer path — offline verify,
  docker load, compose up — and asserts /healthz plus the served login
  page before teardown. Passing locally; wired as a release.yml step,
  which also builds/ships the broker + socket-proxy images now

161 Rust tests + 29 journeys; clean-room rehearsal green.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 11:35:32 -05:00
Omar SobhandClaude Fable 5 05e9612688 Post-1.0: Localhost seccomp on K8s, warm sandbox pool, server HPA
- K8sDriver.with_localhost_seccomp(profile): sandbox pods run under the
  STRICT allowlist instead of the runtime default. Proven live on kind:
  the harness installs the profile onto the node, a pod runs ordinary
  work as uid 10001, and unshare is kernel-denied inside the pod — the
  same probe the Docker suite uses, now passing on both targets
- Helm: sandbox.seccomp=localhost renders a DaemonSet that installs the
  chart-shipped profile into /var/lib/kubelet/seccomp on every node
  (ConfigMap + hostPath); ci/check-helm.sh enforces the chart copy stays
  byte-identical to images/seccomp/agent-profile.json and asserts the
  hardened render (DaemonSet + profile + HPA)
- server HPA (autoscaling/v2, CPU target) behind
  server.autoscaling.enabled
- SandboxManager.warm(n): a background warmer keeps n pre-provisioned
  sandboxes ready so an agent's first exec skips container startup;
  unhealthy pool entries are discarded, reuse never drains the pool,
  shutdown destroys assigned AND pooled. [sandbox] warm_pool config
  (default 0). Real-Docker test: prefill -> assign -> refill -> reuse ->
  clean shutdown

160 Rust tests + 4 live kind tests.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 11:15:49 -05:00
Omar SobhandClaude Fable 5 cbc8d35a2e Clerk authentication: hosted-identity session JWTs as a first-class mode
- tc-auth JwtVerifier: OIDC discovery -> JWKS, RS256 with the issuer
  pinned, 5s leeway (the crate's default 60s would double the life of
  Clerk's 60s session tokens), key cache with one refresh on unknown kid
  (Clerk rotates). Serves auth.mode = clerk AND generic oidc — a Clerk
  instance IS an OIDC issuer, so one verifier covers both
- AuthService.authenticate dispatches: JWT-shaped bearers take the
  hosted-identity path, everything else stays a local opaque session.
  External users JIT-provision keyed by the stable sub claim
  (users.auth_subject, unique partial index in migration 0007); an
  existing local account with the same email is LINKED, not duplicated;
  role tracks the issuer claim every request (org:admin -> Owner)
- Config auth.mode = "clerk" (requires issuer_url; validated), server
  pins the issuer at boot, Helm values/configmap accept mode=clerk
- Tests with REAL crypto, no mocks: fresh RSA keypairs, a live local
  issuer publishing real discovery + JWKS docs, Clerk-shaped tokens —
  JIT + role mapping, repeat-subject no-dup, expired refused (leeway
  regression), wrong-key forgery refused, foreign issuer refused, and
  the full router round trip with Authorization: Bearer <session JWT>
- docs/clerk.md: dashboard session-token customization (email + org
  role claims), config, @clerk/nextjs getToken() wiring, what CI proves

157 Rust + 63 frontend tests + 29 journeys. Air-gapped installs keep
local auth — Clerk is a cloud-only alternative, not a replacement.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 10:29:27 -05:00
Omar SobhandClaude Fable 5 4f253bec93 P6: browser.goto — real Chromium browsing with live web taint
- SandboxSpec gains an egress flag (default false — the kernel suite
  still proves zero-network for agent sandboxes). Egress-enabled
  containers exist ONLY for the browser: no credentials, no broker
  route, bridge network with host-gateway alias for local test pages
- images/agent-browser: Alpine Chromium, uid 10001, setuid bits
  stripped — same non-root hardening as agent-base
- browser.goto tool: headless chromium --dump-dom in the agent's
  browser container; HTML stripped to readable text (4k cap) and
  returned with output_taint=web; viewport screenshot captured,
  base64'd out of the container, stored in the blob store
- Taint semantics tightened: the step that PRODUCED untrusted output
  now carries its own taint (recorded before the step row), not just
  later steps — chat.inbox test updated to the stricter §15 reading
- GET /api/claws/{id}/browser/viewport.png serves the latest capture;
  BrowserApp polls it and renders the live viewport (spec §7.1),
  keeping the empty state until the agent has browsed
- Proven end to end with REAL Chromium against a REAL local page:
  content 'Revenue up 14 percent' returned tainted web; the gated
  email.send that follows carries 'web' in its approval taint_sources
  (untrusted content can never quietly reach outward); screenshot
  verified by PNG magic bytes

152 Rust tests + 63 frontend + 27 Playwright journeys.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 09:41:40 -05:00
Omar SobhandClaude Fable 5 7392ce1d08 P6: shell.exec — agents execute code in their real hardened sandbox
- SandboxManager (tc-runtime): one container per agent, provisioned
  lazily on first use, reused for the manager's lifetime, replaced
  transparently if dead, destroyed on shutdown
- shell.exec tool: sh -lc inside the agent's sandbox; stdout/stderr/
  exit_code return to the model as the step output. No external effects
  declared — the sandbox boundary (uid 10001, no caps, seccomp
  allowlist, read-only rootfs, zero egress) is the §15 control here,
  not an approval gate
- RuntimeConfig.sandboxes (+ with_sandboxes builder); [sandbox] config
  {image, enabled}; the server connects the Docker driver at boot and
  tolerates an absent engine (shell.exec reports it per-call)
- Tests with the REAL DockerDriver: a scripted run executes two
  commands — output proves uid 10001 from inside, and /home/agent state
  written by the first call is read by the second (same sandbox); a
  deployment without a sandbox runtime records honest error steps and
  the run still completes

151 Rust tests + 27 Playwright journeys.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 09:25:38 -05:00
Omar SobhandClaude Fable 5 70ec39f696 P6: S3 blob store, Helm chart, air-gapped installer verify loop
- S3BlobStore (object_store, path-style) behind the same BlobStore trait,
  tested against a REAL MinIO container (round trip, overwrite, NotFound
  on get and delete, nested keys); [storage] backend=local|s3 config with
  validation + server-side selection (S3 creds via env overlay)
- Helm chart: server pod with the secret broker as a SIDECAR sharing a
  private emptyDir unix socket (no network hop carries credentials),
  frontend, optional local PVC vs S3, OIDC/oauth values, unbuffered-SSE
  ingress annotations, NetworkPolicies (frontend->server only), hardened
  securityContexts; ci/check-helm.sh lints AND asserts the rendered
  topology properties
- deploy/airgapped/install.sh: offline signature+checksum verification via
  the bundled teamclaw-bundler BEFORE any docker load; --verify-only mode;
  ci/test-install.sh rehearses clean/tampered/wrong-key paths with the
  real binary
- CI: helm gate + installer rehearsal wired in

149 Rust tests; helm lint + rendered assertions green; installer
verify-path rehearsal green.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 08:19:43 -05:00
Omar SobhandClaude Fable 5 a8efada690 P5 exit: usage metering, credit billing, promo codes, 3-step wizard
- LlmEvent::Usage across all three providers (Scripted deterministic
  word-count accounting; Anthropic message_start/delta usage; OpenAI-compat
  stream_options include_usage)
- tc-billing: ceil(tokens/1000) min 1 credit; lots drain oldest-first under
  FOR UPDATE; balance clamps at zero while the usage ledger records the
  full obligation; promo codes redeem exactly once via CAS (migration 0006)
- Runtime charges every completed run (billing failure never fails a run);
  proven: 1 token in + 3 out -> 1 credit deducted
- API: GET /api/team/usage, POST /api/credits/redeem (409 on reuse, audited)
- Credits page: balance, 7-day usage meter with runway estimate, PromoRedeem
- /claws/new is the full §9 wizard: ?step=identity|access|slack deep-linked
  progress, accent swatches + name randomizer, access toggles, optional
  Slack step, explicit review-and-confirm (creation = live agent), animated
  provisioning state -> straight into chat
- E2E: chat decrements the visible balance and fills the usage meter;
  WELCOME500 adds exactly 500 once then refuses; wizard round trip

140 Rust + 63 frontend tests + 23 Playwright journeys.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 07:19:34 -05:00
Omar SobhandClaude Fable 5 91327e3618 P4 complete: OAuth authorization-code flow, MCP-OAuth, AddApps connects
- migration 0005 oauth_states: one-time states (10-min TTL), consumed by a
  CAS DELETE on callback — replays and forgeries both 404
- POST /api/apps/oauth/start: OIDC discovery on the configured issuer (or
  the custom MCP issuer for authType=mcp_oauth), state row, authorize URL
- GET /api/apps/oauth/callback: code exchanged at the REAL token endpoint
  (client id+secret form POST); the access token goes straight to the
  broker (test proves it never appears unencrypted in Postgres); connection
  row + audit; redirects to the claw's Add Apps panel
- [oauth] config (issuer/client/redirect_base) wired through AppState
- Tests against a real local IdP server (discovery + validating token
  endpoint): full round trip, broker-held token, replay/forged state
  refused, bad code fails exchange, mcp_oauth uses the custom issuer while
  plain oauth refuses without a configured IdP
- AddAppsApp: live connection badges + inline API-key connect per app
  (E2E: connect Notion by key from the directory)

136 Rust + 63 frontend tests + 21 Playwright journeys.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 06:50:41 -05:00
Omar SobhandClaude Fable 5 000b9b3a4b P4 core: broker-held app connections + gated, broker-executed Slack posting
- app_connections repo; POST /api/apps/connect (keys/basic): the credential
  goes to the secret broker over its socket and only the encrypted ref lands
  in the row; disconnect endpoint; /api/apps directory merged with live
  connection status; audit rows for connect/disconnect
- Broker protocol: InvokeHttp carries a JSON body
- slack.post tool (SendsExternally -> gated): marked broker_executed — the
  runtime skips its own grant consumption and the BROKER independently
  verifies + consumes the single-use grant, then calls Slack with the bot
  token injected; the runtime never sees the credential
- Config: [broker] socket_path + [slack] base_url; e2e harness spawns the
  real teamclaw-broker daemon and the server hosts an e2e-only /__slack sink
- SlackApp: Connection tab stores the token via the broker; connected state
- Integration test: blocked while pending -> approved -> sink received
  exactly one post with 'Bearer xoxb-test-token' -> grant replay refused
- E2E journey: connect Slack in the panel -> gated post card with preview ->
  sink empty while pending -> approve -> exactly one post, queue clear

133 Rust + 63 frontend tests + 21 Playwright journeys.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 05:51:19 -05:00
Omar SobhandClaude Fable 5 1fd2c287f1 P3 exit: Computer panel with all 8 apps, themed and deep-linkable
- DevicePanel in the 448px SlidePanel: ?app= routes (home + 8 sub-apps via
  dynamic imports), ?device=full|tablet|phone size toggle, per-agent
  accent-derived wallpaper theme + feTurbulence grain, glassy dock + grid
  home screen, Computer button in the chat header
- Apps: Files (3 drives, real listings), Skills (installed + add from
  library), Routines (list/refresh/empty state), Claw Chat (threads +
  sensitive badge + detail), Settings (push/pop nav: edit profile PATCHes
  the system prompt, Other-Claws access toggle PUTs the policy, confirmed
  destructive delete), Slack (§7.3 pre-connect gate), Add Apps (live
  /api/apps directory + search), Browser (chrome + spec'd empty state)
- /skills Skill Library page + nav entry; curated /api/apps directory
  endpoint; e2e seed gains a catalog skill
- P3 exit E2E (6 journeys): themed home screen + device toggle in URL,
  agent-written file appears in Files, agent-scheduled routine appears in
  Routines, system-prompt edit persists across reload, deep-link cold-load
  of ?app=settings&device=full, every app reachable, library installs

132 Rust + 63 frontend tests + 20 Playwright journeys.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 05:37:18 -05:00
Omar SobhandClaude Fable 5 67f918439c P3 backend: files, skills, routines, claw chat with LIVE taint plumbing
- tc-files: BlobStore trait + LocalBlobStore (traversal-proof keys); wired
  through Runtime (config storage.data_dir in deployments)
- File tools: files.write/files.list (workspace-internal) + files.delete
  (gated FileDeletion — tested: file survives pending, gone after approve);
  GET /api/openclaw/files + /api/shared-drive/files (drive/agent scoped)
- Skills: catalog/library + idempotent install with counter, uninstall;
  GET /api/skills[?clawId=], POST install/uninstall
- tc-scheduler: croner cron math (clock-controlled tests), SKIP LOCKED
  claim-and-advance firing REAL runs into dedicated ' name' sessions
  (reused, exactly-once), paused routines skipped; routines API + agent
  tool routine.schedule; loop spawned in server
- Claw chat: 1:1 threads, chat.send enforcing the target's Other-Claws
  policy, chat.inbox whose output carries inter_agent taint; the run loop
  now ACCUMULATES taint from tool outputs into LoopState, classifies with
  it, and stamps steps + approvals — a poisoned inbox followed by
  email.send produces an approval whose taint_sources says inter_agent
- ScriptedProvider scenario selection now keys on the most recent marker
  (session history kept earlier markers alive)

132 Rust tests green.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 05:28:50 -05:00
Omar SobhandClaude Fable 5 ea5162ac65 P2 complete: Docker sandbox with kernel assertions + secret broker
tc-sandbox:
- SandboxSpec/SandboxDriver + DockerDriver (bollard): uid 10001, cap-drop
  ALL, no-new-privileges, embedded seccomp deny profile (unshare/ptrace/
  bpf/keyctl/mount/...), read-only rootfs with tmpfs /tmp + /home/agent,
  network=none, mem/cpu/pids limits
- agent-base image: non-root, all setuid binaries stripped
- 6 kernel-level assertion tests probing from INSIDE real containers:
  uid + CapEff==0, rootfs read-only, seccomp EPERM on unshare, zero
  traffic-carrying interfaces + failed egress connect, no setuid +
  NoNewPrivs=1, lifecycle

tc-secrets:
- ChaCha20-Poly1305 envelope encryption under a FileKey (generated 0600,
  AEAD tamper detection tested); secrets table ciphertext-at-rest
- teamclaw-broker daemon: length-prefixed JSON over a unix socket; no
  protocol operation ever returns plaintext; InvokeHttp independently
  consumes the single-use execution grant against Postgres BEFORE touching
  any credential, then performs the call itself with the secret injected
- Tests over the real socket + real Postgres + a real local HTTP receiver:
  encrypted at rest, pending approval refused, approved call carries the
  bearer token exactly once, grant replay refused, non-http URLs rejected

116 Rust + 61 frontend tests + 14 E2E journeys green.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 05:07:46 -05:00
Omar SobhandClaude Fable 5 de38449b41 P2 BLOCKING exit green: approval interception chain end-to-end
- tc-tools: Effect declarations -> §15 GatedCategory mapping, deny-by-default
  external reach, taint invariant property-tested (tainted external effects
  are NEVER auto-allowed)
- tc-safety: pending approvals with exact payload+preview, CAS decide with
  audit + single-use grant in one tx, checkpoint suspend/load, exclusive
  resume claim, expiry sweep, decided-unresumed work queue (migration 0004
  adds the outbox the gated email.send tool writes)
- tc-runtime: resumable LoopState checkpointed to agent_runs; gated tool ->
  approval row -> approval_required/run_suspended events -> suspend; resume
  consumes the grant BEFORE executing (spent grant = no execution), rejection
  feeds a structured refusal in-band; durable resume sweeper; continuous
  journal seq across suspension (tested). ContentPart::Text became a struct
  variant — internally-tagged newtype primitives don't serialize
- tc-api: GET/decide approvals endpoints (409 double-decide, tenant
  isolation), decision triggers in-process resume; full chain proven over
  HTTP incl. gateway resumeFrom continuation
- frontend: approval_required/run_suspended events, suspended reply state,
  inline ApprovalCard (§10: summary, category, exact payload preview,
  approve/reject -> decide + stream re-attach), /approvals queue page, nav
- E2E (14 journeys, workers:1 to serialize the shared backend): gated email
  blocks with disabled composer -> approve -> continuation + ✓ step + reload
  replay; reject -> ✗ step, nothing executed; queue page decides pending

106 Rust + 61 frontend tests + 14 Playwright journeys green.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-10 04:58:47 -05:00
Omar SobhandClaude Fable 5 32008c9ef0 P1 backend: chat persistence, tc-llm providers, runtime loop, gateway SSE
- tc-db: sessions/messages/steps/runs/run_events repos (atomic seq assignment,
  history with ordered step traces, journal replay-from-offset); migration 0003
- tc-llm: provider-neutral ChatRequest/LlmEvent; ScriptedProvider (scenario
  TOML, word-level deltas, multi-turn tool legs — ships in production for
  e2e/air-gap smoke), AnthropicProvider (Messages SSE), OpenAiCompatProvider
  (vLLM/Ollama/llama.cpp); opt-in live tests via TC_LIVE_LLM=1
- tc-runtime: run loop with persist-before-emit event journal, real built-in
  clock.now tool, step rows on the reply message, tool-error resilience,
  broadcast channels for live attach
- tc-api: agent CRUD + settings/full (tenant-isolated, RBAC'd, audited),
  sessions create/list/history?tools=true, POST /api/gateway SSE with
  monotonic ids and exact resumeFrom journal replay (tested equal to live)
- teamclaw-server: config-driven provider factory

83 Rust tests green, all against real Postgres / real TCP.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-09 23:16:06 -05:00
Omar SobhandClaude Fable 5 fc173f170d P0 exit: e2e harness, Playwright shell journeys green, deploy skeleton
- teamclaw-server e2e mode (TEAMCLAW_MODE=e2e): idempotent deterministic
  seed through the real registration paths
- Playwright suite (6 journeys) against the real backend + prod Next build:
  login redirect, bad-password error, shell/roster/online-dot, team members,
  seeded credits, sign-out revocation — P0 exit criterion met
- Dockerfiles: musl-static server -> distroless, Next standalone -> distroless
  node (multi-arch via TARGETARCH)
- Air-gapped compose topology with edge/core/sandbox_net/secrets_net
  segmentation (engine-validated in CI), config-file + env-overlay pattern
- CI: compose validation + e2e job with trace upload

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-09 22:48:28 -05:00
Omar SobhandClaude Fable 5 c4349bf292 P0: tc-auth local sessions, tc-api P0 endpoints, teamclaw-server binary
- tc-auth: argon2id passwords, hashed opaque bearer tokens in auth_sessions
  (migration 0002), anti-enumeration login errors, redacted token Debug
- tc-api: axum router with /healthz, /api/auth/login|logout, /api/user/me,
  /api/team/{claws,credits,permissions}; Authed bearer extractor + RBAC
  permission derivation; integration-tested over real TCP vs real Postgres
- teamclaw-server: config -> pool -> self-migrate -> serve

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-06-09 22:30:33 -05:00