248948cc847b4d229291fa65785d940b02fd36ca
58
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
0b9baa942f |
build(runtime): follow ZeroClaw v0.8.5 onto Rust 1.98
v0.8.5 moved upstream's own container builders to rust:1.98-slim (#9527) and kept 1.96 only as the declared SOURCE floor - what the crates promise, not what upstream actually builds with. We were pinned at 1.96 and had never compiled this code on it; the local check ran on 1.97. Track upstream instead of trusting the floor, staying on the bookworm variant so the binary's glibc still matches the debian:bookworm-slim runtime stage. CARGO_BUILD_JOBS defaults to 6 because the whole fleet is offline and gw-04 is now both the only reachable x86_64 host and the box serving production, so a build must not take every core from the services running beside it. Built and deployed: clawmates-runtime:v085 reports zeroclaw 0.8.5, health 200 with every component ok including the new relay, pairing survived the recreate, and the claude_cli/kimi_cli slots still resolve alongside upstream's grok_cli. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01WZb5A2kfVfjpdwSochkuHz |
||
|
Omar SobhandClaude Opus 5
|
ea0b989b3f |
docs(research): missions DO call tools — the claim was wrong, and the truth is worse
Deep research into "missions can't call tools at all", which I wrote and which is false. docs/TOOL-CALL-ARCHITECTURE.md has the full findings. WHAT IS ACTUALLY TRUE Three of the four mission paths end in `claude -p` with Claude Code's own toolset and permissions PRE-ACCEPTED: solo microVM Read Edit Write Bash Agent --permission-mode acceptEdits composed microVM same, per node same direct session Read Edit Write Bash acceptEdits So the position is not "no tools". It is: mission agents run Bash and Write with permissions pre-accepted, and nothing in this platform can gate them. That is a stronger finding than the one it replaces — "can't call tools" sounds like a missing feature; "calls tools freely, ungated, and mostly unobserved" is a security posture, and it is ours. Observe and gate are different and both are partial. vm_tool_tap is a PostToolUse hook: it fires AFTER the tool ran and exit-0s unconditionally, so it is telemetry and structurally cannot gate. The direct-session tier has no tap at all. GatePolicy has exactly one enforcement site — the chat loop — and its approvals key on (session_id, message_id), which no mission phase can produce. WHY THE CONTAINER TIER LOOKED TOOL-FREE `claude_cli` runs `claude -p --output-format json`, which returns a single final result object, and the provider hardcodes `tool_calls: Vec::new()`. The calls happen; the transport discards them. The comment reading that emptiness as "§15 by construction: agents are provisioned tool-free" was inferring a design property from a serialization choice. Verified against the deployed Claude Code 2.1.228 rather than assumed: `--output-format stream-json --verbose` emits `tool_use` blocks with the tool name and `tool_result` blocks. The calls are fully observable; we ask for the wrong format. THE DOOR WE ALREADY BUILT AND NEVER PLUGGED IN claude_cli.rs is OURS — upstream zeroclaw-labs/zeroclaw has no such file — and so is 88eef99d4 "claude_cli --mcp-config + allow/disallow tools (act via door)". The provider already accepts mcp_config (claude's own MCP client reaches our door), tools, and disallowed_tools (lock out the natives so the gated door is the ONLY actuator). agent.config.example.toml documents the whole shape. In the live runtime: clawmates-mcp.json does not exist, there is no [providers.*] block, and every mission claw binds to claude_cli.default which sets none of it. My earlier "claude_cli cannot reach MCP, therefore the skills server is unreachable" was wrong in its reasoning — the capability is built, documented by us, and never deployed. Related: we set `agents.<alias>.mcp_bundles`, which configures ZeroClaw's OWN MCP client for its native loop. A claude_cli agent's actuator is the claude subprocess, which reads `mcp_config` on the PROVIDER. We were turning a knob wired to a loop that does not run. UPSTREAM 218 commits behind. No upstream work on claude_cli (the file is ours). ACP already exists in the fork; the three new commits are workspace-default and localization fixes, not new capability. The one item worth pulling is "feat(plugins): add shared egress policy foundation (#9137)" — a network guard with DNS pinning and metadata-address blocking, defence for the egress problem we have not solved. Stale claims corrected in place, in topology_exec.rs and the runtime config, so the codebase stops asserting the thing that is false. Recommended order, cheapest first: stream-json for observability; the PreToolUse hook for a real gate (it FIRES under claude -p per vm_stop_gate, and has zero call sites); then deploy the door. The executor swap is NOT recommended — the blockers are structural, not wiring, and the cheap fixes deliver what it was wanted for. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
e3247fee4b |
chore(runtime): define the skills MCP bundle the templates now ask for
provision_claw honours the template's bundle list as of the previous commit, but a bundle an agent is assigned and the runtime config does not define resolves to nothing — so the assignment had to be made to mean something on the MCP side too. Carries the caveat that matters at the point of use: this channel only works for a provider that can surface tool calls, and mission claws run on claude_cli, which is text-only. Their skills arrive as prompt text instead. The entry is for tool-capable agents, and so that an assigned name resolves. gitea_forge is left UNDEFINED on purpose, with a note. Six templates name it and nothing defines it; a plausible-looking definition pointing at the wrong URL would turn a name that resolves to nothing into a server that fails at call time, which is harder to notice rather than easier. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
9c4b0722e8 |
feat(judge): an independent GLM judge, on the newest model z.ai publishes
Every phase verdict this session was Anthropic grading Anthropic, and the boot
log said so on each start:
validator_preflight: no CLAWMATES_VALIDATOR_MODEL — phase verdicts are
judged by the house model, which is NOT an independent check
`evaluator.rs` already preferred a cross-provider judge and refused to call a
same-family one `independent`; the local stack simply had no non-Anthropic
credential. It now carries the same `glm` provider gw-04 has had all along —
`format = "anthropic"` is load-bearing, since z.ai's OpenAI-compatible endpoint
is ToS-throttled for raw SDK access while its Anthropic-format one is not.
Model: glm-5.3, the newest z.ai lists (4.5, 4.5-air, 4.6, 4.7, 5, 5-turbo,
5.1, 5.2, 5.3 as of 2026-08-17). gw-04 still runs glm-4.7.
glm-5.3 is a REASONING model: it emits a `thinking` block before its JSON. Our
SSE parser ignores `thinking_delta` and keeps the text, so the wire shape is
compatible — but on a realistic phase-evidence prompt it spent 819 of the
evaluator's 1024 output tokens. A longer phase would truncate the verdict, and
a truncated verdict parses as empty and FAILS CLOSED, burning one of the
phase's passes on a judge that never answered — precisely how mission 01a00bbb
lost a pass. max_tokens raised to 2048.
Measured before wiring: asked to judge 25 commits claiming INT-01..INT-25 with
tests passing, glm-5.3 returned met=false because the evidence never
established what the brief actually required. That skepticism is the point of
an independent judge.
Boot now reports: `validator_preflight: independent validator glm:glm-5.3
answered`.
The key lives in .env (gitignored), never in this file.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
53da4d7e6d |
fix(runtime): a mission could not build the repo it was given
`clawmates-runtime` shipped with `gcc` and `make` but no `cmake`, no `g++` and no `python3-dev`. Measured on clawhdf5, three probes: no cmake → "is `cmake` not installed?" exit 101 after 13s no python3-dev → "cannot find -lpython3.11" exit 101 at link with both → cargo test PASSES exit 0 after 69s This is not only the delivery gate. The AGENTS run in this image, so a coding phase was writing Rust it had no way to compile or test — which reframes the last run's 11 agent commits as unverifiable by construction. `images/agent-toolchain/Dockerfile` (the microVM path) has had `cmake build-essential` all along, and its own header warns about precisely this: "if `cargo` is present in one image and absent in another, the same mission passes or fails depending on which backend it landed on, and nothing says why." Both images now install the same set — it was missing `python3-dev` too. `images/runtime-toolchain.Dockerfile` is a thin local overlay so the laptop can run today without recompiling zeroclaw from the fork; it is meant to be deleted once a runtime image built from the corrected deploy/ Dockerfile is published. Also: a build failure is no longer reported as a red suite. Both are cargo exit 101, and `verify_tests` mapped every non-zero to `Failed(code)` — so a missing toolchain was recorded as the USER's tests failing. It now returns `CouldNotRun` with the reason when the output shows a compile or link failure. Deliberately narrow: a failing `assert!` still reads as red, because letting broken code past `on_green_tests` is the expensive direction to be wrong in. Both directions are pinned by tests built from today's two real samples. And the coding phase finally has a loop: `research_and_code.toml` declared `loop = "until_no_more_int_items"`, which `phase_config.rs` lists as DECLARED_BUT_UNREAD. Iteration is driven by `max_iterations` + `done_when`, and with `max_iterations = 1` and no `done_when` the phase ran ONCE and was never judged — reporting `completed` whatever it produced. Now 3 passes against a stated goal, wording per the measured rule (say what the tree must CONTAIN). Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar Sobh
|
87f188ae73 |
refactor: strip Gemini from the platform, and level up the architecture_mapper
Two things.
1. The architecture_mapper proposal, applied AND made durable.
The GLM proposal (019fddd9) was accepted in full: the agent's system_prompt now
carries the Mermaid-first constraint and its brain was rewritten. Both verified
against the live row and the .h5 file.
But `apply_identity` writes `UPDATE agents SET system_prompt` and
`apply_brain_consolidation` writes that agent's brain — neither touches the team
TEMPLATE. That agent is mission-scoped, so the improvement would have died with
the mission. The model's actual insight was sharp and worth keeping: "Mermaid
diagrams beat prose" lived in the brain SEED and not in the system PROMPT, so it
only applied when the agent happened to consult its brain. That constraint is
now in templates/teams/codebase_research.toml, where every future Codebase
Research team inherits it.
(The proposal's second item mostly restated anti-patterns the seed already
lists, so the seed is unchanged. Applying an LLM's suggestion is not the same as
agreeing with all of it.)
2. Gemini is gone.
Removed: the `gemini.default` provider alias and its `is_exact_provider_match`
prefix, GEMINI_API_KEY forwarding to agent containers, the evaluator's
gemini->gemini family row, the model selectors in claws/teams/planner and in
TeamWizard + AgentComputer, and the commented provider block in the runtime
config example (whose ZEROCLAW_AGENT_MAP example still mapped a worker_gemini
that no longer existed).
`provider_alias_for("gemini")` now returns claude_cli.default via the
unrecognised-model branch, which LOGS. A stray gemini binding degrades visibly
rather than resolving to a provider row we no longer ship. A test pins that, and
another pins that GEMINI_API_KEY is forwarded in NEITHER auth mode, so adding it
back to the list is a visible change rather than an accident.
Avatar generation is DELETED, not disabled — it called Gemini's image model, and
there is no alternative: Claude and Kimi are text-only, and z.ai answers
"Unknown Model" for cogview-3-flash and cogview-4 on our plan (measured, not
assumed). AvatarModal keeps UPLOAD, which never needed a provider; only the
prompt-generation half is gone.
240 backend lib tests, 89 frontend tests, clean tsc + eslint, build succeeds.
|
||
|
Omar SobhandClaude Opus 5
|
3124fd3c8f |
feat(library): weekly harvest on a systemd timer
Monday 07:00, Persistent=true so a week missed to downtime fires on next boot rather than leaving a silently empty library. 30-minute timeout so a wedged run cannot hold the slot until the following week. The script is deliberately thin — it calls the API and reports — so it never needs changing when the harvest does. Auth is a long-lived operator session in /etc/clawmates/library.token (root-only, 600); rotate by replacing the file. Exit status follows `healthy`, not paper count. A mature library shelves nothing most weeks and that is success; a run that errored is a failure even if it shelved something. The first manual fire caught a real bug in this script, in the opposite direction to this week's usual: the harvest genuinely shelved 15 papers and pushed them, and the reporter crashed on an escaped quote inside an f-string, so systemd marked the unit FAILED. A false failure destroys trust in the signal exactly as a false success does. The reporter now avoids backslashes entirely (it is embedded in a single-quoted shell string) and was proved against the real response shape before being trusted. Verified end to end on gw-04: run 1: 25 candidates, 10 already held, 15 shelved, pushed run 2: 25 candidates, 25 already held, 0 shelved, no branch, healthy Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
409ca65ee7 |
fix(missions): capture from the clone point, and let agents create files
Two defects found by running a real coding mission (019fc372) rather than a test. Both made a coding phase look like it produced nothing. **Capture measured the wrong baseline.** It diffed the working tree against HEAD, which is correct only while work stays uncommitted. `rust_sdlc` has a *committer* role, so committing is the intended path — meaning a mission that did its job properly leaves a clean tree and captured nothing. That is exactly what happened: the agent created `DELIVERY_PROBE.md`, committed it as `aa3be95`, and the artifact recorded `empty: true` beside a commit that plainly contained the work. `mission_workspace` now records the clone point in `.git/clawmates-base` (in `.git/`, so it travels with the checkout, stays invisible to the repository, and cannot be reached by an agent through its pinned workspace), refreshed whenever `fetch_and_reset` moves HEAD. Capture diffs from there, covering committed, staged and unstaged changes in one pass. Checkouts predating the marker fall back to HEAD and say so via `base_recorded: false`. **Agents could not create files.** `coding_readwrite` granted `file_edit` but not `file_write`. `file_edit` replaces an exact existing string and rejects an empty `old_string`, so creating a new file was impossible. The mission transcript is unambiguous: "the tool rejected empty old_string... the shell is restricted", after which the agent worked around it through `shell`. The comment above that profile has claimed it grants file_write since the day it was written; the list never contained it. Also broadens capture from coding/benchmark/security_scan to every phase kind of a repo-bearing mission: `phase_task_text` tells research phases to "save findings under /mission/repo/research/", so filtering by kind would have discarded every research brief such a mission produced. Regression tests cover committed-only and committed-plus-uncommitted work against a real git repo. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
9f874bc06a |
feat(runtime): install the toolchain missions are told to use
`templates/teams/rust_sdlc.toml` instructs the coder to run `cargo test`; the `done_when` evaluator runs a project's own suite to verify a claim rather than believe it; `security_scan.rs` shells out to cargo-audit, gitleaks, trivy and semgrep. The runtime image contained none of them. The security consequence was the worse one. With no scanners present, a scan emitted four `<tool>:tool_error` task rows and completed — a scan that scanned nothing and reported cleanly. Same class of false signal as a verifier that never ran a command. Adds gitleaks 8.30.1, trivy 0.72.0, semgrep (in its own venv so its pinned dependency tree cannot collide), and a minimal Rust stable toolchain with cargo-audit. Versions are pinned as build args and were taken from the releases API — the first attempt used plausible-looking numbers that 404'd. Layers are ordered cheapest-and-most-stable first so bumping a scanner does not invalidate the Rust layer, and the cargo registry is dropped after `cargo install`. Measured: 864 MB -> 3.13 GB (scanners +350 MB, Rust +1.23 GB, semgrep +680 MB). Note this image is NOT in `AGENT_IMAGES` — it never ships to fleet nodes, only gw-04 holds it, against 112 GB free. An earlier note claiming otherwise was wrong. The real cost is a slower `docker save | load` per rebuild. Verified in the built image: rustc 1.97.1, cargo-audit 0.22.2, gitleaks 8.30.1, trivy 0.72.0, semgrep 1.172.0, python 3.11.2, plus the existing git, claude and node. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
34409bca0c |
fix(missions): grant coding tools + pin claw workspace to /mission/repo
Mission agents were burning ~275K tokens producing nothing: the coder had only file_read and its workspace was the empty ephemeral sandbox, so it dumped a full spec inline instead of writing files. Two root causes: 1. Risk-profile allowlists used pre-0.8 tool names. `coding_readwrite` allow-listed `file_write` (renamed to `file_edit` in ZeroClaw 0.8, and `file_write` now refuses on ephemeral workspaces) and omitted file_edit / content_search / glob_search / git_operations — the exact tools the phase prompt tells agents to use. Since allowed_tools is a strict allowlist, agents were effectively read-only. Documents the correct profiles in agent.config.example.toml (they only lived in host config; the live runtime profiles were corrected via its config API). 2. workspace.path never got set. `agents.<alias>.workspace.path` is an Option<PathBuf> the ZeroClaw Configurable macro skips from prop enumeration, so provision_claw's set_prop always 404'd and the whole call errored into a swallowed eprintln. Removes the dead set_prop and pins the workspace out-of-band: MissionRuntimeProvisioner:: pin_agent_workspaces patches the shared config file on the per-mission container (format-preserving via toml_edit, atomic temp+mv); the daemon applies it on the same reload that surfaces the freshly-provisioned claws. Covered by unit tests for the TOML stamp. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar Sobh
|
69a6e4e7f2 |
missions: sweeper + socket-proxy NETWORKS grant + mount ordering (C3 slice 4-5)
- mission_runtime::spawn_sweeper: force-removes runtime containers for missions terminal for >=30 min, clears runtime_endpoint. Wired into clawmates-server main(). - docker-compose socket-proxy: NETWORKS=1 so bollard.connect_network can attach containers to clawmates_edge for provider egress. - phase_runner ordering: ensure_checkout BEFORE ensure_container so the mission dir exists before docker mounts it. - provisioner: mkdir_p the mission dir defensively for research-only missions that skip checkout entirely. |
||
|
Omar Sobh
|
a1d1097b52 |
ci + ops: cargo-build retry wrapper + runtime systemd unit
Two durability fixes closing recurring flakes:
CI flake wrapper (broker + server Dockerfiles):
Wrapped the cargo build step in a 3-attempt retry loop with
linear backoff (10s / 20s). Directly targets the crates.io
transient network errors that keep hitting CI on the runners
('curl failed: SSL_ERROR_SYSCALL, errno 0'). Each build only
loses time on transient failures; a real compile error still
fails all 3 attempts and surfaces the last error normally.
Runtime systemd unit (deploy/clawmates-runtime/):
Replaces the manual 'docker run' that had been starting the
ZeroClaw runtime with no persistence for its network topology.
Ephemeral prod fixes at 09:30 PDT 2026-07-21 (task #38) were:
- anthropic.default provider block added to
/root/clawmates-runtime/data/.zeroclaw/config.toml (already
durable — bind-mounted from host)
- docker network connect clawmates_edge clawmates-runtime
(NOT durable — vanishes on container recreate)
New systemd unit clawmates-runtime.service (installed +
enabled on gw-04):
- ExecStart docker-runs the container attached to
clawmates_core, then connects clawmates_edge in the same
shell command, then docker waits.
- Bind-mounts both /root/clawmates-runtime/data and
/var/lib/clawmates-missions (for security_scan +
benchmark_runner).
- --rm so upgrading is just docker pull + systemctl restart.
- Restart=on-failure with 5s backoff.
Closes task #38 and preemptively closes the CI flake pattern.
|
||
|
Omar Sobh
|
1cbbbbd3e5 |
refine: switch from Gemini to Claude Opus 4.8
Prod's Gemini prepayment credits are depleted (429 on every refine
attempt). Switching to Anthropic Claude Opus 4.8 for the mission
Refine flow — ANTHROPIC_API_KEY is already set in prod for ZeroClaw's
provider config, so no new secret plumbing.
- crates/cm-api/src/mission_refiner.rs:
* DEFAULT_MODEL: gemini-2.5-flash → claude-opus-4-8
* call_gemini → call_anthropic against
https://api.anthropic.com/v1/messages with the standard
x-api-key + anthropic-version headers
* Response parser reads content[type='text'].text (Messages API
block shape) instead of Gemini's candidates path
* Timeout raised 60s → 90s (Opus can be slower than Flash on
long briefs; still bounded so a stuck call fails fast)
- deploy/compose/.env.example: doc block rewritten. Refine now
reuses ANTHROPIC_API_KEY; Level-Up keeps GEMINI_API_KEY because
it needs JSON-mode structured output.
Level-Up is NOT switched in this commit — it uses Gemini's JSON mode
which has no drop-in Anthropic equivalent (needs tool-use rewrite).
Filed as a separate concern; Refine is what was actively broken.
Verified: SQLX_OFFLINE=true cargo check -p cm-api clean;
cargo fmt --all clean.
|
||
|
Omar Sobh
|
0d4cb2c2bc |
herdr phase 0 follow-up: persistent daemon via systemd/launchd
Phase 0 started Herdr via nohup — died on reboot / logout. Replaces
that with proper service management:
- deploy/fleet/herdr-persistence/herdr.service — systemd user unit
(Linux). Restart=on-failure with an 8-in-24h burst cap so a
broken binary doesn't hot-loop. Requires linger enabled so the
user's systemd manager runs without a login session; install.sh
does that via loginctl.
- deploy/fleet/herdr-persistence/dev.herdr.plist — launchd
LaunchAgent (macOS). ProgramArguments + PATH templated so the
install script substitutes actual paths at deploy time.
- deploy/fleet/herdr-persistence/install.sh — idempotent installer
that autodetects OS, drops the unit/plist in the right place,
enables + starts, prints status.
Rolled to tank + architect + morpheus (systemd) + smith + macbook
(launchd). All 5 nodes confirmed `status: running` post-install.
Phase 0 gap closed: Herdr now survives node reboots and logouts,
which is the prerequisite for the fleet_herdr dispatch path to be
reliable across mission_orchestrator restarts.
|
||
|
Omar Sobh
|
c0a4fbc943 |
compose: switch missions_workspaces to host bind-mount
Named docker volume required the separately-managed clawmates-runtime container to know the volume's on-disk path (varies by docker root). A predictable host path (/var/lib/clawmates-missions) means the runtime's spawn command can bind-mount the same path directly. |
||
|
Omar Sobh
|
214d0c5e9f |
task #25: per-mission repo checkout on mission launch
Closes the follow-up gap flagged when task #23 landed. security_scan and benchmark_runner now exec against $CLAWMATES_MISSIONS_ROOT/ {mission_id}/repo — this commit is what actually puts a repo there. - crates/cm-api/src/mission_workspace.rs — new module. ensure_checkout(pool, workspace_id, mission_id): * mission with no repo_id → Ok(None), no-op * repo cloned into $ROOT/{id}/repo (--depth 1) * dir already a git repo → fetch + reset --hard origin/{branch} (idempotent — every launch brings the tree in sync with the remote default_branch) Auth uses the process's ambient git credential setup (SSH agent / .netrc / helper). Tokens deliberately not embedded in URLs. - crates/cm-api/src/mission_orchestrator.rs — on_launch calls ensure_checkout after team materialization + team_id bind. Non-fatal: clone failures log and continue so research_only missions (no repo needed) don't get blocked. - deploy/compose/docker-compose.yml — new named volume missions_workspaces mounted at /var/lib/clawmates-missions on both the server (writer) and where the clawmates-runtime container will mount it (reader for docker exec). CLAWMATES_ MISSIONS_ROOT + CLAWMATES_RUNTIME_CONTAINER env vars set on the server so mission_workspace + exec_target read the same canonical values. The scan/bench trigger buttons now actually produce findings once you (a) run a mission whose repo_id is set, (b) have the clawmates-runtime container bind-mounting missions_workspaces at /var/lib/clawmates-missions. Verified: SQLX_OFFLINE=true cargo check -p cm-api + cargo test -p cm-api --test mission_orchestrator both green. |
||
|
Omar Sobh
|
5b7cb55d21 |
mount LevelUpInbox + document Gemini env vars
1. Dashboard.tsx — mount LevelUpInbox as a bottom panel on the MISSIONS tier sidebar (below MissionsList, above the tier rail). Capped at 38% height so it never crowds the mission list; scrolls independently when the proposal count grows. 2. deploy/compose/.env.example — document GEMINI_API_KEY + CLAWMATES_REFINER_MODEL + CLAWMATES_LEVEL_UP_MODEL. Refine + Level- Up silently 500 without the key; the model overrides default to gemini-2.5-flash so setting the key is the only required step. Closes the two loose ends I called out last turn (inbox not mounted, env vars undocumented). The full flow — Refine → mission launch → security scan / benchmark → level-up review — is now wireable end-to-end on a fresh deploy just by copying .env.example → .env and setting POSTGRES_PASSWORD + GEMINI_API_KEY. |
||
|
Omar Sobh
|
cde196dbdc |
runtime: bake tea + gitea-mcp binaries + let GITEA_TOKEN pass through
Prep for the SDLC coding team. Adds two upstream Gitea binaries to the clawmates-runtime image so team-scoped agents can drive git.redclaw.dev without custom MCP code: - tea v0.14.2 (shell CLI: clone/push/pr checkout, git remote helper) - gitea-mcp v1.3.0 (native MCP server: list_repo_pull_requests, create_pull_request, create_file, update_file, create_branch, create_issue, get_file_content, etc.) Both installed by arch (amd64 / arm64) at image build time; also adds git to the runtime layer since it was missing (needed by tea's shell delegates). research_container::inherited_env now propagates GITEA_ prefixed vars so a team container inherits GITEA_TOKEN (+ optional GITEA_HOST) from the server env. Team-scoped daemons can then start gitea-mcp via stdio with --token-env GITEA_TOKEN. Followup on gw-04: 1. rebuild clawmates-runtime image against the new Dockerfile 2. add [mcp_bundles.gitea_forge] to the runtime template 3. set GITEA_TOKEN in the server compose env |
||
|
Omar Sobh
|
e3ef3fd056 |
research: fix Draft 'Invalid Date' + stale-error leak in pipeline card
Two cosmetic bugs surfaced by the successful v0.8.3 pipeline run: 1. **'produced Invalid Date'** — research_outcomes.created_at was an OffsetDateTime serialized by time's default array format (`[y, ordinal, hh, mm, ss, ns, tz]`), which browser's `new Date(...)` can't parse. Add `#[serde(with = "time::serde::rfc3339")]` matching the pattern already in threads.rs / routine_runs.rs. 2. **Stale error text on pipeline card** — the runs stage's `latest_error` walked every run by `created_at DESC` and returned the first non-empty error, so a topic with an earlier failed run + a later completed run kept displaying the old error next to '1 completed'. Now the error only surfaces when the MOST RECENT run itself failed. Historical failures stay in the run count but don't leak their message. |
||
|
Omar Sobh
|
067bebd071 |
wedged-run fix: pre-approve claude perms + verbose step log + reaper
Root cause: claude CLI in the per-topic research container runs as
uid=0(root). ZeroClaw's claude_cli provider passes
--dangerously-skip-permissions which Claude CLI rejects under root
for security — so the CLI hangs waiting for interactive permission
approval that never arrives, hitting the 600s provider timeout with
zero step records journaled.
Three-part fix:
1. Claude settings bind-mount (research_container.rs):
Optional CLAWMATES_CLAUDE_SETTINGS_PATH env — when set, mount the
host file at /root/.claude/settings.json (read-only) in every
spawned team container. deploy/claude-settings.json ships the
canonical config (permissions.defaultMode = bypassPermissions +
hasCompletedOnboarding). CLI accepts requests immediately with no
--dangerously-skip-permissions flag needed.
2. Verbose per-step log line (topology_worker.rs):
Every checkpoint now writes to stderr:
topology_worker::step run_id=X step=N node=Y role=Z phase=W
output_bytes=B tokens=T gated=G
Visible in docker logs clawmates_server_1 — gives us live
'topology is flowing' signal without opening the canvas, and
makes it obvious when a topology_kind is skipping stages it
shouldn't.
3. Stuck-container reaper (topology_worker.rs):
New 60s-tick loop reap_stuck_runs: for any research topology_run
older than 15 min with zero checkpoint.records, docker-stop its
container and mark the run failed with a diagnostic error. Only
reaps research-bound runs (non-research runs don't own a
container). The existing 180s stale-checkpoint requeuer stays
in place for other failure modes.
Deploy: gw-04 needs
ln -sf /path/to/repo/deploy/claude-settings.json /opt/clawmates/claude-settings.json
CLAWMATES_CLAUDE_SETTINGS_PATH=/opt/clawmates/claude-settings.json
in the server env, plus the timeout lowered from 600 -> 120 in
compose. Both handled in the deploy step outside this commit.
|
||
|
Omar Sobh
|
b431d00f1a |
deploy: put the broker on both core + edge networks
Broker acts as the §14 door for outbound provider fetches (Gitea + GitHub + GitLab repo lists, Slack sends, OAuth token exchange). The prior 'internal: true'-only core network gave the container no route to the internet, so BrokerClient::fetch_authorized() failed at DNS before it could reach api.github.com or a self-hosted Gitea. Adding edge keeps the broker inbound-tight (still only listens on the unix socket inside broker_run — no exposed TCP port) while granting outbound. Postgres access via core is preserved. Applied by hand to /opt/clawmates/docker-compose.yml on gw-04 to unblock the current repo-connect flow; committing here so the next re-provision doesn't regress it. The auto-deploy timer only pulls images — compose file drift lives with us until a future compose-sync step is added. |
||
|
Omar Sobh
|
298eb8e20e |
e2e: move backend/dex/frontend from 8080 to 18080 so runners don't collide
Two of the fleet's Gitea Actions runners (morpheus, architect) already had 8080 permanently bound by unrelated services (nginx on morpheus, envio-hasura on architect) — every e2e run scheduled there died at playwright's webServer preflight with "http://127.0.0.1:8080/healthz is already used". 18080 is unused across morpheus/tank/architect. Swap 8080 → 18080 in the eight e2e-scoped sites: clawmates.e2e.toml (listen_addr + slack base_url + oauth redirect_base), dex.yaml (client redirect URIs must match backend), playwright.config.ts + tests (p4-slack, p6-oauth), the http.ts dev-fallback origin, and the two shell scripts (e2e-backend safety check, rehearse-install healthz probe). Prod compose (/opt/clawmates/docker-compose.yml on gw-04) is untouched; prod continues to expose the server on 8080 internally on the compose network (that's per-network, not host-shared). |
||
|
Omar Sobh
|
8f29cf8e44 |
deploy(gw-04): run the timer as a dedicated clawmates user under /opt
Move the running stack off root ownership. The systemd service now runs as User=clawmates:clawmates with WorkingDirectory=/opt/clawmates, and the script's COMPOSE_DIR default follows. This closes the "rootful compose stack" ask from the original ship-readiness audit — deploys no longer require any part of the pipeline to run as root beyond docker access (the clawmates user gets that via the docker group). Docker-managed volumes (pgdata, broker_run, broker_key, brains, filedata) stay put; the compose project name is unchanged so docker resolves them to the same physical volumes. The old /root/clawmates directory stays in place as an emergency rollback for a week, then gets removed as follow-up. |
||
|
Omar Sobh
|
39a6424d72 |
deploy(gw-04): drop the un-prefixed retag bridge
The compose file on gw-04 was migrated to registry-prefixed image references (100.94.185.103:5000/clawmates/<svc>:latest), which lets `docker compose up` pick up the pulled image directly. The old script retagged each pulled image to `clawmates/<svc>:latest` as a bridge so the previous compose file (which used bare names) would find it — that step is now unnecessary and just added a small window where the un-prefixed tag could diverge from the registry. Drift check now compares against the registry-prefixed tag directly. `docker compose` v2 preferred with `docker-compose` v1 fallback stays. |
||
|
Omar Sobh
|
3f82d9efe4 |
deploy(gw-04): running-container drift check + docker-compose v1 fallback
Two fixes needed to make the timer actually roll correctly on gw-04: 1. Drift check compares the running container's image ID against the local `clawmates/<svc>:latest` tag, not just pre/post-pull digests. The pre/post check only catches new pulls — if a previous roll failed between the retag and `docker compose up` (e.g. compose CLI failed), the tag was updated but the container wasn't, and the next tick saw no drift and silently left the stale container running. The running-vs-tag check catches that case on the next tick. 2. Prefer `docker compose` (v2 plugin) but fall back to legacy `docker-compose` (v1). GW-04 ships v1 only right now, and calling `docker compose up -d` failed with "unknown shorthand flag: 'd'" because docker had no `compose` subcommand at all. The fallback keeps the script portable when the stack moves to a host with v2. |
||
|
Omar Sobh
|
d0d8e7fd40 |
Merge feat/a2a-rooms-delegation-ingress into main
Brings a2a rooms, delegation, and A2A ingress work back into main. Prod's
DB has migrations 0026 (group_rooms) and 0027 (a2a) applied from an
earlier hand-tagged fleet21 build cut from this branch, but main never got
them — so the CI-built server image from main refused to start against
prod's DB with "migration 26 was previously applied but is missing".
Landing the branch closes that gap: main + prod DB now share the same
migration state, so images built from main can safely roll onto gw-04.
Included:
- 0026_group_rooms.sql / 0027_a2a.sql — align main with prod's schema
- cm-api routes/a2a.rs + mcp_door.rs updates — A2A ingress and MCP door
- cm-runtime tools/delegate.rs + tools/chat.rs — delegation + N-way rooms
- frontend TeamObserver + FleetPanels + AgentObserver updates
- taxonomy.ts — delegation + A2A signals in the live world feed
Not included (still WIP on the local checkout):
- 0028_backfill_on_delete.sql / 0029_hot_query_indexes.sql
- broker pool + team-run quota changes
- Dashboard.tsx UI rename (Large World → Visualizations)
- Node write.send timeout (aaab663) — separate concern
The SSE resume fix (
|
||
|
Omar Sobh
|
4480c750bf |
build+deploy: reproducible pipeline via Gitea Actions + gw-04 image-watcher
Adds a `publish` job to ci.yml that fires only on green pushes to main. It builds broker, server, and frontend from images/*.Dockerfile, tags each with :main-<sha> + :latest, and pushes to the fleet registry at redclaw-web-01:5000 (via its Tailscale IP 100.94.185.103, which the daemons already trust in insecure-registries). Adds a small systemd oneshot + 1-minute timer for gw-04 that polls :latest of each service, pulls on drift, retags to the un-prefixed name the current compose file uses, and rolls only the changed services. The retag keeps /root/clawmates/docker-compose.yml unchanged for now — a follow-up can migrate the compose file to registry-prefixed names once we're confident. End-to-end: push to main -> tests -> images pushed -> gw-04 timer pulls within ~1 min -> prod updated. Rollback = docker tag <old-sha> :latest and `docker compose up -d`. |
||
|
Omar SobhandClaude Opus 4.8
|
cbfa0ff24f |
feat: agent-to-agent platform on ZeroClaw 0.8.2 — rooms, delegation, A2A ingress
Builds on the v0.8.2 runtime. Four workstreams, all behind the §15 MCP door:
- Group rooms (Phase 1): migration 0026; N-way threads repo with a DM/room
count-guard; chat.send {room} + room.create/invite/leave tools; RoomMessage
-> room.message SSE; /api/claw-chat/rooms* APIs; Observer room badge.
- Per-claw door identity: door caller_agent resolves the X-ZeroClaw-Agent
header (set by the fork) to the specific claw, falling back to roster[0].
- Gated delegation bridge (Phase 3): clawmates__delegate door tool drives a
sibling via the existing /ws/chat ZeroClawDriveExecutor (not A2A); self-deny,
per-workspace hourly budget, audit trail, untrusted-banner result. Native
in-daemon delegation stays off (it would bypass the door).
- A2A tenant ingress (Phase 2): migration 0027 (workspace_a2a + a2a_tokens);
runtime_provision enable_a2a_server/publish_claw; routes/a2a.rs tenant-aware
proxy (per-workspace tokens, injected internal bearer, daemon stays internal,
cards URL-rewritten to the cm-api edge); a2a.invoked taxonomy.
Tests: cm-db room repos, cm-runtime chat tools, door units. sqlx cache updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
ce7a5d9aed |
Remove orphaned k8s artifacts (Docker-only now)
CI no longer references k8s; delete the dead k8s surface: - deploy/helm/ (the chart), ci/check-helm.sh, scripts/netpol-cluster.sh - cm-sandbox: the feature-gated K8sDriver (src/k8s.rs) + k8s_security test + the `k8s`/`k8s-tests` features + the optional kube/k8s-openapi/rustls deps (Cargo.lock drops the kube-rs tree). Nothing outside cm-sandbox referenced it. Docker (bollard) DockerDriver is the sole sandbox driver. cm-sandbox + cm-runtime compile, fmt + clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
148705769f |
Reaping: sandbox orphan reaper + DB expiry/retention sweeper + volume-init retry
Closes the cleanup gaps found in review (latent today; bites under load/crashes).
Sandbox containers (cm-sandbox / cm-runtime):
- label every sandbox `clawmates.sandbox={agent|browser}` at create
- SandboxDriver::list_managed(kind) (Docker label filter + K8s label selector)
- SandboxManager::reconcile_orphans(ttl) + spawn_reaper: removes engine
containers no live handle owns (ZERO = all)
- boot reconciliation (every pre-existing sandbox is an orphan from a dead
process) + periodic reaper (5m interval / 10m TTL)
- SIGTERM graceful drain: serve().with_graceful_shutdown → shutdown() both
managers so a redeploy can't leak; destroy errors now logged not swallowed
DB expiry/retention (cm-db cleanup.rs + cm-api cleanup_sweeper, hourly):
- expire auth_sessions + oauth_states past expires_at (security)
- prune sent outbox(7d), run_events(14d), routine_runs(30d), terminal
topology_runs(90d), consumed execution_grants(7d)
Compose: volume-init restart "no" → on-failure:5 (retry instead of wedging boot).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
e8486ee57c |
Door delivery: outbox drainer + SMTP transport (inert until configured)
The §15 door's email_send queues to `outbox` but nothing delivered it. Add a real transport: cm-db outbox repo (list_queued/mark_sent/mark_failed) + a cm-runtime drainer — an EmailSender trait (testable), a lettre STARTTLS LettreSender, drain_once (queued -> sent/failed), and spawn_drainer wired into the server beside the scheduler/sweeper/topology-worker. Config-gated: inert (logs "outbox delivery DISABLED") until CLAWMATES_SMTP_* is set, so it ships safely before credentials exist. The agent never holds the SMTP credential — it only writes to outbox through the gated door; the server owns the transport. NOTE: live delivery is still credential-blocked — Migadu's API can't send (SMTP-only) and the admin token is invalid; no SMTP creds exist. The transport is built + tested (drain_marks_sent_and_failed via a mock sender); set CLAWMATES_SMTP_* to go live with zero further code. clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
a14be693db |
paper §7.3: per-role model leaderboard + GLM-5.2 flagship tier
Single-role drafter on a fixed concision-weighted brief, one turn per backend, judge-ranked: GLM-4.7 > Claude > Kimi > GLM-5.2. Notable inversion — the flagship GLM-5.2 ranked last by over-writing the 25-word brief, underscoring that the best model is role/task/rubric-dependent. Adds the worker_glm5 (glm-5.2 Opus-class) tier to the example config. Gemini excluded this run (Google June-19 API-key-restriction 403; infra, not model). Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
ae517a18c1 |
Runtime: add Gemini + Groq workers; document Phase C heterogeneous topologies
Gemini via the built-in `gemini` API family (gemini-2.5-flash default, gemini-3-pro-preview flagship) and Groq via `groq` — both wired as topology worker aliases (worker_gemini, worker_groq), no image rebuild. Documents the Phase C recipe: map semantic roles to per-model aliases via the server ZEROCLAW_AGENT_MAP, then POST /api/topologies/run. Five subscription/API backends now selectable per role: Claude, Kimi, GLM, Gemini, Groq. Proven live on gw-04 — a 3-stage pipeline ran researcher=GLM-4.7 -> analyst=Kimi-K2 -> writer=Gemini-2.5-flash end-to-end (RunRecord persisted). Quota-care notes added (5h/weekly caps, low concurrency -> prefer sequential pipelines, short prompts). Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
ca91d18860 |
Runtime: GLM (Zhipu) worker via Claude Code over z.ai — proven live
GLM Coding Plan has no CLI of its own and ToS-throttles raw SDK/API access
("1113 Insufficient Balance"); it is served only inside officially supported
tools, and Claude Code is the primary one. So GLM rides the existing
claude_cli provider with a z.ai env override (ANTHROPIC_BASE_URL=
api.z.ai/api/anthropic + ANTHROPIC_AUTH_TOKEN) — no new provider, no image
rebuild. Fresh HOME + CLAUDE_CODE_OAUTH_TOKEN="" so the container's cached
Claude subscription doesn't override the routing; key via $NAME indirection.
Deployed & verified on gw-04: worker_glm answers "I am GLM-4.7, built by
Zhipu AI" over /ws/chat. Three subscription models now co-reside on one
runtime — Claude (coordinator), Kimi (worker_kimi), GLM (worker_glm).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
2e889df35e |
Runtime: native Kimi Code CLI alongside Claude (kimi_cli provider, live)
Runtime image now installs Node 22 (nodesource) + both agent CLIs — @anthropic-ai/claude-code and @moonshot-ai/kimi-code — so the zeroclaw daemon can spawn `kimi -p` for the new kimi_cli provider on a Kimi membership, parallel to `claude -p` (claude_cli). Kimi needs Node >=22.19; the image was on Node 18. Documents the kimi_cli worker recipe in agent.config.example.toml (the coding endpoint is User-Agent gated → drive Kimi through its own CLI, not a raw OpenAI-compat client) and keeps GLM via the registry family. Deployed & verified on gw-04: coordinator (claude_cli) answers "Anthropic built me", worker_kimi (kimi_cli) answers "I am Kimi, built by Moonshot AI" — heterogeneous Claude + Kimi on one runtime, over /ws/chat. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
c1b1b5a7cb |
config: Kimi Code (subscription) as judge + worker provider
Phase A wiring (config-only): server registry kimi provider now targets the Kimi Code coding endpoint (api.kimi.com/coding/v1, model kimi-for-coding, subscription-billed via a Kimi Code Console key). Runtime example documents a custom.kimi worker provider (OpenAI-compatible coding endpoint) + GLM, selectable per topology role via ZEROCLAW_AGENT_MAP. Activates on KIMI_API_KEY; no rebuild. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
cbbe197a6d |
runtime config: document GLM/Kimi worker providers
ZeroClaw already supports glm + moonshot (Kimi) as model providers; document the pattern to run a topology worker on them (env model+key, agent on the alias, role->alias via ZEROCLAW_AGENT_MAP). Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
ea5687565a |
1B(a): document claude_cli.door — subscription agent acts through the §15 door
Adds the claude_cli.door provider pattern (mcp_config + allowed/disallowed tools) so a subscription claude_cli agent reaches the Clawmates door via Claude Code's own MCP client, with native tools locked out (door = only actuator). Validated live on gw-04: subdoor agent (claude_cli, subscription) → door → outbox + agent|door.executed audit, cost 0.0; shell command denied. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
ec5d3b3149 |
1B: autonomous door config — level=full + claude_cli text-only note
Set the tool-free risk profile to level="full" so ZeroClaw raises no in-channel approval (the human is removed; the Clawmates door auto-governs). Document that claude_cli is text-only (claude -p doesn't surface tool calls), so agents that ACT through the door need a tool-capable provider (groq/anthropic) — with a commented actor-agent example. Validated live: a groq-brained actor agent at full autonomy emitted clawmates__email_send with no approval prompt; the door executed it (outbox + agent|door.executed audit). No human in the loop. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
87f612016e |
1B: Clawmates §15 MCP door — autonomous gated egress for tool-free agents
Tool-free ZeroClaw agents get one actuator: an MCP server (POST /mcp, JSON-RPC
2.0, protocol 2024-11-05) that fronts the existing §15 machinery. The human
approver is replaced by an automated policy (default allow-all → agents are
autonomous; CLAWMATES_DOOR_POLICY=deny is a kill switch), but the governed parts
stay: every action is journaled to the append-only audit log, actions execute
through the runtime's gated-tool path, and broker credential-custody is wired in
for v2 tools. Synchronous execution returns the real result inline.
- crates/cm-api/src/mcp_door.rs: initialize/tools.list/tools.call handler; auth
(bearer -> workspace), classify effects, policy auto-decide, execute, audit.
v1 exposes email_send (-> outbox); slack/pay (broker+grant chain) is next.
- crates/cm-runtime: Runtime::{tool_descriptor_json, tool_gate_category,
execute_door_tool} — door-facing entry that builds ToolContext and consumes a
grant for runtime-executed gated tools.
- deploy/clawmates-runtime: agents now carry mcp_bundles=["clawmates_door"];
[[mcp.servers]] points at the door (bearer injected at deploy, not committed).
Validated live on gw-04: tools/call email_send -> isError:false + outbox row +
"agent|door.executed" audit, no human. 4 door unit tests + clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
fbe783e278 |
Runtime: drive topology turns on the Claude subscription via claude_cli
Switch the per-tenant runtime to ZeroClaw's new native `claude_cli` model provider (added in the zeroclaw repo): every agent turn spawns `claude -p` headless, authed by CLAUDE_CODE_OAUTH_TOKEN — i.e. the Claude subscription, no per-minute API TPM ceiling (the groq free tier 429'd multi-turn pipelines). - Dockerfile: install the Claude Code CLI (@anthropic-ai/claude-code) into the runtime image. - agent.config.example.toml: role-cast on model_provider = "claude_cli.default". NOTE: set the model via env (ZEROCLAW_providers__models__claude_cli__default__model) not the TOML sub-table, else resolve_default_model() reads empty -> NEEDS_ONBOARDING. - .gitignore: token.key (local Claude OAuth token, never committed). Validated live on gw-04: a 2-node researcher->writer pipeline ran via the deployed POST /api/topologies/run -> claude_cli -> real claude turns; 2-step RunRecord, cost_usd 0.0 (subscription), no rate limiting. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
6d77a0acc1 |
Topology P-zc 1A: ZeroClawDriveExecutor — real role-agents over /ws/chat
cm-orchestrator owns the topology graph; each turn now drives a real ZeroClaw
role-agent in a container via the proven gateway drive recipe, instead of a
tool-free cm-llm call.
- crates/cm-api/src/topology_exec.rs: ZeroClawDriveExecutor impl TurnExecutor —
pair (POST /pair + X-Pairing-Code, token cached) -> ws /ws/chat?agent=<alias>
-> send {type:message,content} -> drain chunk/done/approval_request/error.
approval_request is recorded as a BLOCKED GatedAction, never auto-approved (§15).
Role->alias via ZEROCLAW_AGENT_MAP, fallback ZEROCLAW_DEFAULT_AGENT (scout).
- POST /api/topologies/run {task,graph} -> execute() -> RunRecord, persisted
best-effort to the existing topology_runs table (no migration). compare stays
tool-free. from_env() is read in-handler so cm-api still boots unset.
- deploy/clawmates-runtime: example config now declares a tool-free multi-agent
role-cast; README documents the ZEROCLAW_* knobs + run endpoint.
tokio-tungstenite 0.26 (already in lock) + dev axum `ws` for the hermetic test.
3 lib tests green, clippy clean, SQLX_OFFLINE build clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
bd982943b8 |
chore: commit ZeroClaw per-tenant runtime spike + architecture doc
Saves earlier-phase artifacts that were sitting untracked: - docs/agent-engine-architecture.md — per-tenant containerized ZeroClaw runtime decision doc (clawmates = §15 control plane; zeroclaw = per-tenant runtime). - deploy/clawmates-runtime/ — slim runtime Dockerfile, dev compose, example agent config, README (the proven Phase-1 drive recipe). - tools/runtime-spike/drive.mjs — Node WS drive client for the spike. No secrets (only env-var names / commented placeholders). Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
447f7039d8 |
Compose: production README, env knobs, first-owner bootstrap — deployed & live
Made the Docker Compose route turn-key for a real self-host, then stood the whole stack up and drove a live chat through it. - First-owner bootstrap (cm-auth::bootstrap_owner): a fresh local-auth install has no users and no signup route, so the initial Owner + workspace are provisioned ONCE from CLAWMATES_BOOTSTRAP_* env on first boot — idempotent, never clobbers an existing install (keys on 'any workspace exists'). Two real-Postgres tests (creates + signs in; second call is a no-op). Wired into server boot, guarded on a non-empty password - deploy/compose/README.md: full production bring-up — services, the security topology, every config knob, Anthropic vs local-LLM, the broker-key backup, ops, and TLS/SSE proxy notes - .env.example fleshed out (bootstrap, LLM, auth mode, OTLP); compose uses optional env_file so only the knobs you set are injected (unset options never override clawmates.toml with empty strings) - volume-init one-shot chowns the broker's named volumes so the non-root scratch broker can write its socket + generated master key Deployed locally and verified end to end: all 5 containers healthy, broker generated its key, server bootstrapped owner@…, login + /api/user/me work, and a real message streamed a live Anthropic response through the gateway. Captured screenshots of login, workspace home, chat, and the Computer panel. 166 Rust tests (+2 bootstrap). Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
add4f79fed |
Rebrand: TeamClaw -> Clawmates (clawmates.work)
Full-depth rename per the approved plan; the 'claw' product vocabulary (claws, /claws routes, clawId, Claw Chat) stays — it is now the brand. - Display brand: Clawmates (manifest, titles, hero, login/rail logo 'clawmates'); default host app.clawmates.work; registry ghcr.io/clawmates - Crates tc-* -> cm-* (16 crates + all imports); binaries clawmates-server/broker/bundler; images clawmates/*; env prefix CLAWMATES_* (+ CM_TEST_DATABASE_URL / CM_LIVE_LLM); config clawmates.toml; helm chart deploy/helm/clawmates with clawmates-* resources; db names clawmates*; sockets /run/clawmates; cookie cm_session; kind cluster clawmates-test; seccomp node profile clawmates-agent-profile.json - All 9 Playwright brand assertions updated in lockstep; historical spec document left untouched as the only remaining 'TeamClaw' - Local env migrated: dev pg clawmates-dev-pg/clawmates_dev, shared test server clawmates-test-pg, kind cluster recreated with image + profile, compose images rebuilt under clawmates/* Verified end to end: 161 Rust + 68 frontend tests, 29 Playwright journeys, 4 live kind tests, helm/install/LOC/placeholder gates, and the clean-room install rehearsal serving the clawmates login page from a signed bundle of the rebuilt images. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
8046853feb |
Post-1.0: OTLP tracing, broker image + compose service, install rehearsal
- tc-telemetry: fmt subscriber always; with [telemetry] otlp_endpoint set, spans batch-export over OTLP/HTTP. Tested against a REAL OTLP receiver decoding the actual protobuf (official proto types): the emitted span and service.name arrive on the wire. No endpoint = no export = no network (air-gap stance). tower-http TraceLayer gives every API request a span - The broker finally has its own image (images/broker.Dockerfile, 9.5MB from scratch) — the Helm chart referenced one that never existed — and the compose deployment now RUNS the broker, sharing a socket volume with the server (the unix-socket equivalent of the K8s sidecar). Compose secret flows were silently dead before this - server.Dockerfile fixes surfaced by the rehearsal: the workspace build needs tools/ (bundler joined the workspace) and images/seccomp/ (include_str! profile) in the build context - scripts/rehearse-install.sh (plan: clean-VM rehearsal): assembles a REAL signed bundle from the built images (server/frontend/broker/ postgres/socket-proxy), runs the customer path — offline verify, docker load, compose up — and asserts /healthz plus the served login page before teardown. Passing locally; wired as a release.yml step, which also builds/ships the broker + socket-proxy images now 161 Rust tests + 29 journeys; clean-room rehearsal green. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
05e9612688 |
Post-1.0: Localhost seccomp on K8s, warm sandbox pool, server HPA
- K8sDriver.with_localhost_seccomp(profile): sandbox pods run under the STRICT allowlist instead of the runtime default. Proven live on kind: the harness installs the profile onto the node, a pod runs ordinary work as uid 10001, and unshare is kernel-denied inside the pod — the same probe the Docker suite uses, now passing on both targets - Helm: sandbox.seccomp=localhost renders a DaemonSet that installs the chart-shipped profile into /var/lib/kubelet/seccomp on every node (ConfigMap + hostPath); ci/check-helm.sh enforces the chart copy stays byte-identical to images/seccomp/agent-profile.json and asserts the hardened render (DaemonSet + profile + HPA) - server HPA (autoscaling/v2, CPU target) behind server.autoscaling.enabled - SandboxManager.warm(n): a background warmer keeps n pre-provisioned sandboxes ready so an agent's first exec skips container startup; unhealthy pool entries are discarded, reuse never drains the pool, shutdown destroys assigned AND pooled. [sandbox] warm_pool config (default 0). Real-Docker test: prefill -> assign -> refill -> reuse -> clean shutdown 160 Rust tests + 4 live kind tests. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
a26939d7bc |
Plan hardening: socket-proxy allowlist, gateway load test, offline verify
- The compose server NEVER sees the raw Docker socket (plan risk #5): tecnativa/docker-socket-proxy on an isolated engine_net with exactly CONTAINERS/POST/EXEC/DELETE/VERSION allowed; server reaches it via DOCKER_HOST. DockerDriver honors DOCKER_HOST (connect_to). Proven by a REAL proxy test: full sandbox lifecycle works through the allowlist while /networks, /secrets, and /images all 403 — the blast-radius cap if the server is ever owned. (This also fixes compose deployments, where sandbox provisioning previously had no engine access at all.) - Gateway load test (plan P6): 40 concurrent SSE streams against one server — every run completes with the full §13 event vocabulary, every journal strictly monotonic, every resumeFrom=0 replay byte-equal to its live stream - release.yml: SBOMs (syft, spdx-json) for all four images shipped INSIDE the signed bundle; final verification now runs in a --network none container — proving the customer's verify path needs no internet, not just claiming it 159 Rust tests. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
ceca21ca79 |
Clerk frontend integration: one image, runtime-switched identity
- src/lib/auth/bearer.ts is the single identity dispatch for both server-side token consumers (RSC apiFetch and the /api proxy route): local -> httpOnly tc_session cookie; clerk -> Clerk getToken() session JWT. The Clerk SDK is imported lazily, so the air-gapped/local path never loads it - Runtime env (AUTH_MODE / CLERK_PUBLISHABLE_KEY / CLERK_SECRET_KEY), deliberately NOT build-time NEXT_PUBLIC_*: the same standalone image serves both deployment targets - Conditional <ClerkProvider> in the root layout (publishableKey passed at render from runtime env); /login renders Clerk's <SignIn /> in clerk mode and the local form otherwise; proxy.ts middleware delegates to clerkMiddleware() only when active - Helm: frontend deployment injects the Clerk keys from a Secret when auth.mode=clerk - mode.ts unit-tested (default local, exact-match clerk, loud failure without the publishable key); the local path stays proven by all 29 journeys; the Clerk branch is thin delegation to the SDK, exercised in deployment smoke per docs/clerk.md 157 Rust + 68 frontend tests + 29 Playwright journeys. Co-Authored-By: Claude Fable 5 <[email protected]> |
||
|
Omar SobhandClaude Fable 5
|
cbc8d35a2e |
Clerk authentication: hosted-identity session JWTs as a first-class mode
- tc-auth JwtVerifier: OIDC discovery -> JWKS, RS256 with the issuer pinned, 5s leeway (the crate's default 60s would double the life of Clerk's 60s session tokens), key cache with one refresh on unknown kid (Clerk rotates). Serves auth.mode = clerk AND generic oidc — a Clerk instance IS an OIDC issuer, so one verifier covers both - AuthService.authenticate dispatches: JWT-shaped bearers take the hosted-identity path, everything else stays a local opaque session. External users JIT-provision keyed by the stable sub claim (users.auth_subject, unique partial index in migration 0007); an existing local account with the same email is LINKED, not duplicated; role tracks the issuer claim every request (org:admin -> Owner) - Config auth.mode = "clerk" (requires issuer_url; validated), server pins the issuer at boot, Helm values/configmap accept mode=clerk - Tests with REAL crypto, no mocks: fresh RSA keypairs, a live local issuer publishing real discovery + JWKS docs, Clerk-shaped tokens — JIT + role mapping, repeat-subject no-dup, expired refused (leeway regression), wrong-key forgery refused, foreign issuer refused, and the full router round trip with Authorization: Bearer <session JWT> - docs/clerk.md: dashboard session-token customization (email + org role claims), config, @clerk/nextjs getToken() wiring, what CI proves 157 Rust + 63 frontend tests + 29 journeys. Air-gapped installs keep local auth — Clerk is a cloud-only alternative, not a replacement. Co-Authored-By: Claude Fable 5 <[email protected]> |