4f07430e929847c8980f08eddff32cd96b5a65f1
105
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar Sobh
|
956be2cf4f |
wizard: in-place auto-provision team from topic (LLM-derived, sonnet-5)
Step 5 of ResearchWizard was 'assign agents from workspace roster'.
When the roster was empty, the wizard body was hard-swapped for
NoAgentsGate — you couldn't reach step 5 at all.
Now step 5 shows a 'Team' panel:
- Big cyan card: 'Auto-provision team from this topic'. One click
runs an LLM plan pass, gets 3-5 role slots + system prompts back,
materializes claws via the existing build_team pipeline, stamps
runtime posture, returns a shape that drops straight into the
submit body's agents[]. Card flips green with the derived roster.
- Below that: the classic roster picker, but only when the workspace
actually has ≥1 claw AND auto-provision hasn't landed. Otherwise
hidden — no dead empty-state affordance.
Every gate that required agents.length > 0 to render the wizard body
or the footer is gone. canNext gains a step-5 clause: allow Next when
EITHER auto-team is ready OR the user handpicked from a non-empty
roster.
Backend
- POST /api/teams/auto-provision — accepts {title, description,
outcome_kind, topology_kind?, model?, risk_profile?, mcp_bundles?}.
Derives topology from outcome_kind (integrations → pipeline; else
hub_spoke). LLM plan pass yields a JSON roster of 3-5 roles
(role_slot, name, system_prompt). Materializes team + claws via
build_team, stamps risk_profile (default research_web_readonly) +
mcp_bundles (default [clawmates_door, gitea_forge]). Response
carries team_id + agents[] in the shape /api/research already
expects.
- Every provisioned claw runs on claude-sonnet-5 by default;
overridable via the model field.
Follow-ups (not in this slice):
- Same picker in LoopsWizard (slice C — parallel change, same API).
- Post-create 'Team' section on ResearchCanvas / LoopsCanvas so
users can rebind after the fact (slice D).
- Full Teams tier UI + Agents-page deprecation (slice E).
|
||
|
Omar Sobh
|
6066e93889 |
teams: per-team runtime posture (risk_profile + mcp_bundles) + FK from loops/topics
Foundation slice for letting a coding loop bring its own team instead
of reusing the paired research topic's team. Turns out the teams
table already exists (0010_teams.sql) with full CRUD — this scales
back to the minimal missing bits:
Schema (0045_teams.sql)
- ALTER TABLE teams ADD risk_profile TEXT (NULL = template default)
- ALTER TABLE teams ADD mcp_bundles JSONB DEFAULT '[]'
- ALTER TABLE loops ADD team_id UUID REFERENCES teams ON DELETE SET NULL
- ALTER TABLE research_topics ADD team_id UUID REFERENCES teams
- Two partial indexes (team_id NOT NULL) for the future cascade queries
cm-db (dynamic sqlx::query so the existing get_team's compile-time
cache doesn't need regenerating):
- TeamRuntimeConfig struct
- get_team_runtime_config / set_team_runtime_config
- team_for_loop / team_for_research_topic (resolvers)
- set_team_for_loop / set_team_for_research_topic (binders)
cm-api
- GET /api/teams/{id} now surfaces risk_profile + mcp_bundles
- PATCH /api/teams/{id}/runtime-config sets them
Not touched (comes in follow-up slices):
- Wizard picker exposing 'reuse research team' vs 'fresh coding team'
- Runtime container spawn keyed on team_id
- Migration of existing paired coding loops onto their own team
|
||
|
Omar Sobh
|
ae113dd319 |
live-run-logs: add Container tab that tails filtered daemon logs
Steps summaries only fire AFTER each topology step completes — so a
stalled first turn was completely dark. Add a second tab that
streams the team runtime container's daemon log live via a new SSE
endpoint.
Backend
- GET /api/topology-runs/:id/container-log — workspace-scoped SSE
around bollard's docker.logs(follow=true, tail=200). Buffers on
newline so partial mux chunks don't truncate a log line.
- compact_container_log: parse a zeroclaw daemon line
('[actor] ... zc_action=X zc_outcome=Y ... msg') into
'[actor] action (outcome) · msg'. Framing-only continuations are
dropped; non-zc lines (bash echoes, backtraces) pass through as-is
so nothing interesting is lost. ANSI escapes stripped.
- Everything funnels through one async_stream! so early exits
(workspace check / docker connect / no bound topic) yield an
'error' event and return without breaking Sse::new's single stream
type.
Frontend
- LiveRunLogs gets a sub-tabs strip: Steps · Container.
- New useContainerLog(runId, active) hook — gated by tab so we don't
hold two open SSE streams when the operator isn't looking.
- Same terminal widget renders each container line with a level
color (info/done grey, line default, error red). Sub-tab pill
shows count + status live.
|
||
 osobh
|
1d69866bcf | research canvas: collapsible sidebar + live topology-run logs (#6) | ||
|
osobh
|
149ad992bb | wizard: materialize picked repo across clawstor fleet at step-2-next (#5) | ||
|
Omar Sobh
|
212e68f6b1 |
research/probe: end-to-end smoke test endpoint (skip topics/loops/spawn)
New: POST /api/research/probe — one-shot pipeline probe against the
workspace's shared ZeroClaw gateway. Drives a trivial 'reply OK'
turn via ZeroClawDriveExecutor::drive and reports per-step timings.
Purpose: stop guessing what's broken in the wizard flow by exercising
JUST the executor→daemon→claude→response path. If probe succeeds
within a few seconds, we know:
- ZEROCLAW_TOKEN + gateway URL config is correct
- Daemon can reach and authenticate against claude
- The full ws round-trip works
…and every other failure we've been chasing (turn timed out, LLM
request failed, ws connect DNS error, etc.) is spawn-config or
prompt-size specific.
Request body (both optional):
{ "prompt": "Reply OK", "agent": "coordinator" }
Response:
{
"verdict": "ok" | "fail",
"total_duration_ms": …,
"prompt_len": …,
"response_preview": "OK",
"steps": [
{ name: "build_executor", duration_ms: …, status: "ok" },
{ name: "drive_turn", duration_ms: …, status: "ok", detail: "tokens=N, output_len=M" }
]
}
Wire-up:
- new routes/probe.rs
- pub mod probe in routes/mod.rs
- POST /api/research/probe registered in lib.rs router
- ZeroClawDriveExecutor::drive promoted from private to pub so the
probe handler can call it (behavior unchanged, other callers were
all inside the same struct).
Usage from anywhere (curl, browser dev tools, etc.):
curl -X POST https://clawmates.work/api/research/probe \
-H "Content-Type: application/json" \
-H "Cookie: <session cookie>" \
-d '{}'
Follow-up: a small frontend button (e.g. bottom of ResearchList) that
POSTs this and renders the response inline, so users don't need to
curl. Skipping in this commit to ship the useful part first.
|
||
|
Omar Sobh
|
a34be33261 |
loops: N/M INTs progress pill on source-bound loop cards
Users couldn't see how a research-bound loop was progressing through
its integration plan — the consumed_int_ids state existed in the DB
but nothing surfaced it. Now each loop card in the sidebar shows a
cyan pill "3/8 INTs" + source topic title + a thin progress bar, when
the loop is bound to a research topic.
Backend:
- GET /api/loops/progress — bulk read for every loop with a source
topic. Returns {loop_id, source_topic_id, source_topic_title,
source_outcome_version, consumed_count, total_int_count,
current_int_index} per loop. Standalone loops are omitted.
- count_int_ids parses unique INT-<number> ids out of the source
outcome's markdown — same permissive matcher as the completion
hook, so what the pill counts matches what the loop can advance.
- Memoized by topic_id inside the endpoint so N loops sharing 1
source topic only fetch the outcome once.
Frontend:
- listLoopProgress helper + LoopProgress type in the loops API.
- LoopsList fetches loops + progress in parallel on mount.
- Each card looks up progress by loop_id and, when found, renders
under the schedule line: pill "3/8 INTs · Topic title" with a
3px cyan progress bar. Title hover shows artifact version.
Follow-ups:
- Refresh button on the card — re-snapshot artifact into
task_template (cosmetic; the enqueue path already reads latest).
- Reorder rationale extraction — parse "REORDER: <text>" out of run
output, index as a per-loop event log for a mini-timeline UI.
|
||
|
Omar Sobh
|
3bcd18bf50 |
research: split pipeline diagnostics into research_pipeline.rs
Gates step failed on run#211 because research.rs grew to 1313 lines (over the 1250 budget) after adding the pipeline_state handler in the previous commit. Move the diagnostic into its own module — pure extraction, no behavior change. research.rs: 1313 → 1126 lines research_pipeline.rs: new, 187 lines lib.rs: route now points at routes::research_pipeline::pipeline_state |
||
|
Omar Sobh
|
cca6e2be20 |
research: pipeline diagnostics + refuse publish without outcome
Two fixes surfaced by the first prod run of the pipeline:
1) Silent skip-to-published bug (R1 gap):
approve_publish transitioned reviewing → publishing → published
without checking that an outcome existed. Result: pipeline could
fail silently (LLM auth error, network, etc), no outcome would be
written, but state advanced to 'published' and the download endpoint
returned 404 with no user-visible error. Now refuses with 409
Conflict when no outcome exists so the frontend can surface WHY.
2) No end-to-end visibility:
Users had no way to see where a run failed until they clicked
Download and got nothing. Adds
GET /api/research/:id/pipeline-state — a read-only per-stage report
walking:
- staffing (agents assigned)
- repo (bound + cloned)
- container (per-topic team runtime spawned)
- runs (count + failed count + latest error text)
- outcomes (count — the artifact rows get_artifact reads)
- approval (pending flag)
Each stage returns ok / warn / fail / skip plus optional detail text
so the failure reason surfaces at the diagnostic level.
Frontend:
ResearchCanvas shows a compact PIPELINE strip below the topic title,
green/amber/red dots per stage, click to expand a full checklist
with per-stage detail (including the LLM error from the last run
attempt). Polls every 6s while the topic is processing/publishing.
Follow-up:
- Root cause of the specific failure just observed: Claude CLI in
the clawmates-runtime container isn't authenticated. Deploy-side
config sweep (CLAUDE_CODE_OAUTH_TOKEN or ANTHROPIC_API_KEY into
the runtime image env), not a code fix.
- Structured event stream on top of run_events for real per-step
replay in the diagnostic panel.
|
||
|
Omar Sobh
|
f9e8d8d779 |
research: publishing → published + artifact download (R1)
Before this commit, approve_publish left topics stuck in 'publishing' forever — the sidebar 'published' bucket was always empty and nothing surfaced the artifact. Two-part fix: State machine — approve_publish now transitions reviewing → publishing → published in one API call. Real async packaging isn't a thing yet because the artifact IS the markdown already written to research_outcomes when the last run completed (topology_worker). The intermediate 'publishing' state is preserved (schema-level trigger stamps published_at on landing there) so we keep the option to detour through it later for pdf render / mirror-to-store / etc. Download endpoint — GET /api/research/:id/artifact returns the latest outcome as text/markdown with Content-Disposition: attachment. Filename sanitizes the topic title to ascii-alnum + dash and appends the outcome version so accumulated revision drafts (post R2) don't clobber. Workspace ownership check via research_topics::get; 404 if no outcome yet (reviewers browsing before a run completes). Frontend — ResearchList shows a green Download icon-button next to Delete when status === 'published'. Clicks trigger a plain anchor download of the .md — no JS blob dance needed since the response is already an attachment. Follow-up queued: pdf render (server-side or client-side of the md), mirror-to-store (S3-ish or Obsidian vault) as an async step during the publishing→published detour. |
||
|
Omar Sobh
|
acd2a0f287 |
structure polish: post-reify nav + ensure-chain + TeamWizard auto-parent
Two small quality-of-life fixes on top of the reify commit:
Post-reify navigation
OrphanMigrationDialog already returned team_id in its result;
Dashboard now pushes /?team=<team_id> before router.refresh() so
the user lands on the freshly-materialized team and sees exactly
where their agents just moved. Previously they had to hunt for it
in the newly-rebuilt sidebar.
Wizard auto-materialize (POST /api/structure/ensure-chain)
cm-db: ensure_chain(pool, ws, fallback_org, fallback_company) —
fast path returns coordinates of the first org+company already
bound in this workspace (workspace's oldest org, oldest company
under it). Slow path inserts a new org+company with the
fallback names ("My Workspace" / "General") + binds them via
org_companies. Returns { org_id, company_id, created }. Small
txn — leaves the workspace consistent whether it was already
wired or not.
cm-api: POST /api/structure/ensure-chain accepts optional
fallback_org_name and fallback_company_name in the body (trimmed,
else default). Returns the ids.
CreateTeamRequest gains an optional attach_to_company_id. When
set, after build_team() completes, we look up the company
(workspace ownership check enforced by companies::get), count
its existing teams for a stable n_i node id, and insert a
company_teams binding — so the team lands under the parent
atomically instead of a follow-up round-trip.
TeamWizard now calls ensure-chain before POST /api/teams and
passes the returned company_id in attach_to_company_id. Both
calls are best-effort — if ensure-chain fails (network etc.)
we still try to create the team, and the migration dialog stays
available as the fallback UX. Wizard flow now: fresh workspace's
first team is fully wired from the moment it appears in the
tree — no synthetic "My Workspace" scaffolding ever gets
rendered around it.
The Team/Company create paths not touched here (create_team_from_claws,
company create, org create, MasterPlannerModal scaffold) still
work as before — they just won't auto-parent yet. Later commits
can wire them the same way.
|
||
|
Omar Sobh
|
8b789beec0 |
structure: reify-orphans endpoint + "give these a home" dialog
Turns the four synthetic tree containers into a real migration path.
Clicking any of them ("My Workspace", "Teams", "Direct",
"Ungrouped") opens a dialog that creates a real
org → company → team chain and re-parents every orphan into it, all
in one DB transaction.
cm-db (new module structure_reify)
- orphan_agents / orphan_teams / orphan_companies: workspace-scoped
SELECTs of entities without a parent binding in team_members /
company_teams / org_companies. Used both by the dialog's counter
and internally by the migration.
- count_orphans: cheap combined-count via three subqueries in a
single SELECT so the dialog only round-trips once for the header.
- reify_orphans(pool, ws, org_name, company_name, team_name):
1. begins a tx
2. inserts a new org + company + team (all `flat`, empty graphs
— user can shape them later via the existing PATCH endpoints)
3. binds company under org (org_companies "n0")
4. binds team under company (company_teams "n0")
5. inserts team_members rows for every orphan agent (n1, n2, …)
6. inserts company_teams rows for every orphan team
7. inserts org_companies rows for every orphan company
8. commits, returns the created ids + moved counts
cm-api (routes/structure)
- GET /api/structure/orphan-counts → { agents, teams, companies }
- POST /api/structure/reify-orphans → { org_id, company_id, team_id,
moved_* }. Trims + rejects any empty name; validates before
starting the transaction so a 400 never rolls anything back.
Frontend
- New OrphanMigrationDialog: fetches counts on open, three name
fields (defaults: Organization "My Workspace", Company "General",
Team "Everyone"), POSTs on save. "Nothing to migrate" state
disables the save button when the workspace is already fully
wired. Copy explicitly notes that everything is renameable in the
sidebar afterward.
- Dashboard: onTreeSelect now branches on SYNTHETIC_TREE_IDS —
clicking a synthetic node opens the dialog instead of falling
through to the (nonexistent) selection. On successful reify,
router.refresh() so the sidebar + world viz reflect the new real
chain.
What this doesn't do yet (next commit)
- Wizard auto-materialize: when creating a team/company via wizard,
auto-create parent placeholders if they don't exist. Deferred so
this commit stays focused.
|
||
|
Omar Sobh
|
99e5207e69 |
sidebar: click-to-rename org/company/team + strip synthetics from world viz
Two related pieces of the "kill My Workspace" cleanup, landed
together because they share the same file:
Backend
- Three tiny inline-rename endpoints:
PATCH /api/orgs/{id}/name
PATCH /api/companies/{id}/name
PATCH /api/teams/{id}/name
Each takes { name: string }, trims + rejects empty, returns 204.
Backed by rename_org / rename_company / rename_team in cm-db —
single-row UPDATEs scoped to the caller's workspace, NotFound if
the id isn't visible.
- Registered next to the existing PATCH /:id (topology) routes so
they don't collide.
Frontend
- StructureTree accepts an optional onRename and canRename.
TreeRow: click on the label text of a renamable node → the span
becomes an <input>, focus + select-all, save on Enter or blur,
cancel on Escape. The rest of the row (row chevron / row body)
still navigates + selects as before, so single-click behaviour
is preserved for everything except the name text itself.
react-hooks/set-state-in-effect avoided by resetting the draft
in the enterEdit() click handler instead of inside a useEffect.
- Dashboard passes canRename={item.level !== "claw" && !synthetic}
(claws don't have a rename endpoint yet; synthetic scaffolding
gets reified into real rows in the next commit — the wizard
auto-materialize + orphan-migration dialog).
onRename fires the corresponding PATCH and calls router.refresh()
so the label lands in every consumer of the tree.
- World viz seed: new stripSynthetics(roots) helper walks the tree
and lifts children of any synthetic container up to their
grandparent's level. worldCanvasRoots feeds through this before
narrowRoots(). Result: the Live viz no longer shows "My Workspace"
or "Teams" nodes — real agents orbit the world root directly
(which is what you were asking for). Sidebar tree still shows
them so orphaned agents remain visible until the migration lands.
|
||
|
Omar Sobh
|
21ac35c8d4 |
research: spawn per-topic ZeroClaw team container on start (commit 1/3)
Commit 1 of the path-B (real per-topic isolation) plan. The
container spawns and its coordinates persist — nothing talks to
it yet; commit 2 wires ZeroClawDriveExecutor to prefer the topic's
URL when populated. This split keeps each landing verifiable.
Backend
- Migration 0038: research_topics gets zeroclaw_container_name +
zeroclaw_gateway_url columns. Both nullable so a topic can exist
before a spawn and teardown just NULLs them out.
- cm-db: ResearchTopic struct extended; get/list SELECTs updated;
new set_zeroclaw_container(id, workspace_id, name, url) helper
used both for spawn (Some/Some) and teardown (None/None).
- cm-api: bollard added as a workspace dep (matches cm-sandbox's
version). New research_container module:
· connect() → uses DOCKER_HOST when set (prod's socket-proxy
at tcp://socket-proxy:2375) else the local socket. Same
pattern cm-sandbox already uses.
· container_name_for(topic_id) → "research-<uuid>-team"
(deterministic so a re-start reattaches to the same
container instead of orphaning it).
· inherited_env() → propagates ZEROCLAW_*, OPENAI_*,
ANTHROPIC_*, GEMINI_*, GROQ_* from the parent server env
(provider config + tokens), stripping the server's own
ZEROCLAW_GATEWAY_URL/WORKSPACE so the team runtime doesn't
loop back on itself. Appends ZEROCLAW_GATEWAY_PORT=42617
and ZEROCLAW_WORKSPACE=/zeroclaw-data/workspace for the
team's own listener.
· spawn(docker, topic_id, repo_host_path, state_host_path):
- inspect: if the container already exists, start it if
stopped and return its coordinates (idempotent restart).
- else create with:
image = CLAWMATES_RESEARCH_TEAM_IMAGE or
clawmates-runtime:latest
cmd = [daemon, --host, 0.0.0.0]
env = inherited_env()
mounts = repo_host_path → /workspace/repo (rw)
state_host_path → /zeroclaw-data (rw)
network = CLAWMATES_RESEARCH_TEAM_NETWORK or
clawmates_core
labels = clawmates.role=research-team,
clawmates.research.topic_id=<uuid>
- creates state_host_path first so bind doesn't ENOENT.
· stop(docker, name) → stop + remove. Idempotent on 404/304.
- start_topic wires spawn after the clone completes:
· state root = CLAWMATES_RESEARCH_WORKSPACE_ROOT / <topic> /
state
· on success, persists (name, url) on the topic row so commit
2 can look them up when constructing the executor
· every failure (docker connect, docker create/start, DB
persist) is best-effort: logs and continues. A missing team
container leaves the topic pointing at the workspace-wide
gateway URL (env), preserving prior behavior.
Deploy prerequisites (not in this commit)
- The compose stack's clawmates_server service needs bind-mounts
of CLAWMATES_RESEARCH_WORKSPACE_ROOT (e.g.
/var/lib/clawmates-research:/var/lib/clawmates-research) so
paths the server writes to are visible on the host and the
spawned team container mounts the same underlying data.
- socket-proxy's ACL must allow POST + DELETE on /containers
(already the case in prod per the audited compose file).
|
||
|
Omar Sobh
|
316cdbf929 |
research sidebar: delete-with-confirm per topic
Mirror the row-level delete affordance the loops sidebar already
has. Loops was wired earlier; research had a bare title-only card
with no way to remove a stale topic.
- cm-db: research_topics::delete cascades via existing FK rules
(research_topic_agents, research_publish_approvals, and the new
research_outcomes all CASCADE on topic_id; topology_runs's
research_topic_id back-ref is SET NULL so historical runs stay).
- cm-api: DELETE /api/research/{id} → 204. Idempotent.
- Frontend: deleteTopic helper. ResearchList row is now a card
with the existing title/status/outcome header plus a trash icon
that flips the card into an inline "Delete topic + all outcomes?"
confirm strip. Confirm → red Delete / gray Cancel. If the
deleted topic was selected, selection clears; local counter
bumps the list refetch without waiting on a parent.
|
||
|
Omar Sobh
|
637e1bdd69 |
repos: sidebar actions (sync/edit/remove) + edit modal
Sidebar: - Each connection header now has three inline icon buttons: Sync now (spins while in flight), Edit (opens the modal), Remove (opens an inline confirm strip). Removes cascade repos via ON DELETE CASCADE. - The connection's last_sync_error surfaces as a red inline banner under the header — no more 'error status with nowhere to see why'. - Sync is POST /api/repos/connections/:id/sync (already existed); after either sync or delete the sidebar re-fetches so state stays consistent. Edit modal (RepoConnectionEditModal): - Loads GET /api/repos/connections/:id, pre-fills owner/base_url/label - PATCHes only the fields that actually changed; empty string on a Some(&str) field sends explicit null so the backend clears it - Sync-now + Remove reachable from inside the modal too - Rotating the token is out of scope: the modal says as much and points the user at delete + re-create through the wizard (the broker doesn't expose an update path, and rotating in place would require duplicating the whole broker->store_secret flow here) Backend: - GET /api/repos/connections/:id — same ConnectionSummary shape - PATCH /api/repos/connections/:id — owner/base_url use Option<Option<T>> double-nesting so 'omit = leave alone' and 'null = clear' round-trip distinctly through serde - repo_connections::update with COALESCE-per-field so the SQL matches the double-Option semantics without an OR-chain per field |
||
|
Omar Sobh
|
6d087bf537 |
repos: backend — schema, /api/repos routes + GitHub sync provider
Migration 0034: two tables. repo_connections carries the workspace's per-provider config (owner, base_url, label, last_synced_at, last_sync_error) and points at an app_connections row for the PAT. repos is the per-connection cache with (connection_id, external_id) unique so upsert is idempotent across re-syncs. Cascading deletes clean up cleanly on connection removal. cm-secrets grows a FetchAuthorized op — GET with the stored PAT injected as bearer, returns status + JSON body without ever exposing the credential to cm-api. This is the least-privilege door for read-only provider APIs (list repos), distinct from the InvokeHttp path that still requires a single-use approval grant for outbound writes. cm-api::routes::repos wires: - POST /api/repos/connections (broker store_secret + insert both rows + initial sync + mark_synced) - GET /api/repos/connections - DELETE /api/repos/connections/:id - POST /api/repos/connections/:id/sync - GET /api/repos (500 cap, newest provider_updated first) - GET /api/repos/:id (full detail incl. clone_url + html_url) GitHub provider inline for v1 — paginated pull of /orgs/:owner/repos (when owner set) or /user/repos (when absent), 100/page, capped at 20 pages (~2k repos) to keep first-sync latency bounded. Non-2xx surface back to the caller as sync_error; parse failures are best-effort per repo (skipped, logged, don't abort the batch). Gitea + GitLab providers land in a follow-up — mostly URL swap + response-shape adapter. |
||
|
Omar Sobh
|
973eeb272e |
research: publish approval gate + explicit state transitions
Fourth commit of the Research + Loops arc. Completes the state machine
for research topics with the publish approval gate the spec asked for.
Migration 0032 — research_publish_approvals
Dedicated small table (id, workspace_id, topic_id, requested_by,
status, decided_by/at, created_at). Keeping it separate from the
existing `approvals` table (0001) because that one is tightly coupled
to gated tool calls inside an agent run — session_key + run_id +
action_type + category + payload + preview + requested_by_agent, all
NOT NULL. Forcing those nullable would ripple through cm_safety;
cleaner to give publish approvals their own two-transition state
machine.
New endpoints
POST /api/research/:id/submit-review processing → reviewing
(v1 caller-driven; the
orchestrator hook comes
when we wire actual runs)
POST /api/research/:id/request-publish creates a pending
approval. Rejects with
409 if the topic already
has one open.
GET /api/research/publish-approvals list workspace's pending
POST /api/research/publish-approvals/:id/approve flips approval to
approved + transitions
the topic
reviewing → publishing
(which stamps
published_at)
POST /api/research/publish-approvals/:id/reject stays in reviewing; new
requests allowed
The approve/reject write is an atomic UPDATE ... WHERE status = 'pending';
the decide() repo function returns whether the caller won the race so
concurrent double-approves collapse to a single topic transition.
State machine after this commit:
standby ─POST /start─▶ processing ─POST /submit-review─▶ reviewing
─POST /request-publish + approve─▶ publishing ─(future: artifact
assembly)─▶ published
|
||
|
Omar Sobh
|
4b48c521eb |
loops: backend routes + repo + cron scheduler + HMAC webhook
Third commit of the Research + Loops arc. Lights up loops as durable
recurring topology executions:
GET /api/loops list workspace's loops
POST /api/loops create — returns webhook_token +
signing_key ONCE when webhook trigger
is enabled; never exposed again
GET /api/loops/:id detail
PATCH /api/loops/:id update definition
DELETE /api/loops/:id delete
POST /api/loops/:id/run trigger one iteration NOW
POST /api/loops/:id/enable set enabled=true
POST /api/loops/:id/disable set enabled=false
POST /webhooks/loops/:token public; HMAC-SHA256-verified
Scheduler (cm_runtime::spawn_loop_scheduler) wakes every 10s, queries the
partial index on (next_fire_at) for due loops, enqueues one topology_runs
row per fire with loop_id + iteration + parent_run_id chained back to the
previous iteration. Uses croner via the existing scheduling::next_occurrence
helper. Missed windows fire ONCE and skip the backlog — next_fire_at is
always computed strictly AFTER now(), so a late scheduler doesn't drain a
buildup.
Webhook signatures follow the same pattern as the Stripe billing webhook
(HMAC-SHA256 with constant-time hex compare). Token + signing key are
24-byte OS-RNG values; the URL uses base64-url for the token, and the
signing key is base64-std. Both surface exactly once at create time.
All three fire paths (scheduler, immediate-run, webhook) funnel through
`cm_db::repo::loops::enqueue_iteration` so the invariants stay in one
place. `iters` repeat policy is enforced by the scheduler tick; `until`
and `on_completion` land with the orchestrator hook in commit 4.
Adds cm-llm as a direct cm-api dep, getrandom for the webhook material
generator, and wires the scheduler spawn into the server binary alongside
the resume sweeper and outbox drainer.
|
||
|
Omar Sobh
|
fb047879c2 |
research: backend routes + repo + wizard refine (behind /api/research)
Second commit of the Research + Loops arc. Builds on 0030 by lighting up
the container CRUD, the agent-slot attach/detach, the standby→processing
transition, and the wizard's one-shot LLM refine.
GET /api/research list workspace's topics
POST /api/research create (accepts wizard output)
GET /api/research/:id detail (topic + attached agents)
PATCH /api/research/:id update non-status fields
POST /api/research/:id/agents attach agent (idempotent)
DELETE /api/research/:id/agents/:agent detach
POST /api/research/:id/start standby → processing
POST /api/research/wizard/refine one-shot LLM refine
The refine endpoint accumulates the workspace's default LLM provider's
stream into a JSON object (`{title, description}`) with the tight system
prompt at the top of the module. Same provider routing as agent runs
(via runtime.provider()), so a workspace already using GLM/Kimi gets it
for free.
State machine's remaining transitions (processing → reviewing on last
run_completed; reviewing → publishing via approvals gate) land with the
orchestrator hookup + approvals extension. Publish approval and loops
are separate commits still to come.
Adds cm-llm as a direct cm-api dep (previously only pulled transitively
via cm-runtime) so the refine endpoint can build a ChatRequest. Uses
sqlx::query! for compile-time verification; .sqlx cache generated on
morpheus against a fresh migrated DB.
|
||
|
Omar SobhandClaude Opus 4.8
|
7589f62aca |
feat(a2a): External access panel + GET /api/a2a/settings
Adds GET /api/a2a/settings (enabled + publicBaseUrl) and an A2ASection in the Fleet overview: enable/disable A2A for the workspace, mint/list/revoke external bearer tokens (token shown once), and the discovery URL. Claw/skill publishing is configured per-claw (follow-up). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
cbfa0ff24f |
feat: agent-to-agent platform on ZeroClaw 0.8.2 — rooms, delegation, A2A ingress
Builds on the v0.8.2 runtime. Four workstreams, all behind the §15 MCP door:
- Group rooms (Phase 1): migration 0026; N-way threads repo with a DM/room
count-guard; chat.send {room} + room.create/invite/leave tools; RoomMessage
-> room.message SSE; /api/claw-chat/rooms* APIs; Observer room badge.
- Per-claw door identity: door caller_agent resolves the X-ZeroClaw-Agent
header (set by the fork) to the specific claw, falling back to roster[0].
- Gated delegation bridge (Phase 3): clawmates__delegate door tool drives a
sibling via the existing /ws/chat ZeroClawDriveExecutor (not A2A); self-deny,
per-workspace hourly budget, audit trail, untrusted-banner result. Native
in-daemon delegation stays off (it would bypass the door).
- A2A tenant ingress (Phase 2): migration 0027 (workspace_a2a + a2a_tokens);
runtime_provision enable_a2a_server/publish_claw; routes/a2a.rs tenant-aware
proxy (per-workspace tokens, injected internal bearer, daemon stays internal,
cards URL-rewritten to the cm-api edge); a2a.invoked taxonomy.
Tests: cm-db room repos, cm-runtime chat tools, door units. sqlx cache updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
osobhandClaude Opus 4.8
|
96d1409a23 |
style: cargo fmt --all
Apply rustfmt (toolchain 1.96.0) to satisfy the CI Format check. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
ed339c2121 |
Fleet tools: one-click per-node update (Phase 2)
The ↑ badge on each tool card is now a button: confirm → POST
/api/nodes/{id}/tools/{tool}/update → daemon runs the tool's own updater + re-probes.
- daemon: tool_update op (spawned task so the 170s update can't stall the WS loop;
re-probes + re-sends node_tools after). Fixed command allow-list (no arbitrary
shell): claude/glm → `claude update`; kimi → `uv tool upgrade kimi-cli`; ollama →
brew upgrade (mac) / install.sh (linux); else unsupported. 4KB output cap.
- cm-api: call_timeout/request_timeout (long ops); POST .../tools/{tool}/update
(workspace-scoped, allow-list) → {ok,output}.
- frontend: ↑latest becomes an Update button → confirm → spinner → refresh/err.
Note: claude/kimi/glm are user-space (no sudo); ollama on Linux uses install.sh
(needs sudo — works on passwordless nodes, returns an error otherwise; surfaced in UI).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
9263418fcb |
Fleet: per-node dev-tool version cards + nightly latest-check (Phase 1, read-only)
Each node card now shows installed versions of Docker / Claude Code / Kimi / GLM /
Ollama (conditional per node) under the ssh card, with an "update available" badge.
- daemon: probe_tools() finds docker/claude/kimi-cli/ollama across candidate bin dirs,
extracts semver from --version, reports {"t":"node_tools",...} on connect + every 15m.
- migration node_tools + tool_latest; cm-db repo node_tools (upsert/list/latest).
- cm-api: fleet.rs NodeTools uplink → upsert; tool_versions.rs spawn_latest_checker
(24h, npm/pypi/github; docker display-only); GET /api/nodes/{id}/tools (glm mirrors
claude). Spawned in clawmates-server.
- frontend: NodeTools cards on each HostCard with the ↑latest badge.
Phase 2 (one-click update execution) intentionally deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
3554a3aaf2 |
CI: remove k8s stages, fix the Docker-level pipeline green
Survey + fixes so the pipeline passes at the Docker level (no k8s).
- Remove k8s: drop the `sandbox-k8s` job (kind/Calico/--features k8s-tests) and the
"Helm chart lints" gate step. release.yml was already k8s-clean.
- Rust job:
- `cargo fmt --all` — fix pre-existing formatting drift (fmt --check was failing).
- clippy -D warnings: fix 3 lib warnings (cm-brain sort_by_key→Reverse, cm-api
fleet.rs doc list indentation, node_rules map_or→is_none_or).
- Regenerate the .sqlx offline cache (was missing the cm-runtime run_loop test
query → offline compile failed). DB-backed tests use testcontainers at runtime.
- Set SQLX_OFFLINE=true on the rust + e2e jobs so query! macros compile against
the committed cache deterministically (no DB needed at compile time).
- Frontend job:
- Fix the 1 ESLint error (useAgentTelemetry: no setState-synchronously-in-effect;
tag the slice with agentId + derive null on mismatch).
- Fix 2 stale panel-params tests (`terminal` is a valid app id now; assert the
current APP_IDS + use a genuinely-unknown id for the reject case).
Verified locally: fmt clean, clippy --all-targets -D warnings clean (offline),
frontend lint 0 errors, tsc clean, 86/86 frontend tests pass, build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
89c147b742 |
Command center: edit brain sections inline from the cards
Add PATCH /api/claws/{id}/brain (edit_brain): writes any of system_prompt / agent_md
/ persona / skills_md into the claw's .brain (best-effort) + persists system_prompt
to Postgres (authoritative) + commits a ClawSync revision.
Frontend: a reusable EditableSection (pencil → textarea → Save/Cancel → PATCH →
re-fetch brain). The SYSTEM PROMPT, HOW I OPERATE (AGENTS.md), and PERSONALITY cards
in the command center are now editable inline; saving writes back to the mapped
brain section. AGENTS.md card now always shows (so it can be authored when empty).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
c94784bab2 |
Fleet: actionable executions — rules engine + metrics-aware placement (Phase 2)
Turn the Beszel-tapped metrics into a self-managing loop. - migration node_rules (workspace/node-scoped: metric op threshold, for_seconds, action JSONB, last_fired). - cm-db: repo/node_rules.rs (CRUD + list_enabled); node_metrics::eval_all merges Beszel + heartbeat scalars per node + a headroom() heuristic; nodes::status_of; heartbeat now PRESERVES a `draining` status across heartbeats (so a cordon sticks). - cm-api: node_rules.rs evaluator (spawn_evaluator, 20s) — when a metric condition holds for the rule's window it fires drain / undrain / alert (in-memory sustained + cooldown tracking, modeled on the node sweeper); routes/beszel.rs rules CRUD (GET/POST/PATCH/DELETE /api/fleet/rules); spawned in clawmates-server. - cm-runtime: placement_node() is metrics-aware — a `draining` node stops receiving new agent sandboxes (falls back to local), so the drain rule is actionable. - frontend: FleetRules section in the Local view — build rules (node · metric · op · threshold · duration → action), toggle/delete, with fired-history. The loop: hot/overloaded node → rule drains it → placement avoids it → recovers → undrain rule brings it back. Deployed; node_rules migration applied. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
36a227566b |
Fleet: Beszel hub integration — rich per-node metrics + per-node monitor (Phase 1)
Tap each node's Beszel metrics (GPU/temps/disk-IO/network/per-container — beyond
our basic heartbeat) by reading the workspace's Beszel hub. The agents run in
WS-only mode with no locally-readable socket, so (per the de-risk) the server taps
the hub's PocketBase API instead of the daemon reading agents — no daemon changes.
- migrations: workspace_beszel (BYO hub URL + login, server-side only, mirrors the
Tailscale BYO pattern) + node_metrics (latest scalar columns + JSONB blob).
- cm-db: repo/fleet_beszel.rs, repo/node_metrics.rs; nodes SELECT joins node_metrics
(gpu_pct/temp_max surfaced on node_json for the live cards).
- cm-api: beszel.rs client (auth-with-password, poll `systems`, map to nodes by
hostname, upsert metrics) + a 15s spawn_poller; routes/beszel.rs (connect/status/
disconnect + GET /api/nodes/{id}/metrics with history proxied live from the hub).
- frontend: HostCard gains a GPU/temp readout + a Monitor button; NodeMonitor is a
full-width per-node page (current panel + CPU/mem/GPU/temp/net/disk charts from the
hub's 1m history); a "Beszel monitoring" connect form in the Local view.
Reachability confirmed: gw-04 → the hub over the tailnet (100.123.224.84:8090). Needs
the user to connect their hub login to activate the poller.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
94828ed887 |
Fleet: robust real-time connectivity + mosh-inspired reconnecting terminal
Nodes flapped online/offline and the terminal died on the first blip. WebSockets are the right transport (outbound, NAT-friendly); the fixes harden around it. Server (cm-api): - Anti-clobber connection epoch: a reconnecting daemon gets a fresh epoch; a stale run_channel's teardown only clears the hub + sets offline if it still owns the slot — so a lingering old channel can't flip a live reconnection offline (the main false-offline cause). - WS keepalive: run_channel now pings every 15s and tears down if no inbound frame (incl. pong) for 35s — dead links detected in seconds, not minutes. - Staleness sweeper backstop: spawn_node_sweeper (8s tick / 20s window) wired in clawmates-server, so a vanished node goes offline within ~28s even if its channel hangs (mark_stale_offline was defined but never called). Daemon (clawmates-node v0.3.0): - Heartbeats off the select thread (dedicated thread owns System + blocking docker/tailscale/disk CLIs) so a slow op never starves heartbeats/pongs. - Each handle_frame runs on its own task; added a 40s inbound idle deadline so a half-open socket triggers a reconnect. Frontend: - useNodes streams /api/nodes/live (SSE push) instead of a 3s poll; isLive() derives online from lastSeen freshness (<15s) so a transient column flip never shows a healthy node down. - Node terminal: clean auto-reconnect loop (re-mint ticket -> reconnect -> tmux re-attaches and redraws the live screen = mosh-style snap-to-state over TCP), replacing the [disconnected] dead-end. Mosh evaluated: harvest principles (session/transport decoupling, snap-to-state, already given by tmux), don't adopt — UDP is incompatible with our browser+CF+NAT topology and it's GPLv3. Removed temporary terminal debug traces + /api/debug route. Verified: node holds steadily online (heartbeat 1-3s, no flap) and goes cleanly offline when the daemon stops. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
27f5d05f96 |
Fleet terminal: daemon self-diagnosis + immediate banner + debug route
The server trace showed pty_open is sent but the daemon (morpheus, v0.2.0) emits
no pty_out — so the PTY spawn was dying silently. Instrument it:
- daemon open_pty: log open/tmux/first-read/EOF/error/total to stdout, and send
an IMMEDIATE banner pty_out ("[clawmates] host shell on <host> — starting…") so
the browser confirms the relay even before the shell draws. If open_pty fails,
send the error as pty_out (was a silent pty_exit). Bump to v0.2.1.
- cm-api: temp GET /api/debug/node-pty/{id}?dbg=… opens a node terminal and reads
~2s of output with no browser/auth, to test the relay in isolation.
- NodeTerminalApp + ticket/ws routes already log each hop ([node-term]/[fleet-term]).
Diagnostic logic: banner shows + shell doesn't → relay ok, shell is the problem;
nothing shows → relay broken; daemon "EOF after N bytes" → shell exited.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
fb59378aa2 |
Fleet P2b: run agent sandboxes on connected nodes (RemoteDriver + placement)
Agents can now provision their sandbox on a connected fleet node instead of the gateway host. Local stays the strict default, so existing agents are byte-for- byte unaffected until explicitly placed elsewhere. Security parity: the daemon links the REAL cm-sandbox DockerDriver and runs the typed container ops (sb_provision/sb_exec/sb_destroy/sb_health/sb_list) through it — identical hardening (cap-drop ALL, seccomp, no-net, read-only, non-root) to local sandboxes. cm-sandbox spec types are now Serialize/Deserialize so the spec crosses the channel. - cm-api: RemoteDriver (impl SandboxDriver over the node channel) + HubDriverProvider (impl cm_runtime::NodeDriverProvider, hands out a driver only for connected nodes via a sync online set) + NodeHub.call/is_connected. AppState.with_node_hub so the hub is shared with the placement provider. - cm-runtime SandboxManager: driver_for(node_id) routes by the recorded agent_containers.node_id (local default = existing driver, identical path); placement_node() reads the workspace setting and falls back to local if the node is offline; exec/release route accordingly. NodeDriverProvider trait. - DB: 0020_workspace_placement + repo (for_agent/get/set/clear). - main.rs: build the NodeHub first; inject HubDriverProvider into the agent manager + share the hub with AppState. - API+UI: GET/PUT /api/fleet/placement + a "Run agents on: Local / <node>" selector in the Fleet overview. Note: a node must be able to pull the agent image (the daemon docker-pulls it); interactive PTY for agent containers on remote nodes is not wired (Terminal app stays local) — the in-dashboard node shell already covers host access. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
33aa9c0693 |
Fleet P2b: node sandbox-readiness check (hardened workload on a node)
Proves a connected node can host hardened agent workloads end-to-end, without
touching the agent run loop (zero blast radius on existing agents).
- Daemon: typed `sb_check` op — pulls a tiny image and runs it fully locked down
(cap-drop ALL, no-new-privileges, no network, read-only rootfs, non-root,
memory/pids caps), then tears it down. Fixed command; nothing caller-supplied
runs (preserves the exec-hardening invariant).
- cm-api: NodeHub.sandbox_check + POST /api/nodes/{id}/sandbox-check.
- UI: a shield "sandbox check" button on each online node card streams the
result (✓ SANDBOX READY + container id/uname).
This validates the full provision→run→destroy mechanism on nodes. The remaining
P2 work — wiring real agent deploys to auto-place onto nodes — is its own
subsystem (a RemoteDriver reusing the local DockerDriver for security parity,
agent-image distribution to nodes, and node-routing in SandboxManager) and is
best done as a focused pass; it is intentionally NOT bundled here to keep the
core agent path untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
f5f96508eb |
Fleet P2a: in-dashboard remote terminal (PTY over the WSS channel)
You can now open a real shell on any connected node from the dashboard — the
daemon spawns a host PTY and streams it over the existing outbound control
channel (no inbound port, no Tailscale brokering needed).
Daemon:
- portable-pty host shell sessions: pty_open/pty_in/pty_resize/pty_close ops; a
reader thread streams base64 pty_out frames. Outbound frames now funnel through
one mpsc channel so PTY output and heartbeats interleave.
cm-api NodeHub:
- per-connection pty_sinks + sid multiplexing; open_terminal/terminal_input/
terminal_resize/terminal_close; in-memory single-use terminal tickets (the
browser WS can't carry a bearer, and the session is instance-local anyway).
- routes/nodes.rs: POST /api/nodes/{id}/terminal/ticket + GET .../terminal/ws
(bridges browser xterm <-> node PTY: binary = keystrokes, text = resize).
Frontend:
- NodeTerminal xterm modal (reuses the agent Terminal's xterm setup); a Terminal
button on each online node card opens a shell.
This proves the bidirectional streaming-over-channel mechanism the RemoteDriver
will reuse. Remaining P2: RemoteDriver + placement (run agent workloads on nodes).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
7332d69f8a |
Fleet P1: BYO Tailscale + network metrics, Tailscale SSH, exec hardening
Security hardening: - The gateway no longer sends arbitrary shell to nodes. The WSS exec op is replaced by a typed `verify` op the daemon runs itself (fixed host+docker check); future container ops are typed too. cm-api NodeHub.verify() + the daemon's handle_command only dispatches vetted ops. BYO Tailscale: - migrations/0019_workspace_tailscale.sql + cm-db fleet_tailscale repo (store the user's Tailscale API key + tailnet, server-side only). - cm-api routes/tailscale.rs: POST/GET/DELETE /api/fleet/tailscale + GET /api/fleet/tailscale/devices (proxies api.tailscale.com device list). - Daemon: --tailscale-authkey → `tailscale up --authkey … --ssh` (enables Tailscale SSH for keyless user access); else `tailscale set --ssh=true`. Reports its tailscale IP (already). UI: - Fleet overview gains a Tailscale section: connect (key+tailnet) + live tailnet device status (online/last-seen/IP/os). Node cards show a copyable Tailscale SSH target (ssh <ip>). Remaining: P2 — RemoteDriver + placement (run agents on nodes) and the in-UI remote terminal (PTY proxied over the WSS channel). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
2bdd0a23e8 |
Fleet P0: node registry + daemon + health + connect-host wizard
Users can connect their own local-hardware nodes into a fleet. Each node runs a
new Rust daemon that dials home over an outbound WebSocket, reports host health,
and runs commands we send.
Backend:
- migrations/0018_fleet_nodes.sql: nodes + node_health tables + agent_containers
(node_id, workspace_id) index. cm-domain NodeId.
- cm-db repo/nodes.rs: create/auth/list+health/get/heartbeat/set_status/delete
(unchecked sqlx, no .sqlx regen).
- cm-api fleet.rs NodeHub: live daemon channels (node_id→sender) + the WS channel
runner (heartbeat→DB upsert, exec request/response framing). routes/nodes.rs:
POST /pair, GET /nodes, SSE /nodes/live, POST /{id}/exec-test, DELETE /{id},
WS /nodes/agent (token-auth). Wired into AppState + router.
Daemon (new crate crates/bins/clawmates-node):
- sysinfo host metrics (cpu/mem/pressure/swap/disk/load/containers), outbound WSS
dial + reconnect, heartbeat loop, exec command handling, tailscale-ip probe.
install.sh convenience installer.
Frontend:
- Fleet sidebar item + FleetOverview + LocalHardware node-health cards (live via
/api/nodes, 3s poll) + ConnectHostWizard (install → verify connection →
exec-test). InfraStage dispatches fleet/local; default selection = fleet.
Deferred: P1 (BYO Tailscale + network metrics), P2 (RemoteDriver + placement so
agents actually run on connected nodes).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
53279e6339 |
World: Phase C — replay scrubber (Gource-style playback)
- /api/world/replay?hours= — reconstructs a sorted, timestamped taxonomy timeline from the workspace's run history (agent_runs ⋈ sessions) for client playback. - engine.clearWorldNodes() — drop transient world nodes/targets for a loop restart. - WorldCanvas: a WorldClock control (Live ⇄ Replay) feeding the same engine — play/pause, 1x/2x/4x speed, seek bar, loop. Live subscriptions detach in replay. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
39bacbd1d2 |
World page: live WebGL Gource-style visualization (Phases A+B)
Replaces the static React-Flow world forest with a real-time WebGL view of the swarm working, plus the live event contract that feeds it. Phase A — the live contract + feed: - frontend/src/lib/live/taxonomy.ts — the 13-event Clawmates Event Taxonomy as typed TS (the shared World/Observe contract). - frontend/src/lib/live/useClawmatesLive.ts — native shared-singleton live client (EventSource to /api/world/live, typed fan-out, replay-of-last-state to late subscribers, refcounted, synthetic fallback so views are always alive). - crates/cm-api/src/routes/world.rs — GET /api/world/live, authed + workspace- scoped SSE emitting the taxonomy (real agent.status from live-container state, a topology.update of the workspace's agents, telemetry with real doorsPending), mirroring run_events_sse. normalize() seam documented for run_events->world.touch. Phase B — the WebGL engine: - frontend/src/components/world/engine.ts — Gource-inspired force-directed model: org/company/team tree (sibling repulsion + parent spring + friction), agent pawns that converge on the touched node and beam it, world nodes that glow with heat and fade when idle. Framework-agnostic (renderer-independent) state+math. - frontend/src/components/world/WorldCanvas.tsx — three.js scene (ortho cam, UnrealBloom), render loop syncing engine state, camera auto-fit, HTML labels, raycast click->select, Hierarchy/Flat/Live formation switch. - Dashboard.tsx: swap <WorldFlow/> -> <WorldCanvas/> at the world-tier seam (shared claw-tier pieces untouched; WorldFlow.tsx kept for now). - Adds three (+ @types/three). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
e9ce368ec1 |
Scaling Phase 1: multi-tenant onboarding + replica-safe coordination
Decouples "many users" + "many server replicas" from "many machines" so the platform is tenant-isolated and horizontally safe on the current single node. - Per-signup workspaces (cm-auth): a new hosted-identity sign-in provisions and owns its own workspace instead of joining the first. Config-gated by auth.per_signup_workspace (default off); concurrent first-logins serialized by a per-subject advisory lock so no duplicate workspaces. - Terminal tickets in Postgres (migration 0016, hashed, single-use): any replica can redeem a ticket minted by another. Drops the in-process ticket map. - Container registry in Postgres (migration 0017, agent_containers): Terminal and Sandbox managers resolve an agent's container through a shared registry, so a 2nd replica reuses it instead of spawning a duplicate. node_id recorded as 'local' (Phase 2 hook). Boot reconcile removes only true orphans, so terminals now survive a redeploy (tmux sessions resume). - Per-workspace quotas (cm-api/quota.rs): plan-tier caps on agents + live containers, enforced at agent create + terminal spin-up (reconnects allowed), returned as HTTP 402. New GET /api/quota surfaces usage vs limits. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
e61724ff82 |
Agent computer: terminal (tmux + drives + tabs), Obsidian vault, UI polish
Terminal app (xterm ⇄ WebSocket ⇄ per-agent themed container):
- zsh + oh-my-zsh + powerlevel10k image (agent-terminal), runs as uid 65532 to
share read-write ownership of the file-drive volume with the server.
- Interactive PTY in cm-sandbox (bollard exec tty/attach + resize) + a
TerminalManager; ticket-authed WS bridge routed straight to the backend via a
Traefik PathRegexp(/ws) rule. MOTD greets the user by name.
- tmux resumable sessions; multi-tab (one tmux session per tab, same container),
drag-to-reorder, rename, and a Save that persists named tabs to the server
(terminal_tabs, migration 0014) so they survive logout / a new device.
- Files drives mounted per-agent (subpath) at ~/drives/{documents,received,
shared}; a reconciler keeps the Files app's index in sync with terminal writes.
Storage moved to a shared `filedata` volume (CLAWMATES_STORAGE__DATA_DIR).
Obsidian vault (a markdown "second brain" per agent):
- New `vault` FileDrive (migration 0015) mounted into the terminal at ~/obsidian;
a file-content read route; a purple Obsidian tile + a vault viewer app.
Computer UI:
- Draggable computer-panel width (min = phone preset) keeping the size presets.
- Green Terminal glyph, "Claw Chat" → "Chat", colored gradient-outline app icons.
- Agent page: avatar↔activity-grid spacing + larger, uniform section fonts with
colored section-tinted tag chips.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
34f744734b |
Large World graph, agent platform, brain stack & dashboard rebuild
Frontend - Large World: collapse org/company/team tiers into one expandable React Flow hierarchy (WorldFlow) with per-click expand, persisted node positions, a compact tree sidebar, wrench multi-select delete across levels, and a sized right slide-out (phone/tablet/full) showing an agent summary + drill button. - Agent page: GitHub-style animated contribution grid (VitalsCard), collapsible System Prompt + Personality cards, restructured anatomy cards, bigger avatar with name/title header row, Markdown/JSON-aware rendering, brain registry + history, avatar generate/upload. - User-icon menu (Infrastructure/Brains/Tools/Profile/Credits) + ToolPanel; Master Planner deploy wizard (Specialists/Swarm/Scheduled/Triggered); Team Runs view; reap-progress modal; dashboard is the single live interface. Backend - cm-brain crate (.brain as the agent definition) + brain apply/history. - Hard-purge reap (FK-ordered) + sandbox release + SSE batch-delete. - Swarm self-verifying loop, mode-aware planner, web.search tool, webhooks (migration 0013), org/company/team delete endpoints, scheduler sweeps. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
148705769f |
Reaping: sandbox orphan reaper + DB expiry/retention sweeper + volume-init retry
Closes the cleanup gaps found in review (latent today; bites under load/crashes).
Sandbox containers (cm-sandbox / cm-runtime):
- label every sandbox `clawmates.sandbox={agent|browser}` at create
- SandboxDriver::list_managed(kind) (Docker label filter + K8s label selector)
- SandboxManager::reconcile_orphans(ttl) + spawn_reaper: removes engine
containers no live handle owns (ZERO = all)
- boot reconciliation (every pre-existing sandbox is an orphan from a dead
process) + periodic reaper (5m interval / 10m TTL)
- SIGTERM graceful drain: serve().with_graceful_shutdown → shutdown() both
managers so a redeploy can't leak; destroy errors now logged not swallowed
DB expiry/retention (cm-db cleanup.rs + cm-api cleanup_sweeper, hourly):
- expire auth_sessions + oauth_states past expires_at (security)
- prune sent outbox(7d), run_events(14d), routine_runs(30d), terminal
topology_runs(90d), consumed execution_grants(7d)
Compose: volume-init restart "no" → on-failure:5 (retry instead of wedging boot).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
540c74f42e |
Adopt design comps: dark system, new landing/auth, dashboard shell + canvas
Re-skins the whole app to the dark design comps and wires the new surfaces to
the backend (the /api proxy + auth + schemas are unchanged).
Design system:
- globals.css: remapped @theme tokens to the comp palette (#08080a base, coral
#ff6f61, status cyan/green/amber/purple/teal); token names preserved
- MeshMark: triangle + 3-node brand glyph; cm-flow/cm-blink/cm-halo keyframes
- marketing flipped light → dark
Backend (migration 0012):
- agents.model_binding (persisted on team deploy) + GET /api/claws/{id}/runtime-config
- routine_runs table + scheduler journaling + GET /api/routines/runs
- GET /api/claws/{id}/compartments (anatomy aggregate)
- GET /api/structure/stats (workspace counts)
Frontend:
- Landing: full dark marketing page (hero constellation, deploy ladder,
12-topology taxonomy, recursive execution, compare/Pareto, safety, self-host)
- Auth: dark split-panel AuthShell + comp LoginForm + Clerk SignIn themed dark
- Dashboard shell: TopBar (breadcrumb + live stats + deploy + user) + StatusBar
(runner/sandbox/doors); rail slimmed to 60px + 252px context column
- ConstellationCanvas (radial recursive) replaces the graph view in StructureCanvas;
selecting a claw opens ComputerPanel (apps/now-running/dock); RoutinesPanel
- Claw anatomy view (/claws/[id]/anatomy) from compartments + runtime-config
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
3eca4ed70c |
Recursive deploy ladder: Company + Org tiers, mesh mark, two-tier rail
Completes the scale ladder (single → team → company → org). Every tier is a
topology whose nodes are the tier below; running a parent recursively runs each
child's sub-topology down to the leaf claws.
Backend:
- migration 0011: companies/company_teams, orgs/org_companies, topology_runs.tier
- cm-db repos for companies + orgs (mirror teams)
- TurnRequest.attrs (forwarded from node.attrs) for child-id binding
- SubTopologyExecutor (recursive_exec.rs): a parent "turn" runs the child's
sub-topology; durability via parent updated_at keepalive + cancel propagation
+ depth cap; boxed future breaks the org→company recursion
- topology_worker selects executor by job.tier
- routes: /api/companies, /api/orgs (create/list/get/run) + unified
/api/structure/{level}/{id} for the zoom canvas
Frontend:
- MeshMark: node-mesh brand glyph (replaces the claw PNG), tier variants
- TopologyGraphView: optional onNodeClick/nodeMeta + dark-token theming
- StructureCanvas + Breadcrumb: one recursive zoom view for every tier
(drill down on node click, breadcrumb up); TeamRunPanel extracted + shared
- two-tier Discord-style rail: StructureRail (mesh mark + org/company/team
glyphs + tools popover + deploy + user) | RosterColumn (selected group's
children, or your claws); SecondaryNav for cross-cutting tools
- ComposeWizard (company/org) wired into DeployWizard; /companies + /orgs pages
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
8123a27bcf |
Teams (deploy ladder rung 1): schema + provisioning + API
A team = a baseline topology staffed with real claws. Migration 0010 (teams + team_members node→claw bindings) + cm-db repo/teams.rs. runtime_provision.rs turns a claw into a live runtime agent claw_<id> via the synced gateway config API (#7468): create agent + bind model_provider (mapped from chosen model) + risk_profile=toolfree + clawmates_door bundle — atomic, immediately drivable. routes/teams.rs: POST /api/teams (create claws + provision + build(kind,roles) + bind node.attrs["agent"]=claw_<id> + persist), GET /api/teams[/{id}], POST /api/teams/{id}/run (enqueue a durable run of the team graph — reuses the topology worker + SSE). v1 persona = topology role via the prompt builder; the claw's system_prompt stays its chat identity. Spike confirmed: runtime agent provisioning works; IDENTITY.md persona works for API models (Gemini/Groq), masked by CLI models (Claude/Kimi Code). 16 cm-api tests + provision unit tests pass, clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
6e87433c66 |
Close gaps: GLM-judge (anthropic registry) + run cancel + deep-link
#2 GLM judge (closes #114): registry NamedProvider gains a `format` field; build_provider_registry builds an AnthropicProvider for format="anthropic". GLM's coding/OpenAI endpoint is ToS-throttled for raw SDK, but its Anthropic endpoint (api.z.ai/api/anthropic) accepts raw API calls (verified x-api-key -> glm-4.7), so CLAWMATES_JUDGE_MODEL=glm:glm-4.7 routes the door governor / topology judge through GLM with no runtime-routing. (Kimi-as-judge still needs a Platform key — coding key is agent-only.) #3 run-control: POST /api/topology-runs/{id}/cancel (workspace-scoped, queued/running only); the worker honors it at the step boundary (checks current_status in the checkpoint callback) and won't clobber a cancel with `failed`. Frontend Run tab gains a Cancel button and clickable recent runs that deep-link into a live/replayed stream (SSE replays from checkpoint). cm-* tests (incl. new cancel test) + clippy + frontend lint/typecheck green. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
f845dfb15f |
Topology runs: SSE live-progress (replace polling)
Backend: GET /api/topology-runs/{id}/events streams Server-Sent Events by
tailing the durable per-step checkpoint the worker already writes — a `step`
event per newly-completed step (replayed on connect so reload/reconnect
re-attaches), then a terminal `done` event with the final output/error. Each
step carries its index as the SSE id, so the browser's Last-Event-ID resumes
without duplicates on reconnect. No new table, no worker change — reuses the
checkpoint; avoids run_events' agent_runs FK.
Frontend: the Run tab now opens an EventSource (through the same-origin proxy,
which adds the bearer) instead of polling — appending steps as they stream and
finalizing on `done`. One streaming connection, lower latency, auto-resume.
cm-api builds + clippy clean; frontend lint + typecheck + next build clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
272669e1f5 |
Durable topology jobs (3+4/4): background worker + async run API
POST /api/topologies/run now ENQUEUES a durable job and returns 202
{run_id, status:queued} instead of executing inside the HTTP request — the
prerequisite for long-horizon runs (no client/proxy/LB timeout, survives
restarts).
topology_worker: a spawned loop that requeues stale running jobs, claims the
next queued one (CAS via FOR UPDATE SKIP LOCKED), drives it through
execute_resumable, and checkpoints RunProgress after every step; on crash the
stale sweep requeues it and the next claim resumes from the last checkpoint.
Wired into server startup beside the scheduler + resume sweeper.
GET /api/topology-runs/{id} now reports lifecycle status/kind/error/checkpoint
+ the result blob (kept the `comparison` field name for back-compat with the
compare UI; null until completed). list_runs includes status + kind.
Tests: durable lifecycle (enqueue→claim→checkpoint→complete) + stale-requeue
resume, both green; p0 endpoints (compare path) unchanged. 13 + 2 tests pass,
clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
87f612016e |
1B: Clawmates §15 MCP door — autonomous gated egress for tool-free agents
Tool-free ZeroClaw agents get one actuator: an MCP server (POST /mcp, JSON-RPC
2.0, protocol 2024-11-05) that fronts the existing §15 machinery. The human
approver is replaced by an automated policy (default allow-all → agents are
autonomous; CLAWMATES_DOOR_POLICY=deny is a kill switch), but the governed parts
stay: every action is journaled to the append-only audit log, actions execute
through the runtime's gated-tool path, and broker credential-custody is wired in
for v2 tools. Synchronous execution returns the real result inline.
- crates/cm-api/src/mcp_door.rs: initialize/tools.list/tools.call handler; auth
(bearer -> workspace), classify effects, policy auto-decide, execute, audit.
v1 exposes email_send (-> outbox); slack/pay (broker+grant chain) is next.
- crates/cm-runtime: Runtime::{tool_descriptor_json, tool_gate_category,
execute_door_tool} — door-facing entry that builds ToolContext and consumes a
grant for runtime-executed gated tools.
- deploy/clawmates-runtime: agents now carry mcp_bundles=["clawmates_door"];
[[mcp.servers]] points at the door (bearer injected at deploy, not committed).
Validated live on gw-04: tools/call email_send -> isError:false + outbox row +
"agent|door.executed" audit, no human. 4 door unit tests + clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
6d77a0acc1 |
Topology P-zc 1A: ZeroClawDriveExecutor — real role-agents over /ws/chat
cm-orchestrator owns the topology graph; each turn now drives a real ZeroClaw
role-agent in a container via the proven gateway drive recipe, instead of a
tool-free cm-llm call.
- crates/cm-api/src/topology_exec.rs: ZeroClawDriveExecutor impl TurnExecutor —
pair (POST /pair + X-Pairing-Code, token cached) -> ws /ws/chat?agent=<alias>
-> send {type:message,content} -> drain chunk/done/approval_request/error.
approval_request is recorded as a BLOCKED GatedAction, never auto-approved (§15).
Role->alias via ZEROCLAW_AGENT_MAP, fallback ZEROCLAW_DEFAULT_AGENT (scout).
- POST /api/topologies/run {task,graph} -> execute() -> RunRecord, persisted
best-effort to the existing topology_runs table (no migration). compare stays
tool-free. from_env() is read in-handler so cm-api still boots unset.
- deploy/clawmates-runtime: example config now declares a tool-free multi-agent
role-cast; README documents the ZEROCLAW_* knobs + run endpoint.
tokio-tungstenite 0.26 (already in lock) + dev axum `ws` for the hermetic test.
3 lib tests green, clippy clean, SQLX_OFFLINE build clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|