1c9be5229141eadef2e6caa7b265c5ae1cdb9dd3
139
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
f848248fac |
feat(missions): goal conditions and phase iteration, judged on the subscription model
A phase used to complete when its topology_runs reached a terminal state --
purely structural. It marked itself done whether the agents produced the
artifact or wrote nothing at all, and it ran exactly once: execute_resumable's
skip(start) is resume, not repeat, and the only re-run path was a human
hitting the retry endpoint.
A phase can now carry `done_when`, a completion condition judged after each
pass against the evidence the agents actually surfaced. Not met and passes
remain -> the phase goes back to pending with iteration bumped, and the
verdict's reason is appended to the next pass's task text. That feedback is
what makes iteration converge rather than repeat -- the same mechanism /goal
uses, and that swarm.rs already uses for rejected work.
The evaluator runs on the SUBSCRIPTION model. CLAWMATES_EVALUATOR_MODEL
defaults to judge_model(), and a `runtime:<alias>` spec routes through
ZeroClawDriveExecutor -- a container agent on claude_cli, i.e. Claude Code on
the OAuth subscription, needing no platform API key. Same routing the door
governor uses.
Two deliberate departures from the governor's contract, both required:
- FAIL-CLOSED. Runtime::judge is fail-open and reads a verdict by
!contains("DENY"), so a model explaining why it *would* deny reads as
approval and an empty reply reads as approval. For completion that is
backwards: unsure must mean not done. The contract is swarm.rs's strict
JSON {"met","reason"} with .unwrap_or(false). Six tests cover the closed
paths -- prose, empty, missing field, non-boolean, transport error.
- judge_raw returns the raw reply; judge collapses to a bool too early to
carry a structured verdict.
Iteration scoping is the subtle part and has its own test: on pass 2 the
phase's own iteration is 1 but pass 1's completed run is still in the table,
so "are this phase's runs all finished?" must ask about the CURRENT pass or
that stale row closes out pass 2 the instant it is enqueued.
Evidence comes from phase_summarizer::collect_evidence, extracted from the
existing collect_material so the evaluator and the summary card cannot
disagree about what a phase produced.
done_when/max_iterations are promoted from phase config into columns (the
sweep filters on them every tick) and max_iterations is clamped to 20 at
insert -- the UI limits it too, but a runaway loop must not be one crafted
request away.
A phase with no condition completes exactly as before; that regression guard
is the first test in the file.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
49bcf53b84 |
feat(missions): wire the workflow registry so phase config reaches the database
workflow_registry.rs had zero call sites -- lib.rs declared the module and
nothing ever called load() or get(). So templates/workflows/*.toml was never
read, and because the client's TEMPLATE_PRESETS carries only {kind, order_idx}
with no config, PhaseSpec.config defaulted to Value::Null and every
wizard-created mission stored a null mission_phases.config.
Every per-phase setting was therefore inert. `loop = "until_no_more_int_items"`
and `commit_policy = "on_green_tests"` described a scheduler that does not
exist AND had no path to the database. benchmark_runner and security_scan
already read phase_config(); they were reading from null.
- Mission create derives phases from the recipe when none are sent, and
backfills config per phase (matched on kind+order_idx, then kind) when the
caller sends shape without config. An explicit config always wins.
- phases_for_create takes Option<&WorkflowRecipe> rather than reaching for the
global, because the registry resolves its directory relative to the process
cwd -- which under cargo test is the crate root, not the repo root.
- GET /api/workflows serves the catalog; the wizard fetches it and falls back
to TEMPLATE_PRESETS. Adding a TOML now adds a template with no FE change.
- load() runs at boot so a malformed recipe appears in the boot log instead of
silently producing a mission with no phase config.
Also fixes a latent bug in all five recipes: `default_team_template` was
written below the first [[phases]] block, and TOML scopes a bare key after a
table header INTO that table -- so it parsed as
phases[last].config.default_team_template and the real field was always None.
Invisible while the registry was dead code. Moved above the phases, with a
test asserting it neither returns None nor leaks into a phase config.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
0785ac9c79 |
feat(missions): document reader + three-tab IA for the mission page
The mission page made its own output unreadable. Reviewing a research
brief meant scrolling a 300px <pre> nested inside a 260px run box nested
inside the page scroller (plus a 4th scroll region for the description) —
and the text was capped at 6,000 chars server-side with no way to fetch
the rest, so a 53kB brief showed ~11% of itself and silently dropped the
remainder. Eight flat tabs (overview/phases/tasks/team/live/artifacts/
benchmarks/pane) mixed lifecycle, work items, people, telemetry, outputs
and infra at one level, so nothing indicated where the deliverable lived.
Reader:
- GET /api/missions/{id}/documents lists every agent output (titles +
sizes, no bodies); GET .../documents/{run_id}/{index} returns one in
full. Scoped to the mission so a run id from elsewhere can't be read.
- MissionOutputReader: rail (documents grouped by phase) · document ·
outline (headings, click to jump). Exactly one scroll container per
column, never nested. Copy + download .md.
- MarkdownBlock gains fenced code blocks (agent output is full of ```rust,
previously mangled into paragraphs), h4-h6, heading anchors, and an
outlineOf() helper.
Information architecture:
- Three primary tabs with shallow sub-views: RUN (phases/tasks/live) ·
OUTPUT (documents/artifacts/benchmarks) · SETUP (overview/team/pane).
- PhaseRunsList shows a short excerpt with no inner scrollbar and points
at the reader for the full text.
- The header description is clipped, not scrollable; its full text now
has a home in Setup → Overview.
Missions list:
- /api/missions returns MissionListItem — Mission flattened plus
phases_total/phases_done/current_phase, so the JSON stays a strict
superset. Cards render a progress bar and "Coding · 1/2" instead of a
bare status dot.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar Sobh
|
5c63ef0ed3 |
missions: phase-completion summary card (Claude Opus 4.8 synthesized)
New phase_summarizer background worker fires on any mission_phase
transition to a terminal state (completed/failed). Aggregates every
topology_runs.checkpoint.outputs[] + mission_tasks + mission_artifacts
bound to that phase and asks Claude Opus 4.8 to produce a structured
JSON card:
{ narrative, metrics, sources, tooling, next_actions }
Rendered inline on the mission page under each completed phase via
new PhaseSummaryCard component. Metrics grid is kind-specific:
research surfaces insights/sources/int_cards/artifacts, coding
surfaces cards_picked_up/commits/tests/issues, benchmark surfaces
regressions/improvements, security surfaces findings-by-severity.
New table: mission_phase_summaries (migration 0060), unique per
phase_id — regenerates on retry.
New endpoint: GET /api/missions/{id}/phases/{phase_id}/summary.
Model overridable via CLAWMATES_SUMMARIZER_MODEL. Reuses the
ANTHROPIC_API_KEY prod already carries for mission_refiner.
|
||
|
Omar Sobh
|
5d24fd3460 |
missions: schema + provisioner skeleton for per-mission runtime containers (C3 slice 1)
- migration 0058: adds missions.runtime_container_name + runtime_endpoint
- new mission_runtime module (bollard): ensure_container /
teardown_container. Container is spawned on clawmates_core +
clawmates_edge networks with just /var/lib/clawmates-missions/{id}
bind-mounted so agents scoped to /mission/repo can only see this
missions repo.
- provider API keys forwarded from the server envs so per-mission
runtimes inherit them.
- Mission struct + repo helpers updated for the two new columns +
set_runtime_binding().
- Unit tests cover container naming determinism + entropy.
Not wired to the orchestrator yet — that lands in slice 2.
|
||
|
Omar Sobh
|
e2956cdfed |
missions: surface run output on terminal phase runs
Adds GET /api/topology-runs/{id}/output — trimmed view of the
runs checkpoint (totals + per-turn output previews, capped at
12 turns × 6kB each). The full checkpoint blob can be hundreds
of KB so it was never viable to send through mission polling.
Phase card run rows now expose a "show output" toggle for any
terminal run (completed/failed/cancelled), rendering turns,
tokens, records count, and per-turn agent text. Running rows
still get the live activity stream from the prior slice.
Diagnostic value: on a mission that "completed" without visible
work, this immediately shows whether the agents produced real
output (workspace missing / instructions vague / etc.) or
whether nothing ran at all.
|
||
|
Omar Sobh
|
94fecb526c |
missions: retry failed phases + auto-purge on re-launch
Every re-attempted phase now starts with a clean slate:
- phase_runner::launch_phase DELETEs prior status IN ('failed',
'cancelled') topology_runs for the phase before enqueuing the
new ones. Completed runs are kept for audit; only the failure
noise from earlier attempts goes.
- POST /api/missions/{id}/phases/{phase_id}/retry — resets a
failed/cancelled phase to 'pending' (auth-scoped to the calling
workspace + guarded on mission.status='running'). phase_runner
picks it up on the next 10s tick.
- MissionCanvas phase card grows a coral 'Retry' button, visible
only when phase.status='failed' and mission.status='running'.
Click → resets + refreshes; the prior failed run rows disappear
from the card as soon as phase_runner enqueues the new attempt.
Design: auto-purge in phase_runner rather than a separate 'clear
failed runs' endpoint. Users don't have to manually clean up before
retrying; the runner does it as part of the natural work of firing
a fresh attempt.
Verified: cargo check + tsc + eslint --quiet all green.
|
||
|
Omar Sobh
|
277189ea9b |
missions: phase_runner — actually execute mission phases
Root-cause fix for "we hit launch, waited overnight, nothing ran."
mission_orchestrator materialized teams + agents fine, but nothing
enqueued the actual work — mission_phases stayed 'pending' forever
and topology_runs count for the mission was 0.
New crates/cm-api/src/phase_runner.rs — background worker on 10s
poll that does three things:
1. start_pending_phases — for every mission_phase with
status='pending' AND parent mission.status='running' AND all
lower-order phases already 'completed', enqueue one
topology_runs row per team whose (mission_id, purpose) matches
the phase kind:
phase=research → teams with purpose='research'
phase=coding → teams with purpose='coding'
phase=benchmark → teams with purpose='coding' (fallback)
phase=security_scan → teams with purpose 'security' | 'coding'
Each run gets a phase-kind-specific task text combining the
mission title/description + a directive for that phase.
Flips phase to 'running' after enqueue.
2. close_finished_phases — SQL sweep that flips phases whose
topology_runs are all terminal to 'completed' (or 'failed' if
any run failed).
3. close_finished_missions — same shape for missions whose phases
are all terminal.
Spawned alongside task_card_worker in clawmates-server main.rs.
Ordering enforced by mission_phases.order_idx — a coding phase
doesn't fire until its research phase completes.
Idempotent: every state transition is guarded so double-firing on a
race is safe. When a mission has no matching teams for a phase (bad
wizard state), the phase stays pending and the runner logs a skip
rather than getting stuck in a fail loop.
Existing topology_worker picks up the queued runs and drives them
through the ZeroClaw executor as usual.
|
||
|
Omar Sobh
|
b8b8cb452e |
missions: multi-team model — pick research + development teams
Directly addresses "we want to pick one or more teams to assign to a
mission, first screen research teams, next screen dev teams." A
mission now materializes N teams, each tagged with a phase purpose.
Backend:
- 0056_mission_teams.sql — new join table
mission_teams(mission_id, team_id, purpose). team_id PK because a
team belongs to one mission-purpose. missions.team_id kept as
legacy pointer to the first minted team for single-team surfaces.
- mission_orchestrator::on_launch — reads mission.config.phase_teams
(JSONB shape { research: [tid,...], coding: [tid,...] }), mints
one team per (purpose, template) pair, records each in
mission_teams, binds the first to mission.team_id. Legacy fallback:
if config.phase_teams is absent, uses missions.team_template_id.
Hard error if both are absent.
- GET /api/missions/{id}/teams — returns
[{ team_id, purpose, team_name }], sorted by created_at asc.
Frontend wizard (step 3 rewrite):
- researchTeamIds / devTeamIds — Set<string> multi-selects
- Reusable TeamMultiSelect component (checkbox-style cards)
- Panels rendered conditionally by preset:
hasResearchPhase → "Research teams" panel
hasCodingPhase → "Development teams" panel
neither → "Teams" panel (bench/security-only missions)
- canNext enforces at least one pick in every visible panel
- submit builds config.phase_teams and passes it via CreateMissionRequest
- Review step shows both selections by name
MissionTeamTab:
- Fetches /api/missions/{id}/teams and groups by purpose
- Each purpose renders a section with per-team cards
- Falls back to a single "mission" pseudo-row for legacy missions
that only have missions.team_id (no mission_teams rows)
CreateMissionRequest no longer sends team_template_id from the wizard
— the multi-team config.phase_teams path supersedes it. The backend
still accepts team_template_id for API callers.
Verified: cargo check --workspace + tsc + eslint --quiet all green.
|
||
|
Omar Sobh
|
3ba0485e7d |
mission progress UI: auto-refresh + Team tab + Live events tab
Fills the biggest UX gap surfaced during the deploy walk: hosted
missions had no live-progress surface at all. Now they do.
Auto-refresh:
- MissionCanvas grows a second useEffect that polls getMission
every 3s while mission.status === 'running'. Stops immediately
on terminal state (completed / failed / cancelled). Phases,
Tasks, Artifacts, Benchmarks all update without a manual click.
Team tab (new):
- MissionTeamTab.tsx — fetches /api/teams/{id} + /api/team/claws,
shows a card per member with role slot + an "Open" pill that
calls onOpenClaw(clawId) → Dashboard flips to AGENT tier with
that claw selected, dropping the operator into the existing
ClawCommandCenter surface (WorkingOnNow, ReasoningStream, etc).
Live events tab (new):
- MissionLiveEvents.tsx — polls /api/missions/{id}/runs every 5s
for the topology_runs bound to this mission, opens one
EventSource per active run against /api/topology-runs/{id}/events,
renders as a chronological scrolling feed with per-event kind
pills + per-run short-id badges. Auto-scrolls unless the
operator scrolled up. New runs auto-attach; terminal runs
close cleanly.
Backend:
- cm-db::repo::topology_runs::list_by_mission — SELECT ... FROM
topology_runs WHERE mission_id = $1 ORDER BY created_at DESC.
Uses runtime sqlx::query (not the macro) to avoid a sqlx cache
regen just for this route.
- TopologyRunSummary gains #[derive(Serialize)] + rfc3339 codecs.
- GET /api/missions/{id}/runs — workspace-scoped, returns
{ runs: [...] }.
Dashboard wires onOpenClaw on MissionCanvas → setAgentId + setTier("claw").
Verified: cargo check --workspace + tsc --noEmit + eslint --quiet
all green.
|
||
|
Omar Sobh
|
d8c8793c4a |
ci fixes: cargo fmt, eslint entities, max-lines split
CI on
|
||
|
Omar Sobh
|
bf4af48c80 |
herdr phase 3: INFRA tier Herdr sessions surface
New INFRA category "Herdr sessions" (purple sparkles icon between
Fleet and Local hardware). Shows a card per online fleet node with:
- Node name + hostname + IP
- Per-workspace agent state pills (working / blocked / done /
idle / unknown), colored dots + pane count
- "Open" button → renders that node's full Herdr TUI inline via
xterm.js (same nodeHerdrConnector + WebRTC-with-fallback the
MissionCanvas Live Pane uses)
Backend:
- node daemon: herdr_workspaces + herdr_snapshot ops
(`herdr workspace list`, `herdr api snapshot`)
- fleet_herdr::snapshot helper on top of hub.call_timeout
- GET /api/nodes/{id}/herdr/session route
Fetch flow: /api/nodes filtered to status='online' → for each,
/api/nodes/{id}/herdr/session in parallel. Snapshot errors surface
per-card without failing the whole grid.
The "Open" xterm is separate from the MissionCanvas Live Pane —
this one is scoped to the whole node's Herdr TUI (any workspace),
not a specific mission's pane. Operator toggles between nodes via
the buttons.
Verified: cargo check --workspace + tsc --noEmit both green.
|
||
|
Omar Sobh
|
47f986257f |
herdr phase 1b: fleet_herdr dispatch module + node daemon ops
The second-runtime path uses the existing NodeHub control channel —
NOT SSH. Node daemons already accept typed ops over their outbound
websocket; adding three herdr_* ops keeps everything on the auth
model that already works fleet-wide (control-channel token, no new
SSH key management, no server-container-mounted keys).
Node daemon (clawmates-node):
- New herdr_op handler in main.rs dispatching:
* herdr_dispatch — workspace create + pane split + rename + run
* herdr_status — pane get JSON (agent, agent_status, cwd)
* herdr_read — recent-unwrapped scrollback, N lines
- Herdr binary resolved from ~/.local/bin, brew, /usr/local/bin.
Missing binary returns clean error so cm-api can distinguish
"node not set up for Herdr yet" from "Herdr op failed".
cm-api:
- crates/cm-api/src/fleet_herdr.rs — dispatch / status /
read_transcript / wait_for_completion helpers on top of
hub.call_timeout(). wait_for_completion polls until agent_status
hits 'done' or an idle-after-working state, matching the SKILL
file's "either idle or done is completed" semantic.
- routes::missions::herdr_dispatch — POST /api/missions/{id}/
herdr-dispatch { cli, prompt }. Requires runtime_kind = 'local_herdr'
and target_node_id set. Manual trigger so Phase 1b is exercisable
end-to-end before Phase 1c wires the wizard + orchestrator.
Not yet wired: mission_orchestrator::on_launch still ignores
runtime_kind. Phase 1c adds the wizard picker AND the on_launch
branch that auto-dispatches on draft→running for local_herdr
missions. This commit only adds the primitives.
Verified: SQLX_OFFLINE=true cargo check --workspace green.
Phase 0 (Herdr install on fleet nodes) is the blocker to actually
exercising this end-to-end.
|
||
|
Omar Sobh
|
1f0117e35a |
mission canvas: add / edit / delete toolbar controls
Top-right toolbar grows three CRUD controls per your request:
- Plus (always visible) — opens MissionWizard, selects the new
mission on create
- Pencil (draft-only) — opens EditMissionModal for title +
description; PATCHes /api/missions/{id}
- Trash (always visible) — window.confirm then DELETEs; sidebar
selection clears via new onDeleted callback
Backend:
- cm-db::repo::missions::update_meta(id, ws, title?, description?)
— COALESCE-based partial patch
- cm-db::repo::missions::delete(id, ws) — hard delete, cascades
via FKs on phases/tasks/artifacts/benchmark_snapshots
- PATCH /api/missions/{id} (draft-only) + DELETE /api/missions/{id}
Frontend:
- lib/api/missions — updateMission + deleteMission clients
- MissionCanvas — three toolbar buttons, EditMissionModal
(title + textarea for description), local wizard state
- Dashboard — passes onSelect + onDeleted so sidebar reacts to
create + delete without stale selection
Edit is draft-only (backend enforces + button hidden past draft) so
in-flight missions can't have their brief mutated out from under
running agents. Delete is unconditional — operator responsibility to
Cancel first if a run is live.
|
||
|
Omar Sobh
|
214d0c5e9f |
task #25: per-mission repo checkout on mission launch
Closes the follow-up gap flagged when task #23 landed. security_scan and benchmark_runner now exec against $CLAWMATES_MISSIONS_ROOT/ {mission_id}/repo — this commit is what actually puts a repo there. - crates/cm-api/src/mission_workspace.rs — new module. ensure_checkout(pool, workspace_id, mission_id): * mission with no repo_id → Ok(None), no-op * repo cloned into $ROOT/{id}/repo (--depth 1) * dir already a git repo → fetch + reset --hard origin/{branch} (idempotent — every launch brings the tree in sync with the remote default_branch) Auth uses the process's ambient git credential setup (SSH agent / .netrc / helper). Tokens deliberately not embedded in URLs. - crates/cm-api/src/mission_orchestrator.rs — on_launch calls ensure_checkout after team materialization + team_id bind. Non-fatal: clone failures log and continue so research_only missions (no repo needed) don't get blocked. - deploy/compose/docker-compose.yml — new named volume missions_workspaces mounted at /var/lib/clawmates-missions on both the server (writer) and where the clawmates-runtime container will mount it (reader for docker exec). CLAWMATES_ MISSIONS_ROOT + CLAWMATES_RUNTIME_CONTAINER env vars set on the server so mission_workspace + exec_target read the same canonical values. The scan/bench trigger buttons now actually produce findings once you (a) run a mission whose repo_id is set, (b) have the clawmates-runtime container bind-mounting missions_workspaces at /var/lib/clawmates-missions. Verified: SQLX_OFFLINE=true cargo check -p cm-api + cargo test -p cm-api --test mission_orchestrator both green. |
||
|
Omar Sobh
|
ad1cee0b08 |
refine polish: before/after diff view + accept/cancel/restore
Refine no longer clobbers the mission description on click. Flow:
1. Click Refine → server generates the rewrite, returns
{ original, refined } WITHOUT persisting
2. RefineDiffModal shows a side-by-side pane (raw before,
Markdown-rendered after)
3. User picks:
- Accept → PATCH /api/missions/{id}/description commits refined
- Cancel → discards the proposal, description unchanged
- Restore original → forces a write of `original` (undo path
for accidentally-accepted refines, since Accept+Cancel is
still a two-step confirmation)
Backend:
- mission_refiner::refine returns a RefineResult { original, refined }
struct instead of persisting + returning the text
- routes::missions::refine now returns { original, refined }
- routes::missions::set_description added on PATCH
/api/missions/{id}/description (draft-only)
Frontend:
- lib/api/missions — refineMission return type is now RefineResult;
added setMissionDescription
- MissionCanvas — RefineDiffModal + DiffPane subcomponents;
accept / cancel / restore handlers wired to state
Closes task #20.
|
||
|
Omar Sobh
|
fdb8cfeecc |
slice 9 cleanup: drop legacy research/loops backend + tables
Retires the legacy research/loops backend after the missions arc
(slices 1-9) fully replaced it. Frontend cutover was 4663348; this
commit finishes the job on the backend + database.
Migration:
- 0053_drop_legacy_research_loops.sql — drops the 8 legacy tables
(research_topics, research_topic_agents, research_outcomes,
research_publish_approvals, loops, loop_agents, loop_orgs,
loop_teams) and the 3 topology_runs FK columns
(research_topic_id, loop_id, iteration). parent_run_id stays;
recursive_exec still uses it.
Files deleted (11):
- crates/cm-api/src/routes/{research,loops,research_setup,
research_pipeline,wizard_repo,probe}.rs
- crates/cm-api/src/research_container.rs
- crates/cm-db/src/repo/{research_topics,research_outcomes,
research_publish_approvals,loops}.rs
- crates/cm-runtime/src/loops.rs
- crates/cm-api/tests/research_publish_role.rs
Files edited:
- crates/cm-api/src/lib.rs — dropped 20 legacy route registrations
(all /api/research/* + /api/loops/* + /webhooks/loops + probe)
and module decls
- crates/cm-api/src/topology_worker.rs — deleted legacy dispatch
(freeze_research_outcome, advance_loop_after_completion,
continue_initial_burst, maybe_transition_research_topic,
parse_reorder_rationale, per-topic/loop gateway resolver).
reap_stuck_runs now keys on mission_id (not topic_id).
Executor path unconditionally uses ZeroClawDriveExecutor::from_env
— mission_orchestrator provisions each claw as an agent inside
the shared runtime via RuntimeProvisioner, so per-team gateway
resolution is no longer applicable.
- crates/cm-api/src/routes/topology.rs — deleted container-log SSE
endpoint (research/loop-specific), dropped loop_id filter and
iteration field from ListRunsQuery/RunSummary
- crates/cm-api/src/routes/world.rs — removed
active_research_topics/active_loops/preseed_repo_paths;
World SSE no longer emits repo:{topic}/loop:{id} landmark orbs
(follow-up task #21 tracks adding mission:{id} equivalents)
- crates/cm-api/src/runtime_provision.rs — removed now-unused
mint_workspace_service_token
- crates/cm-db/src/repo/topology_runs.rs — removed 9 legacy
helpers (research_topic_id lookup, loop_id_for_run,
iteration_for_run, active_runs_for_research_topic, etc.)
- crates/cm-db/src/repo/teams.rs — removed 4 dead helpers
(team_for_loop, team_for_research_topic + setters)
- crates/cm-api/tests/topology_jobs.rs — removed loop/topic
tests, dropped enqueue_run_with_topic helper
- crates/bins/clawmates-server/src/main.rs — removed
spawn_loop_scheduler call
- crates/cm-api/src/routes/mod.rs, crates/cm-db/src/repo/mod.rs,
crates/cm-runtime/src/lib.rs — module decls stripped
sqlx cache: regenerated against post-migration schema
(71 files changed, ~+70 / -8896 net)
Test/build: SQLX_OFFLINE=true cargo check --workspace clean;
cargo test --workspace --no-run clean.
Follow-up (task #21): World view lost the in-flight-work landmarks
when repo:{topic} / loop:{id} orbs disappeared. Add mission:{id}
orbs as the missions-era replacement.
|
||
|
Omar Sobh
|
56201a6985 |
mission canvas: add Refine button + markdown-rendered description
Adds a Refine button to the left of Refresh + Launch on the mission
detail toolbar (draft-only). Clicking it POSTs to a new endpoint that
calls Gemini 2.5 Flash to rewrite the user's freeform description into
a coherent, sectioned Markdown brief (Objective / Context / Scope /
Constraints / Acceptance Criteria / Open Questions) ready for the
research + coding agents to ingest cleanly.
Backend:
- crates/cm-api/src/mission_refiner.rs — Gemini call with a
system prompt that preserves user-provided facts, avoids
invention, and emits raw markdown (not JSON).
- POST /api/missions/{id}/refine — draft-only, 400 on empty
description or non-draft state.
- cm-db::repo::missions::set_description helper.
Frontend:
- MarkdownBlock — tiny zero-dep renderer for h1/h2/h3, bullet +
numbered lists, **bold**, `code`, paragraphs. Deliberately
small; the refiner emits a bounded subset.
- MissionCanvas — Refine button (Sparkles icon, secondary style)
to the left of Refresh; description now renders through
MarkdownBlock instead of a single <p>. Disabled while
description is empty or a refine is in flight.
- lib/api/missions — refineMission client.
|
||
|
Omar SobhandClaude Opus 4.7
|
9b5e63cbb7 |
slice 8.5: per-agent + per-team level-up endpoints
Level-up analyzes an agent's brain + recent run outcomes (or a
whole team's aggregate state), calls Gemini 2.5 Flash for structured
JSON proposals, and persists them as pending level_up_proposals
rows. Reviewer approves a subset via /apply; the applier commits
only those items.
Migration 0052 adds level_up_proposals (id, workspace_id, agent_id
XOR team_id via CHECK constraint, status, payload JSONB,
applied_items[], model, created_by, approved_by, created_at,
applied_at) + workspace/pending/agent/team indexes.
Rust surface:
- cm_db::repo::level_up::{insert, get, list_pending, mark_applied,
mark_rejected}
- cm_api::level_up::{propose_agent, propose_team, apply}
Item kinds handled by apply():
identity_refinement → UPDATE agents.system_prompt
skill_add → agent_skills_ext INSERT
skill_candidate → workspace-scoped skills INSERT
(deterministic id per (workspace, name))
brain_consolidation → set_agent_md on the brain (unlike
brain_seed::ingest, this overwrites)
roster_change / mcp_bundle_change — logged as
"not auto-applied, human runs
team-wizard" (structural changes need
human review of side effects).
API:
- POST /api/claws/{id}/level-up → { proposal_id }
- POST /api/teams/{id}/level-up → { proposal_id }
- GET /api/level-up-proposals → pending list
- GET /api/level-up-proposals/{id}
- POST /api/level-up-proposals/{id}/apply { approved_item_ids }
- POST /api/level-up-proposals/{id}/reject
Uses Gemini 2.5 Flash with response_mime_type: "application/json"
so the model returns structured JSON directly (no ```json fence
stripping needed). Configurable via CLAWMATES_LEVEL_UP_MODEL.
Follow-ups:
- Frontend diff-review UI (pick items, approve/reject)
- roster_change / mcp_bundle_change appliers (currently manual)
- Anthropic + OpenAI proposer variants
- Promote workspace-scoped skills to builtin via a curator flow
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
58963d5083 |
slice 8: security scan runner + trigger endpoint
Runs the security template's tool set (cargo-audit / gitleaks /
trivy fs / semgrep) inside the mission's team container and
materializes each finding as a mission_task keyed on the tool's
canonical id. The subsequent coding phase picks up the tasks and
applies remediations; the committer closes them by emitting
`COMPLETED: <external_id>` (Slice 5's task-card parser handles it).
Rust surface:
- cm_api::security_scan::run(mission_id, phase_id)
- Per-tool runners with JSON output parsing:
cargo audit --json → vulnerabilities[].advisory.id
gitleaks detect --report-format=json → [{ Fingerprint, RuleID }]
trivy fs --format=json → Results[].Vulnerabilities[].VulnerabilityID
semgrep --config=auto --json → results[] w/ rule+path+line fingerprint
- Tool errors surface as a `warning` task instead of failing the
scan — operator sees which need installing/fixing without a
silent no-op.
- Findings map to mission_tasks with external_id = "<tool>:<id>"
(e.g. cargo_audit:RUSTSEC-2024-0001, gitleaks:<sha>,
trivy_fs:CVE-2024-1234, semgrep:<rule>@<file>:<line>).
API:
- POST /api/missions/{id}/security-scan { phase_id }
→ { findings, tasks[] } — full task list after upsert so the
canvas can render immediately.
Frontend:
- triggerSecurityScan helper in lib/api/missions.ts. Findings
show up in the existing Tasks tab (Slice 5's UPSERT path).
Container requirements (opt-in):
- Runs inside the team container via `docker exec`, so the tools
must be present in that image. Missing = warning task, not fail.
- `-w /workspace/repo` so scanners see the mounted repo. Reads
teams.zeroclaw_container (populated on first phase run).
Follow-ups:
- MCP bundle wrapping the same tools as agent-callable functions
(currently agents scan by shelling out to `cargo audit` etc.
directly; a typed MCP wrap lands with clean audit trail).
- Auto-fire from the security_hardening workflow template on
phase transition (currently manual via API trigger).
- Bundle the four tools into the runtime image (or a dedicated
security-tools image) so operators don't have to install them.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
f843c9ddb1 |
slice 7: before/after benchmark runner
Executes a benchmark harness inside the mission's team container
and records the resulting metrics as a benchmark_snapshots row keyed
on (phase_id, iteration). Baseline pass (iteration=0) captures
before_metrics; each post-iteration call captures after_metrics +
computes delta vs baseline.
Rust surface:
- cm_db::repo::missions::upsert_benchmark_snapshot / benchmark_snapshots_for
- cm_api::benchmark_runner::{baseline, after_iteration, run}
- Harness enum: Auto | Criterion | CargoBench | VitestBench |
PytestBench | Shell (each with a command() vector)
- Auto detection peeks at the repo layout inside the container
(Cargo.toml → CargoBench, package.json → VitestBench, pyproject
→ PytestBench). Falls back to a Shell echo when nothing
identifiable.
- Bencher-format line parser extracts (name, ns_per_iter,
plusminus) so criterion + `cargo bench` output become structured
samples the canvas can diff.
- compute_delta pairs samples by name, emits {before_ns, after_ns,
delta_pct, direction: improved|regressed}.
API:
- POST /api/missions/{id}/benchmark { phase_id, slot, iteration? }
triggers baseline or after run and returns the mission's full
snapshot list.
- GET /api/missions/{id} now includes `benchmarks[]` in the detail
payload.
Frontend:
- New Benchmarks tab on MissionCanvas with iteration + driver
header, plus a 4-column grid (bench / before / after / Δ%) when
delta samples are present. Improved deltas render green,
regressions red.
- TS types + triggerBenchmark() helper in lib/api/missions.ts.
Wiring notes:
- team_container_for_mission reads teams.zeroclaw_container — that's
populated by topology_worker::try_team_gateway_url on first run,
so trigger baseline AFTER the mission's first phase spawns the
container.
- Not auto-fired yet by phase execution; that's the "template phase
executor" work that spans Slices 4-8. Manual API trigger works
today; automated hook is a follow-up.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
3ac3d53da7 |
slice 6: LLM + Chromium PDF renderer worker
Watches mission_artifacts for MD entries with render_pdf_status='pending'
and turns them into styled PDFs via:
1. Read source MD from <mission_root>/<path>
2. Call configured LLM (default gemini-2.5-flash) with a document-
typesetter system prompt that constrains style to a self-contained
HTML doc with inline CSS + our color palette
3. Print to PDF via `chromium --headless=new --print-to-pdf`
4. Save alongside source MD (foo.md → foo.pdf) + update
mission_artifacts.rendered_pdf_path + render_pdf_status='done'
Graceful degradation: GEMINI_API_KEY unset OR chromium missing =
row marked failed with a descriptive error, worker keeps ticking.
The frontend's "Open PDF" affordance (Slice 2) light up automatically
when render succeeds.
Boot ordering: PDF worker spawns after task_card_worker. Poll every
30s over up to MAX_PARALLEL=2 rows at a time — respects LLM rate
limits and keeps chromium's peak RAM under control.
Env knobs:
GEMINI_API_KEY — required for LLM step
CLAWMATES_PDF_RENDERER_MODEL — model id, default gemini-2.5-flash
CHROMIUM_BIN — chromium binary, default `chromium`
CLAWMATES_MISSIONS_ROOT — artifact dir root, default /var/lib/clawmates-missions
Dockerfile now installs chromium + fonts-liberation and sets
CHROMIUM_BIN=/usr/bin/chromium so the container image has everything
the renderer needs.
Also bumps workspace tokio deps to include the `process` feature
(required for tokio::process::Command).
Follow-ups:
- Anthropic + OpenAI provider variants (only Gemini in this slice)
- SSE stream on /api/missions/{id}/artifacts for the "PDF ready"
notification instead of poll-via-mission-GET
- Per-template PDF style overrides (currently one house style
for all missions)
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
f40ec075a5 |
slice 5: task-card parser + background worker
Watches topology_runs' event stream for the INT-XX marker protocol
(see skills/foundation/int-xx-marker-protocol.md) and materializes
mission_tasks rows with typed status so the canvas Tasks tab renders
a live timeline instead of raw agent chatter.
Migration 0051 adds mission_id + mission_phase_id columns to
topology_runs (nullable) so runs enqueued by a mission phase can be
attributed. Populated by future phase executors; NULL for legacy
research/loops runs (parser skips them cleanly).
New Rust surface:
- task_card_parser::parse(text) — line-scanner over TASK/WORK/
HANDOFF/TEST_PASS/TEST_FAIL/REVIEW_APPROVE/REVIEW_BLOCK/COMPLETED
markers. Strict: exact kind + colon + INT- prefix, no in-prose
matches, no bold/code-fence wrappers.
- task_card_parser::apply_for_run(pool, run_id) — reads the run's
mission binding, walks its event payloads, extracts text/output/
content/message string fields (matching every ZeroClaw event
shape we see), parses markers, UPSERTs mission_tasks via the
(phase_id, external_id) unique key from Slice 1.
- task_card_worker::spawn — 15s poller over runs updated in the
last 5 minutes. Idempotent + generous window survives server
restarts + task-scheduling jitter.
Boot wires the worker after the content loaders. Silent no-op when
mission wiring isn't populated yet.
MarkerKind → status mapping (monotonic-forward):
TASK → created
WORK → working
HANDOFF → validating
TEST_PASS → validating
TEST_FAIL → failed
REVIEW_APPROVE → validating
REVIEW_BLOCK → failed
COMPLETED → complete
Follow-ups:
- Wire phase executor to populate topology_runs.mission_id +
mission_phase_id (Slice 6/7/8 work)
- Assign assigned_agent_id via the event's producing agent alias
(currently always None)
- SSE stream on /api/missions/{id}/tasks for live canvas updates
(currently the canvas polls via mission GET)
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
565f6cae65 |
slice 4: 5 workflow templates as TOML recipes + mission-launch orchestrator
Two things land together:
1. Ships the 5 workflow recipes as TOML files under
templates/workflows/*.toml:
- research_only (hub_spoke research → MD + PDF)
- research_and_code (research → coding loop until INT-XX drained)
- security_hardening (scan → research patches → coding with
reviewer approval + full MCP bundle)
- refactor (single-pass coding with dep audit
preamble + before/after benchmarks)
- benchmark (author + baseline benchmarks per stack)
Each declares phases[], per-phase config, default_team_template.
Loaded read-only into an in-memory registry (workflow_registry)
via OnceLock — no DB row per recipe.
2. Ships the mission-launch orchestrator that closes the loop from
Slice 3.5d's mechanics. When a mission transitions draft→running,
`mission_orchestrator::on_launch`:
- Reads mission.team_template_id (skips if unset)
- Loads the team template detail (roles + skills bindings)
- Builds a topology graph from role slots via cm_topology::build
- Inserts the teams row + stamps template_id/version/risk_profile/mcp_bundles
- For each role: agent insert, model binding, runtime provision
(opt-in via RuntimeProvisioner::from_env), brain_seed::ingest
(Slice 3.5d), agent_template_link::upsert (Slice 3.5d),
team_members bind, audit trail
- UPDATE missions SET team_id = ...
Wired into routes::missions::set_status when prior.status='draft'
and new='running'. Failures log + are non-fatal (mission still
flips to running so the user can inspect + retry).
With this, Slice 3.5d's brain-seed + link machinery actually gets
populated, and the MCP skills server's template-defaults-merge path
(Slice 3.5b/d) starts serving real bindings to real agents.
Follow-ups (Slice 5-8):
- Task-card parser watches run events for TASK/COMPLETED markers
→ mission_tasks rows
- PDF renderer worker turns MD artifacts into PDFs
- Before/after benchmark runner honors phases[].config.benchmark
- Security scan MCP bundle exposes cargo-audit/gitleaks/trivy/semgrep
- Level-up endpoints diff learned-vs-seeded via agent_template_link
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
85a97dffca |
slice 3.5d: agent_template_link + brain seed helper + skills merge
Ships the lineage layer that ties agents back to their team template
and wires the MCP skills server to actually merge template default
skills with per-agent overrides.
Migration 0050 adds `agent_template_link` (agent_id PK, template_id,
template_version, role_slot, seeded_at, created_at + indexes for
template/role lookups). Populated at agent-materialization time by
Slice 4's mission-launch orchestrator; read here by the skills MCP
server and by future level-up (Slice 8.5).
New Rust surface:
- cm_db::repo::agent_template_link (upsert / get / mark_seeded /
agents_for_template — the last is what level-up's "prompt upgrade
on template N+1" query needs)
- cm_api::brain_seed::ingest(claw_id, seed_md, identity_prompt)
opens cm_brain::ClawBrain on spawn_blocking, sets system_prompt
on first touch, writes seed as agent_md, commits. Idempotent —
skips when agent_md already populated.
- cm_api::mcp_skills::mcp_skills tools/call now resolves the caller
agent's template + role via agent_template_link and merges
template default skills with per-agent overrides (was overrides-
only in Slice 3.5b).
- cm_api::team_template_loader now binds template_role_skills after
upserting each template — looks up each declared skill by name,
attaches with pin_in_context=true for foundation skills and the
first two role skills. Missing skills log + skip.
- Boot ordering: skills load BEFORE team templates so the binding
lookup resolves.
Follow-up (Slice 4): mission-launch orchestrator calls brain_seed::ingest
+ agent_template_link::upsert when minting a team from a template.
Until that lands, the link is populated only by manual writes; the
MCP merge is silent-no-op for agents without a link (falls through
to overrides-only), which matches the pre-3.5d behavior.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
7b23f61632 |
slice 3.5c: seed 15 built-in skills across the 6 stacks
Hand-authored skill catalog anchored to real 2026-07 versions:
- Rust 1.97.1 (stable), edition 2024
- React 19.2.7, Server Components + Actions
- TailwindCSS 4.3.3 (CSS-first config, Oxide engine)
- three.js r185 (WebGPURenderer stable, BatchedMesh matured)
- React Native 0.86 / Expo SDK 54+ (New Architecture default)
- cargo-nextest 0.9.140, gitleaks 8.20+, cargo-audit 0.21+
- Postgres 17 (18 in beta, don't rely on)
- CUDA Blackwell, Metal Apple7+, ROCm CDNA3
Ships 15 skills across the categories:
foundation/ workspace-repo-commit-protocol
small-focused-commits
tdd-red-green-refactor
code-review-checklist
int-xx-marker-protocol
decompose-int-items
rust/ write-rust-current-edition
rust-error-handling
cargo-test-driven-development
rust-async-tokio-idioms
backend/ postgres-migrations-forward-only
postgres-index-selection
api-pagination-day-1
frontend/ react-19-server-components
tailwind-v4-idioms
component-4-state-model
mobile/ expo-managed-vs-bare
rn-flashlist-perf
gpu/ gpu-coalescing-and-occupancy
roofline-model
threejs/ threejs-perf-and-teardown
security/ cargo-audit-workflow
secret-scanning-gitleaks
skills_loader.rs walks skills/**/*.md, parses YAML frontmatter
(name, description, when_to_use, tags), upserts via
skills_catalog::upsert_builtin. Idempotent per boot — bumps version
+ appends skill_versions row ONLY when body changes. Deterministic
sha256-derived ids so builtins are stable across boots.
Dockerfile copies skills/ to /etc/clawmates/skills. Server boot
task spawns loader alongside team_template_loader.
Follow-ups (Slice 3.5c continuation, future PRs):
- 20-30 more skills (duckdb, shadcn composition, a11y, WebGPU
migration, metal frame capture, rocprof, deep gitea forge
integration, semgrep rulepacks)
- Bind skills to team template roles (add [role.skills] refs to
templates/teams/*.toml + wire template_role_skills population
in team_template_loader)
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
3524579c9a |
slice 3.5b: clawmates_skills MCP server
Exposes the skills catalog as MCP resources so agents can discover
+ read skill markdown on demand — the delivery half of the "skills
teach how to think, MCP gives ability to act" model.
New endpoint: POST /mcp/skills (parallel to the existing /mcp door).
Protocols supported:
- initialize — handshake, advertises resources + tools
- resources/list — one entry per skill visible to the caller's
workspace. URI scheme:
skill:global/<name>
skill:workspace/<ws>/<name>
description carries the skill's description
+ when_to_use so the LLM can decide when
to reach for it.
- resources/read {uri} — returns body as text/markdown. Enforces
workspace scope on workspace-authored
skills.
- tools/list — one entry: skills__pinned_bundle
- tools/call — skills__pinned_bundle concatenates every
skill pinned for the calling agent (via
X-ZeroClaw-Agent header) with H1 headers.
Cheaper than N resources/read at turn start.
Adds the `clawmates_skills` bundle to the runtime template config
alongside `clawmates_door`, and teaches the MCP bearer rewriter
(prewrite_daemon_config_with_risk) to inject the workspace-owner
service session bearer on both `clawmates` and `clawmates_skills`
servers.
Follow-ups:
- Slice 3.5c: seed ~40-60 builtin skills across the 6 stacks
- Slice 3.5d: agent_template_link lineage — until it lands, the
pinned_bundle tool can only surface per-agent overrides, not
template defaults (documented inline)
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar Sobh
|
ade04d24cb | fmt | ||
|
Omar Sobh
|
f178e70620 | fix: mount skills catalog at /api/skills-catalog to avoid legacy route conflict | ||
|
Omar SobhandClaude Opus 4.7
|
1153d72d00 |
slice 3.5a: skills catalog — the "how to think" layer
Introduce the skills catalog: the second half of the two-layer agent
model (skills teach agents HOW to think about a problem; MCP servers
give them the ABILITY to act). Delivery via MCP resources lands in
Slice 3.5b; this slice ships the data model + API surface.
Migration 0049 extends the legacy `skills` table (from 0001_init.sql,
originally a workspace catalog of markdown snippets) with the richer
typing we need — name, when_to_use, tags, source_kind, current_version
— rather than duplicating tables. Also adds:
- skill_versions (version history for level-up promotions +
rollback; back-pointer via promoted_from
JSONB records agent_id / research artifact /
brain memory that produced it)
- template_role_skills (m2m binding skills to team-template roles
with pin_in_context + order_idx)
- agent_skills_ext (per-agent overlay: include=true adds a skill
to the bundle; include=false prunes a
template default for this specific agent)
Rust surface:
- cm_db::repo::skills_catalog with typed Skill/SkillVersion/
AgentSkillBinding structs + upsert_builtin (idempotent — bumps
version + appends to skill_versions ONLY when body changes) +
list_visible/get/get_by_name reads + template + agent binding
helpers + effective_for_agent (merges template defaults with
agent overrides, applies exclude precedence, batch-fetches skill
bodies)
- cm_api::routes::skills_catalog with:
GET /api/skills — list visible
GET /api/skills/{id} — detail
GET /api/claws/{id}/skills — effective binding (accepts
template_id + role_slot as query args to merge in template
defaults)
Follow-ups:
- Slice 3.5b: clawmates_skills MCP server exposes catalog as MCP
resources, honoring pin_in_context for auto-injection
- Slice 3.5c: seed ~40-60 builtin skills across the 6 stacks
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar Sobh
|
ee675b976a | fmt: apply cargo fmt to team template + loader | ||
|
Omar SobhandClaude Opus 4.7
|
9ba5c06a1a |
slice 3: 6 team templates seeded from TOML recipes
Team templates are the canonical rosters + tool bundles that mint
concrete teams for a mission. Every builtin ships as a TOML recipe
under templates/teams/*.toml, loaded into the DB at server boot.
Migration 0048 adds:
- team_templates (id, key, name, stack, default_topology,
risk_profile, mcp_bundles, version, source,
workspace_id)
- template_roles (m2m: template_id + slot; system_prompt,
skills[], brain_seed)
- teams gets template_id + template_version for level-up lineage
Ships 6 builtins:
- rust_sdlc — planner/coder/tester/reviewer/committer for Rust
- backend — api_designer/db_engineer/coder/tester/committer
(Postgres, DuckDB, graph DBs, wire protocols)
- frontend — designer/coder/tester/committer (React + Tailwind + ShadCN)
- mobile — designer/coder/tester/committer (Expo, RN, iOS, Android)
- gpu — arch_analyst/kernel_author/bench_engineer/coder/committer
(CUDA, Metal, ROCm from Rust)
- threejs — scene_designer/coder/shader_author/perf_engineer/
committer (three.js, WebGL, WebGPU)
Each role has a versioned system_prompt + skill list + brain_seed
markdown. Skills column is a name array today; Slice 3.5a promotes it
to a typed m2m join with the real skills catalog.
Server boot:
- team_template_loader::load_builtins reads TOML from
/etc/clawmates/templates/teams (container) or templates/teams (dev),
upserts idempotently. Deterministic uuid per template key (sha256
of a fixed namespace + key) so ids are stable across boots.
- Dockerfile copies templates/ to /etc/clawmates/templates.
Read API:
- GET /api/team-templates — list all
- GET /api/team-templates/{id} — detail with roles
Wizard:
- Step 3 rewired from a raw team_id text field to a template picker
with "LLM auto-provision" as the default option + one card per
builtin, showing stack, topology, risk profile, and description.
- Mission create now passes team_template_id (not team_id) so phase
execution knows which template to mint from.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
fbefc67878 |
slice 1: missions data model + migration
Introduce the unified `missions` tier that will replace the current
research_topics + loops split. This slice ships the data model +
backfill + skeleton REST surface; the old wizards keep working in
parallel until Slice 9's big-bang cutover.
Migration 0047 adds:
- missions (top-level workflow: template_kind + team +
schedule + status + config)
- mission_phases (ordered {research|coding|benchmark|
security_scan} phases per mission)
- mission_tasks (typed units of work, e.g. INT-XX cards,
UPSERT-keyed on (phase_id, external_id))
- mission_artifacts (MD/PDF/benchmark/security/diff files with
a pending queue for the PDF renderer worker)
- benchmark_snapshots (before/after pairs per iteration)
Backfill copies existing research_topics + loops rows into the new
tables as one-shot missions with the appropriate template_kind, so
Slice 2's UI can render the full history immediately.
New Rust surface:
- cm_domain: MissionId, MissionPhaseId, MissionTaskId, MissionArtifactId
- cm_db::repo::missions: Mission/MissionPhase/MissionTask/
MissionArtifact structs + insert (txn-wrapped)/get/list/set_status/
phases_for/set_phase_status/upsert_task/tasks_for/register_artifact/
artifacts_for/next_pdf_pending/set_pdf_result
- cm_api::routes::missions: skeleton list/create/get/set_status
routes registered at /api/missions/*
Follow-up slices layer richer behavior (template dispatch, phase
execution, task parsing, artifact rendering) on this foundation.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar Sobh
|
6e37802f73 |
claws: PATCH /api/claws/{id}/model — swap the runtime-bound model
Auto-provisioned agents (via the research + loops team wizard) show
up on the Agents page (same agents::insert → agents::roster path),
and PATCH /api/claws/{id} already handled name/job_title/system_prompt/
avatar/accent/wallpaper. The one thing that was NOT reachable from the
Agents page: swapping the model.
Add a dedicated endpoint:
- PATCH /api/claws/{id}/model { model: string }
- Persists via agents::set_model_binding (DB)
- Best-effort runtime rebind via RuntimeProvisioner::provision_claw
(idempotent — overwrites agents.<alias>.model_provider on the
shared ZeroClaw config)
- Audit-logged as 'agent.model_changed' with the new model in payload
Now an operator can open the Agents page, click a claw that was
auto-provisioned by the team wizard, and swap its model
(claude-sonnet-5 → glm-5.2 → whatever) without recreating the team.
Same DB row, same claw_id, same brain — just a new provider on the
next turn.
Frontend affordance not shipped in this commit — the endpoint is
usable via curl/psql/scripts today; a UI 'Model' picker on the claw
detail card can land in the next Agents-page pass.
|
||
|
Omar Sobh
|
956be2cf4f |
wizard: in-place auto-provision team from topic (LLM-derived, sonnet-5)
Step 5 of ResearchWizard was 'assign agents from workspace roster'.
When the roster was empty, the wizard body was hard-swapped for
NoAgentsGate — you couldn't reach step 5 at all.
Now step 5 shows a 'Team' panel:
- Big cyan card: 'Auto-provision team from this topic'. One click
runs an LLM plan pass, gets 3-5 role slots + system prompts back,
materializes claws via the existing build_team pipeline, stamps
runtime posture, returns a shape that drops straight into the
submit body's agents[]. Card flips green with the derived roster.
- Below that: the classic roster picker, but only when the workspace
actually has ≥1 claw AND auto-provision hasn't landed. Otherwise
hidden — no dead empty-state affordance.
Every gate that required agents.length > 0 to render the wizard body
or the footer is gone. canNext gains a step-5 clause: allow Next when
EITHER auto-team is ready OR the user handpicked from a non-empty
roster.
Backend
- POST /api/teams/auto-provision — accepts {title, description,
outcome_kind, topology_kind?, model?, risk_profile?, mcp_bundles?}.
Derives topology from outcome_kind (integrations → pipeline; else
hub_spoke). LLM plan pass yields a JSON roster of 3-5 roles
(role_slot, name, system_prompt). Materializes team + claws via
build_team, stamps risk_profile (default research_web_readonly) +
mcp_bundles (default [clawmates_door, gitea_forge]). Response
carries team_id + agents[] in the shape /api/research already
expects.
- Every provisioned claw runs on claude-sonnet-5 by default;
overridable via the model field.
Follow-ups (not in this slice):
- Same picker in LoopsWizard (slice C — parallel change, same API).
- Post-create 'Team' section on ResearchCanvas / LoopsCanvas so
users can rebind after the fact (slice D).
- Full Teams tier UI + Agents-page deprecation (slice E).
|
||
|
Omar Sobh
|
6066e93889 |
teams: per-team runtime posture (risk_profile + mcp_bundles) + FK from loops/topics
Foundation slice for letting a coding loop bring its own team instead
of reusing the paired research topic's team. Turns out the teams
table already exists (0010_teams.sql) with full CRUD — this scales
back to the minimal missing bits:
Schema (0045_teams.sql)
- ALTER TABLE teams ADD risk_profile TEXT (NULL = template default)
- ALTER TABLE teams ADD mcp_bundles JSONB DEFAULT '[]'
- ALTER TABLE loops ADD team_id UUID REFERENCES teams ON DELETE SET NULL
- ALTER TABLE research_topics ADD team_id UUID REFERENCES teams
- Two partial indexes (team_id NOT NULL) for the future cascade queries
cm-db (dynamic sqlx::query so the existing get_team's compile-time
cache doesn't need regenerating):
- TeamRuntimeConfig struct
- get_team_runtime_config / set_team_runtime_config
- team_for_loop / team_for_research_topic (resolvers)
- set_team_for_loop / set_team_for_research_topic (binders)
cm-api
- GET /api/teams/{id} now surfaces risk_profile + mcp_bundles
- PATCH /api/teams/{id}/runtime-config sets them
Not touched (comes in follow-up slices):
- Wizard picker exposing 'reuse research team' vs 'fresh coding team'
- Runtime container spawn keyed on team_id
- Migration of existing paired coding loops onto their own team
|
||
|
Omar Sobh
|
ae113dd319 |
live-run-logs: add Container tab that tails filtered daemon logs
Steps summaries only fire AFTER each topology step completes — so a
stalled first turn was completely dark. Add a second tab that
streams the team runtime container's daemon log live via a new SSE
endpoint.
Backend
- GET /api/topology-runs/:id/container-log — workspace-scoped SSE
around bollard's docker.logs(follow=true, tail=200). Buffers on
newline so partial mux chunks don't truncate a log line.
- compact_container_log: parse a zeroclaw daemon line
('[actor] ... zc_action=X zc_outcome=Y ... msg') into
'[actor] action (outcome) · msg'. Framing-only continuations are
dropped; non-zc lines (bash echoes, backtraces) pass through as-is
so nothing interesting is lost. ANSI escapes stripped.
- Everything funnels through one async_stream! so early exits
(workspace check / docker connect / no bound topic) yield an
'error' event and return without breaking Sse::new's single stream
type.
Frontend
- LiveRunLogs gets a sub-tabs strip: Steps · Container.
- New useContainerLog(runId, active) hook — gated by tab so we don't
hold two open SSE streams when the operator isn't looking.
- Same terminal widget renders each container line with a level
color (info/done grey, line default, error red). Sub-tab pill
shows count + status live.
|
||
 osobh
|
1d69866bcf | research canvas: collapsible sidebar + live topology-run logs (#6) | ||
|
osobh
|
149ad992bb | wizard: materialize picked repo across clawstor fleet at step-2-next (#5) | ||
|
Omar Sobh
|
212e68f6b1 |
research/probe: end-to-end smoke test endpoint (skip topics/loops/spawn)
New: POST /api/research/probe — one-shot pipeline probe against the
workspace's shared ZeroClaw gateway. Drives a trivial 'reply OK'
turn via ZeroClawDriveExecutor::drive and reports per-step timings.
Purpose: stop guessing what's broken in the wizard flow by exercising
JUST the executor→daemon→claude→response path. If probe succeeds
within a few seconds, we know:
- ZEROCLAW_TOKEN + gateway URL config is correct
- Daemon can reach and authenticate against claude
- The full ws round-trip works
…and every other failure we've been chasing (turn timed out, LLM
request failed, ws connect DNS error, etc.) is spawn-config or
prompt-size specific.
Request body (both optional):
{ "prompt": "Reply OK", "agent": "coordinator" }
Response:
{
"verdict": "ok" | "fail",
"total_duration_ms": …,
"prompt_len": …,
"response_preview": "OK",
"steps": [
{ name: "build_executor", duration_ms: …, status: "ok" },
{ name: "drive_turn", duration_ms: …, status: "ok", detail: "tokens=N, output_len=M" }
]
}
Wire-up:
- new routes/probe.rs
- pub mod probe in routes/mod.rs
- POST /api/research/probe registered in lib.rs router
- ZeroClawDriveExecutor::drive promoted from private to pub so the
probe handler can call it (behavior unchanged, other callers were
all inside the same struct).
Usage from anywhere (curl, browser dev tools, etc.):
curl -X POST https://clawmates.work/api/research/probe \
-H "Content-Type: application/json" \
-H "Cookie: <session cookie>" \
-d '{}'
Follow-up: a small frontend button (e.g. bottom of ResearchList) that
POSTs this and renders the response inline, so users don't need to
curl. Skipping in this commit to ship the useful part first.
|
||
|
Omar Sobh
|
a34be33261 |
loops: N/M INTs progress pill on source-bound loop cards
Users couldn't see how a research-bound loop was progressing through
its integration plan — the consumed_int_ids state existed in the DB
but nothing surfaced it. Now each loop card in the sidebar shows a
cyan pill "3/8 INTs" + source topic title + a thin progress bar, when
the loop is bound to a research topic.
Backend:
- GET /api/loops/progress — bulk read for every loop with a source
topic. Returns {loop_id, source_topic_id, source_topic_title,
source_outcome_version, consumed_count, total_int_count,
current_int_index} per loop. Standalone loops are omitted.
- count_int_ids parses unique INT-<number> ids out of the source
outcome's markdown — same permissive matcher as the completion
hook, so what the pill counts matches what the loop can advance.
- Memoized by topic_id inside the endpoint so N loops sharing 1
source topic only fetch the outcome once.
Frontend:
- listLoopProgress helper + LoopProgress type in the loops API.
- LoopsList fetches loops + progress in parallel on mount.
- Each card looks up progress by loop_id and, when found, renders
under the schedule line: pill "3/8 INTs · Topic title" with a
3px cyan progress bar. Title hover shows artifact version.
Follow-ups:
- Refresh button on the card — re-snapshot artifact into
task_template (cosmetic; the enqueue path already reads latest).
- Reorder rationale extraction — parse "REORDER: <text>" out of run
output, index as a per-loop event log for a mini-timeline UI.
|
||
|
Omar Sobh
|
3bcd18bf50 |
research: split pipeline diagnostics into research_pipeline.rs
Gates step failed on run#211 because research.rs grew to 1313 lines (over the 1250 budget) after adding the pipeline_state handler in the previous commit. Move the diagnostic into its own module — pure extraction, no behavior change. research.rs: 1313 → 1126 lines research_pipeline.rs: new, 187 lines lib.rs: route now points at routes::research_pipeline::pipeline_state |
||
|
Omar Sobh
|
cca6e2be20 |
research: pipeline diagnostics + refuse publish without outcome
Two fixes surfaced by the first prod run of the pipeline:
1) Silent skip-to-published bug (R1 gap):
approve_publish transitioned reviewing → publishing → published
without checking that an outcome existed. Result: pipeline could
fail silently (LLM auth error, network, etc), no outcome would be
written, but state advanced to 'published' and the download endpoint
returned 404 with no user-visible error. Now refuses with 409
Conflict when no outcome exists so the frontend can surface WHY.
2) No end-to-end visibility:
Users had no way to see where a run failed until they clicked
Download and got nothing. Adds
GET /api/research/:id/pipeline-state — a read-only per-stage report
walking:
- staffing (agents assigned)
- repo (bound + cloned)
- container (per-topic team runtime spawned)
- runs (count + failed count + latest error text)
- outcomes (count — the artifact rows get_artifact reads)
- approval (pending flag)
Each stage returns ok / warn / fail / skip plus optional detail text
so the failure reason surfaces at the diagnostic level.
Frontend:
ResearchCanvas shows a compact PIPELINE strip below the topic title,
green/amber/red dots per stage, click to expand a full checklist
with per-stage detail (including the LLM error from the last run
attempt). Polls every 6s while the topic is processing/publishing.
Follow-up:
- Root cause of the specific failure just observed: Claude CLI in
the clawmates-runtime container isn't authenticated. Deploy-side
config sweep (CLAUDE_CODE_OAUTH_TOKEN or ANTHROPIC_API_KEY into
the runtime image env), not a code fix.
- Structured event stream on top of run_events for real per-step
replay in the diagnostic panel.
|
||
|
Omar Sobh
|
f9e8d8d779 |
research: publishing → published + artifact download (R1)
Before this commit, approve_publish left topics stuck in 'publishing' forever — the sidebar 'published' bucket was always empty and nothing surfaced the artifact. Two-part fix: State machine — approve_publish now transitions reviewing → publishing → published in one API call. Real async packaging isn't a thing yet because the artifact IS the markdown already written to research_outcomes when the last run completed (topology_worker). The intermediate 'publishing' state is preserved (schema-level trigger stamps published_at on landing there) so we keep the option to detour through it later for pdf render / mirror-to-store / etc. Download endpoint — GET /api/research/:id/artifact returns the latest outcome as text/markdown with Content-Disposition: attachment. Filename sanitizes the topic title to ascii-alnum + dash and appends the outcome version so accumulated revision drafts (post R2) don't clobber. Workspace ownership check via research_topics::get; 404 if no outcome yet (reviewers browsing before a run completes). Frontend — ResearchList shows a green Download icon-button next to Delete when status === 'published'. Clicks trigger a plain anchor download of the .md — no JS blob dance needed since the response is already an attachment. Follow-up queued: pdf render (server-side or client-side of the md), mirror-to-store (S3-ish or Obsidian vault) as an async step during the publishing→published detour. |
||
|
Omar Sobh
|
acd2a0f287 |
structure polish: post-reify nav + ensure-chain + TeamWizard auto-parent
Two small quality-of-life fixes on top of the reify commit:
Post-reify navigation
OrphanMigrationDialog already returned team_id in its result;
Dashboard now pushes /?team=<team_id> before router.refresh() so
the user lands on the freshly-materialized team and sees exactly
where their agents just moved. Previously they had to hunt for it
in the newly-rebuilt sidebar.
Wizard auto-materialize (POST /api/structure/ensure-chain)
cm-db: ensure_chain(pool, ws, fallback_org, fallback_company) —
fast path returns coordinates of the first org+company already
bound in this workspace (workspace's oldest org, oldest company
under it). Slow path inserts a new org+company with the
fallback names ("My Workspace" / "General") + binds them via
org_companies. Returns { org_id, company_id, created }. Small
txn — leaves the workspace consistent whether it was already
wired or not.
cm-api: POST /api/structure/ensure-chain accepts optional
fallback_org_name and fallback_company_name in the body (trimmed,
else default). Returns the ids.
CreateTeamRequest gains an optional attach_to_company_id. When
set, after build_team() completes, we look up the company
(workspace ownership check enforced by companies::get), count
its existing teams for a stable n_i node id, and insert a
company_teams binding — so the team lands under the parent
atomically instead of a follow-up round-trip.
TeamWizard now calls ensure-chain before POST /api/teams and
passes the returned company_id in attach_to_company_id. Both
calls are best-effort — if ensure-chain fails (network etc.)
we still try to create the team, and the migration dialog stays
available as the fallback UX. Wizard flow now: fresh workspace's
first team is fully wired from the moment it appears in the
tree — no synthetic "My Workspace" scaffolding ever gets
rendered around it.
The Team/Company create paths not touched here (create_team_from_claws,
company create, org create, MasterPlannerModal scaffold) still
work as before — they just won't auto-parent yet. Later commits
can wire them the same way.
|
||
|
Omar Sobh
|
8b789beec0 |
structure: reify-orphans endpoint + "give these a home" dialog
Turns the four synthetic tree containers into a real migration path.
Clicking any of them ("My Workspace", "Teams", "Direct",
"Ungrouped") opens a dialog that creates a real
org → company → team chain and re-parents every orphan into it, all
in one DB transaction.
cm-db (new module structure_reify)
- orphan_agents / orphan_teams / orphan_companies: workspace-scoped
SELECTs of entities without a parent binding in team_members /
company_teams / org_companies. Used both by the dialog's counter
and internally by the migration.
- count_orphans: cheap combined-count via three subqueries in a
single SELECT so the dialog only round-trips once for the header.
- reify_orphans(pool, ws, org_name, company_name, team_name):
1. begins a tx
2. inserts a new org + company + team (all `flat`, empty graphs
— user can shape them later via the existing PATCH endpoints)
3. binds company under org (org_companies "n0")
4. binds team under company (company_teams "n0")
5. inserts team_members rows for every orphan agent (n1, n2, …)
6. inserts company_teams rows for every orphan team
7. inserts org_companies rows for every orphan company
8. commits, returns the created ids + moved counts
cm-api (routes/structure)
- GET /api/structure/orphan-counts → { agents, teams, companies }
- POST /api/structure/reify-orphans → { org_id, company_id, team_id,
moved_* }. Trims + rejects any empty name; validates before
starting the transaction so a 400 never rolls anything back.
Frontend
- New OrphanMigrationDialog: fetches counts on open, three name
fields (defaults: Organization "My Workspace", Company "General",
Team "Everyone"), POSTs on save. "Nothing to migrate" state
disables the save button when the workspace is already fully
wired. Copy explicitly notes that everything is renameable in the
sidebar afterward.
- Dashboard: onTreeSelect now branches on SYNTHETIC_TREE_IDS —
clicking a synthetic node opens the dialog instead of falling
through to the (nonexistent) selection. On successful reify,
router.refresh() so the sidebar + world viz reflect the new real
chain.
What this doesn't do yet (next commit)
- Wizard auto-materialize: when creating a team/company via wizard,
auto-create parent placeholders if they don't exist. Deferred so
this commit stays focused.
|
||
|
Omar Sobh
|
99e5207e69 |
sidebar: click-to-rename org/company/team + strip synthetics from world viz
Two related pieces of the "kill My Workspace" cleanup, landed
together because they share the same file:
Backend
- Three tiny inline-rename endpoints:
PATCH /api/orgs/{id}/name
PATCH /api/companies/{id}/name
PATCH /api/teams/{id}/name
Each takes { name: string }, trims + rejects empty, returns 204.
Backed by rename_org / rename_company / rename_team in cm-db —
single-row UPDATEs scoped to the caller's workspace, NotFound if
the id isn't visible.
- Registered next to the existing PATCH /:id (topology) routes so
they don't collide.
Frontend
- StructureTree accepts an optional onRename and canRename.
TreeRow: click on the label text of a renamable node → the span
becomes an <input>, focus + select-all, save on Enter or blur,
cancel on Escape. The rest of the row (row chevron / row body)
still navigates + selects as before, so single-click behaviour
is preserved for everything except the name text itself.
react-hooks/set-state-in-effect avoided by resetting the draft
in the enterEdit() click handler instead of inside a useEffect.
- Dashboard passes canRename={item.level !== "claw" && !synthetic}
(claws don't have a rename endpoint yet; synthetic scaffolding
gets reified into real rows in the next commit — the wizard
auto-materialize + orphan-migration dialog).
onRename fires the corresponding PATCH and calls router.refresh()
so the label lands in every consumer of the tree.
- World viz seed: new stripSynthetics(roots) helper walks the tree
and lifts children of any synthetic container up to their
grandparent's level. worldCanvasRoots feeds through this before
narrowRoots(). Result: the Live viz no longer shows "My Workspace"
or "Teams" nodes — real agents orbit the world root directly
(which is what you were asking for). Sidebar tree still shows
them so orphaned agents remain visible until the migration lands.
|
||
|
Omar Sobh
|
21ac35c8d4 |
research: spawn per-topic ZeroClaw team container on start (commit 1/3)
Commit 1 of the path-B (real per-topic isolation) plan. The
container spawns and its coordinates persist — nothing talks to
it yet; commit 2 wires ZeroClawDriveExecutor to prefer the topic's
URL when populated. This split keeps each landing verifiable.
Backend
- Migration 0038: research_topics gets zeroclaw_container_name +
zeroclaw_gateway_url columns. Both nullable so a topic can exist
before a spawn and teardown just NULLs them out.
- cm-db: ResearchTopic struct extended; get/list SELECTs updated;
new set_zeroclaw_container(id, workspace_id, name, url) helper
used both for spawn (Some/Some) and teardown (None/None).
- cm-api: bollard added as a workspace dep (matches cm-sandbox's
version). New research_container module:
· connect() → uses DOCKER_HOST when set (prod's socket-proxy
at tcp://socket-proxy:2375) else the local socket. Same
pattern cm-sandbox already uses.
· container_name_for(topic_id) → "research-<uuid>-team"
(deterministic so a re-start reattaches to the same
container instead of orphaning it).
· inherited_env() → propagates ZEROCLAW_*, OPENAI_*,
ANTHROPIC_*, GEMINI_*, GROQ_* from the parent server env
(provider config + tokens), stripping the server's own
ZEROCLAW_GATEWAY_URL/WORKSPACE so the team runtime doesn't
loop back on itself. Appends ZEROCLAW_GATEWAY_PORT=42617
and ZEROCLAW_WORKSPACE=/zeroclaw-data/workspace for the
team's own listener.
· spawn(docker, topic_id, repo_host_path, state_host_path):
- inspect: if the container already exists, start it if
stopped and return its coordinates (idempotent restart).
- else create with:
image = CLAWMATES_RESEARCH_TEAM_IMAGE or
clawmates-runtime:latest
cmd = [daemon, --host, 0.0.0.0]
env = inherited_env()
mounts = repo_host_path → /workspace/repo (rw)
state_host_path → /zeroclaw-data (rw)
network = CLAWMATES_RESEARCH_TEAM_NETWORK or
clawmates_core
labels = clawmates.role=research-team,
clawmates.research.topic_id=<uuid>
- creates state_host_path first so bind doesn't ENOENT.
· stop(docker, name) → stop + remove. Idempotent on 404/304.
- start_topic wires spawn after the clone completes:
· state root = CLAWMATES_RESEARCH_WORKSPACE_ROOT / <topic> /
state
· on success, persists (name, url) on the topic row so commit
2 can look them up when constructing the executor
· every failure (docker connect, docker create/start, DB
persist) is best-effort: logs and continues. A missing team
container leaves the topic pointing at the workspace-wide
gateway URL (env), preserving prior behavior.
Deploy prerequisites (not in this commit)
- The compose stack's clawmates_server service needs bind-mounts
of CLAWMATES_RESEARCH_WORKSPACE_ROOT (e.g.
/var/lib/clawmates-research:/var/lib/clawmates-research) so
paths the server writes to are visible on the host and the
spawned team container mounts the same underlying data.
- socket-proxy's ACL must allow POST + DELETE on /containers
(already the case in prod per the audited compose file).
|
||
|
Omar Sobh
|
316cdbf929 |
research sidebar: delete-with-confirm per topic
Mirror the row-level delete affordance the loops sidebar already
has. Loops was wired earlier; research had a bare title-only card
with no way to remove a stale topic.
- cm-db: research_topics::delete cascades via existing FK rules
(research_topic_agents, research_publish_approvals, and the new
research_outcomes all CASCADE on topic_id; topology_runs's
research_topic_id back-ref is SET NULL so historical runs stay).
- cm-api: DELETE /api/research/{id} → 204. Idempotent.
- Frontend: deleteTopic helper. ResearchList row is now a card
with the existing title/status/outcome header plus a trash icon
that flips the card into an inline "Delete topic + all outcomes?"
confirm strip. Confirm → red Delete / gray Cancel. If the
deleted topic was selected, selection clears; local counter
bumps the list refetch without waiting on a parent.
|
||
|
Omar Sobh
|
637e1bdd69 |
repos: sidebar actions (sync/edit/remove) + edit modal
Sidebar: - Each connection header now has three inline icon buttons: Sync now (spins while in flight), Edit (opens the modal), Remove (opens an inline confirm strip). Removes cascade repos via ON DELETE CASCADE. - The connection's last_sync_error surfaces as a red inline banner under the header — no more 'error status with nowhere to see why'. - Sync is POST /api/repos/connections/:id/sync (already existed); after either sync or delete the sidebar re-fetches so state stays consistent. Edit modal (RepoConnectionEditModal): - Loads GET /api/repos/connections/:id, pre-fills owner/base_url/label - PATCHes only the fields that actually changed; empty string on a Some(&str) field sends explicit null so the backend clears it - Sync-now + Remove reachable from inside the modal too - Rotating the token is out of scope: the modal says as much and points the user at delete + re-create through the wizard (the broker doesn't expose an update path, and rotating in place would require duplicating the whole broker->store_secret flow here) Backend: - GET /api/repos/connections/:id — same ConnectionSummary shape - PATCH /api/repos/connections/:id — owner/base_url use Option<Option<T>> double-nesting so 'omit = leave alone' and 'null = clear' round-trip distinctly through serde - repo_connections::update with COALESCE-per-field so the SQL matches the double-Option semantics without an OR-chain per field |