6e5ccc25a6a28dbc703c26a5bb6bfcbf0c79d170
93
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar SobhandClaude Opus 5
|
716ee9a304 |
feat(missions): capture a coding phase's diff to durable storage
First half of mission delivery: the work is captured before anything is published. A coding mission has until now produced nothing durable — the checkout is deleted thirty minutes after completion and `register_artifact` had no callers at all, so the only surviving output was an LLM narrative of what the agents said they did. `capture_phase_diff` writes `diff.patch`, `diffstat.txt` and `delivery.json` under `<missions_root>/_outputs/<mission>/<phase>/` and registers a `code_diff` artifact. That directory is a *sibling* of the per-mission directories the sweeper removes, and outside every bind mount handed to a container — so teardown cannot take the record with it and agents cannot edit their own evidence. Three details that decide whether this works at all: - `git add --intent-to-add` before diffing. Untracked files are invisible to `git diff`, and a phase that only *creates* files is the likeliest shape for generated code — silently capturing an empty patch would be the worst possible failure. The index is reset afterwards so capture leaves the tree exactly as the agents left it, which the test asserts. - Build output is excluded by pathspec (`target`, `node_modules`, `.venv`, …). A phase that ran `cargo build` leaves a directory larger than the repo. - An empty diff is still an artifact, flagged `empty: true`. "This coding phase wrote no code" is currently invisible to an operator and is worth saying out loud. `RegisterArtifact` gains `metadata`, which the column has had since 0047 and nothing ever wrote; the diffstat and base sha go there. No migration needed — `kind` is unconstrained TEXT and the column already exists. Tests run against a real `git init` repo rather than a mock: every bug in this area so far came from git behaving differently than assumed, and a fake git would have agreed with the assumption. `capture_phase_diff_at` takes explicit paths so parallel tests cannot race through the process-global CLAWMATES_MISSIONS_ROOT — the first version of these tests did exactly that and two of four failed non-deterministically. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
2c7d619cf0 |
fix(scheduler): a firing could be lost between rescheduling and dispatch
`tick` advanced `next_run_at` before dispatching the work, with nothing
recording that the occurrence was owed. A process that died between the two
dropped it silently.
The window is narrower than it first looks — `claim_due` sets `last_run_at`
but does not clear `next_run_at`, so a crash *before* `set_next_run` leaves
the routine due and it re-fires on the next tick. The loss is specifically
between the reschedule and the dispatch. That is tolerable for a message
routine and not tolerable for a scheduled mission, which is why this lands
before mission scheduling does.
`routine_fires` holds one row per (routine, occurrence), claimed before
dispatch and settled after:
- Fresh — nobody has it; fire.
- Retry — claimed, never settled: a crash mid-fire. Safe to fire again, as
no completion was recorded and nothing downstream saw a result.
- Settled — already dispatched; advance the clock and do not run the work.
This is what keeps a scheduled mission to one container across
restarts.
A failed dispatch settles terminally rather than staying retryable. Retrying
a persistently failing action every tick is how a broken routine becomes a
denial-of-service against whatever it talks to; the error is kept on the row.
The claim uses `xmax = 0` to distinguish a real insert from a no-op update in
a single statement — `ON CONFLICT DO NOTHING` returns no row at all, so two
schedulers racing one occurrence could both read it as unclaimed.
Also: fan-out capped at 25 per tick with the remainder logged and deferred (a
clock jump or an accidental every-minute cron would otherwise dispatch every
missed occurrence at once — one container each for topology routines), and
`spawn` no longer discards tick errors, so a scheduler that has stopped firing
no longer looks identical to one with nothing to do.
The pre-existing exactly-once test still passes: the claim changes
recoverability, not firing semantics.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
3b943df3c2 |
fix(templates): a template stopped accepting edits once it minted an agent
`upsert_builtin` replaced the role set with DELETE + reinsert. That looks
equivalent to an upsert and is not: `agent_template_link` carries a plain FK
on (template_id, role_slot), so the delete is rejected as soon as one agent
has been minted from the template, rolling back the whole transaction.
The failure mode was silent and self-targeting. The loader logs the error and
continues, so the on-disk TOML and the DB drifted apart — and only for the
templates someone had actually used. Running the smoke mission against
insight_research is what put it on the boot log:
failed to load insight_research.toml: violates foreign key constraint
"agent_template_link_template_id_role_slot_fkey"
which also means that template never received the skill-name fix.
- Upsert each role in place via ON CONFLICT (template_id, slot), the table's
primary key.
- Prune only slots the TOML dropped, and skip a slot still referenced by a
live agent with a log line. Keeping one stale role row is a smaller failure
than discarding every edit to the template.
- Regression test drives the real sequence — upsert, mint an agent, link it,
upsert again — and asserts both the prompt and skill edits land. Verified to
fail without the fix with the same 23503 the server logged.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
ca45597c79 |
feat(credentials): make provider substitution and runtime auth mode visible
Three guardrails around which credential pays for what. 1. Boot announces the mission-runtime auth mode, and warns when subscription auth is configured on a deployment with more than one user. A consumer subscription credential may only run the account holder's own work, and that condition is otherwise invisible -- it holds today and quietly stops holding the first time someone else signs up. Adds users::count_all (dynamic query, so the offline cache needs no regeneration). 2. Reject an ANTHROPIC_API_KEY shaped like a subscription OAuth token (sk-ant-oat...) at boot rather than failing on the first model call far from the mistake. Both credentials start sk-ant-, so the confusion is easy to make and hard to spot. 3. provider_alias_for's GLM/Kimi -> anthropic.default fallback was documented as deliberate but was silent in effect: a user picking "kimi" in the UI got an agent spending the Anthropic key, with nothing saying so. It now logs the substitution, and is_exact_provider_match() lets callers tell a real family match from a substitution so a UI can say which model will actually run. Behaviour is unchanged -- only the silence is. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 5
|
fe57ce4ed1 |
feat(missions): surface goal conditions and per-pass verdicts in the UI
Makes the completion evaluator usable and observable.
- GET /api/missions/{id}/phases/{phase_id}/evaluations returns every verdict
for a phase, newest pass first, scoped like the summary endpoint.
- MissionPhase gains done_when / max_iterations / iteration, so the phase card
can show what the phase is working toward and which pass it is on.
- PhaseStatus gains 'evaluating' (amber) -- the state between "runs finished"
and "phase done" that only conditioned phases enter.
- New PhaseGoalStrip renders on the phase card, and renders NOTHING for phases
without a condition so unconditioned missions look exactly as before. It
polls only while the phase is running or being judged.
- Mission wizard step 2 gains the condition + a max-passes field.
Two deliberate emphases in the UI:
The evaluator's `reason` is the most prominent element, because it is both the
explanation of why a phase iterated and the literal text handed back to the
agents as guidance -- it is what tells an operator whether the condition is
written well.
The hint copy states the constraint that actually governs whether a condition
works: the judge cannot run commands, it only reads what the agents wrote, so
the condition has to be provable from their output. "cargo test reported 0
failures" works; "the code is well factored" does not. Getting this wrong is
the difference between a phase that converges and one that burns every pass.
An evaluator error is rendered distinctly from a negative verdict, so a judge
outage doesn't read as a judgement on the work.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
f848248fac |
feat(missions): goal conditions and phase iteration, judged on the subscription model
A phase used to complete when its topology_runs reached a terminal state --
purely structural. It marked itself done whether the agents produced the
artifact or wrote nothing at all, and it ran exactly once: execute_resumable's
skip(start) is resume, not repeat, and the only re-run path was a human
hitting the retry endpoint.
A phase can now carry `done_when`, a completion condition judged after each
pass against the evidence the agents actually surfaced. Not met and passes
remain -> the phase goes back to pending with iteration bumped, and the
verdict's reason is appended to the next pass's task text. That feedback is
what makes iteration converge rather than repeat -- the same mechanism /goal
uses, and that swarm.rs already uses for rejected work.
The evaluator runs on the SUBSCRIPTION model. CLAWMATES_EVALUATOR_MODEL
defaults to judge_model(), and a `runtime:<alias>` spec routes through
ZeroClawDriveExecutor -- a container agent on claude_cli, i.e. Claude Code on
the OAuth subscription, needing no platform API key. Same routing the door
governor uses.
Two deliberate departures from the governor's contract, both required:
- FAIL-CLOSED. Runtime::judge is fail-open and reads a verdict by
!contains("DENY"), so a model explaining why it *would* deny reads as
approval and an empty reply reads as approval. For completion that is
backwards: unsure must mean not done. The contract is swarm.rs's strict
JSON {"met","reason"} with .unwrap_or(false). Six tests cover the closed
paths -- prose, empty, missing field, non-boolean, transport error.
- judge_raw returns the raw reply; judge collapses to a bool too early to
carry a structured verdict.
Iteration scoping is the subtle part and has its own test: on pass 2 the
phase's own iteration is 1 but pass 1's completed run is still in the table,
so "are this phase's runs all finished?" must ask about the CURRENT pass or
that stale row closes out pass 2 the instant it is enqueued.
Evidence comes from phase_summarizer::collect_evidence, extracted from the
existing collect_material so the evaluator and the summary card cannot
disagree about what a phase produced.
done_when/max_iterations are promoted from phase config into columns (the
sweep filters on them every tick) and max_iterations is clamped to 20 at
insert -- the UI limits it too, but a runaway loop must not be one crafted
request away.
A phase with no condition completes exactly as before; that regression guard
is the first test in the file.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 5
|
0785ac9c79 |
feat(missions): document reader + three-tab IA for the mission page
The mission page made its own output unreadable. Reviewing a research
brief meant scrolling a 300px <pre> nested inside a 260px run box nested
inside the page scroller (plus a 4th scroll region for the description) —
and the text was capped at 6,000 chars server-side with no way to fetch
the rest, so a 53kB brief showed ~11% of itself and silently dropped the
remainder. Eight flat tabs (overview/phases/tasks/team/live/artifacts/
benchmarks/pane) mixed lifecycle, work items, people, telemetry, outputs
and infra at one level, so nothing indicated where the deliverable lived.
Reader:
- GET /api/missions/{id}/documents lists every agent output (titles +
sizes, no bodies); GET .../documents/{run_id}/{index} returns one in
full. Scoped to the mission so a run id from elsewhere can't be read.
- MissionOutputReader: rail (documents grouped by phase) · document ·
outline (headings, click to jump). Exactly one scroll container per
column, never nested. Copy + download .md.
- MarkdownBlock gains fenced code blocks (agent output is full of ```rust,
previously mangled into paragraphs), h4-h6, heading anchors, and an
outlineOf() helper.
Information architecture:
- Three primary tabs with shallow sub-views: RUN (phases/tasks/live) ·
OUTPUT (documents/artifacts/benchmarks) · SETUP (overview/team/pane).
- PhaseRunsList shows a short excerpt with no inner scrollbar and points
at the reader for the full text.
- The header description is clipped, not scrollable; its full text now
has a home in Setup → Overview.
Missions list:
- /api/missions returns MissionListItem — Mission flattened plus
phases_total/phases_done/current_phase, so the JSON stays a strict
superset. Cards render a progress bar and "Coding · 1/2" instead of a
bare status dot.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
||
|
Omar Sobh
|
37f3f5abfd | fix: mission_runtime_pairing_code in single-row mapping + fmt | ||
|
Omar Sobh
|
b569688e04 |
fix(mission_runtime): per-mission auto-pair via container log scrape (C3 auth)
The seed-mount approach didnt work: even with the shared runtimes data dir bind-mounted, a fresh gateway instance mints a new pairing key and requires re-pairing. The topology_worker connect returned 401 forever. New approach — per-mission gateways self-pair: - Provisioner tails container logs after start, extracts the X-Pairing-Code from the boot banner - Persists it on missions.runtime_pairing_code (migration 0059) - topology_worker constructs ZeroClawDriveExecutor with THAT code via from_env_for_gateway_with_code, which triggers the lazy /pair handshake on first turn and caches the returned bearer Drops the shared-runtime data-dir mount — each per-mission gateway now owns its own state, restoring the C3 isolation guarantee. |
||
|
Omar Sobh
|
5d24fd3460 |
missions: schema + provisioner skeleton for per-mission runtime containers (C3 slice 1)
- migration 0058: adds missions.runtime_container_name + runtime_endpoint
- new mission_runtime module (bollard): ensure_container /
teardown_container. Container is spawned on clawmates_core +
clawmates_edge networks with just /var/lib/clawmates-missions/{id}
bind-mounted so agents scoped to /mission/repo can only see this
missions repo.
- provider API keys forwarded from the server envs so per-mission
runtimes inherit them.
- Mission struct + repo helpers updated for the two new columns +
set_runtime_binding().
- Unit tests cover container naming determinism + entropy.
Not wired to the orchestrator yet — that lands in slice 2.
|
||
|
Omar Sobh
|
f5bba67e38 |
missions: surface per-phase run errors on the phase card
Adds inline failure debugging to the Phases tab. When you see a
phase card marked FAILED, click the collapsed error summary and the
full topology_run.error text expands under it — the exact stack
trace / provider error / whatever the worker recorded.
Backend:
- TopologyRunSummary gains mission_phase_id + team_id + error
fields. list_by_mission SELECT extended; other constructor
(list_recent) explicitly passes None for the new fields.
- GET /api/missions/{id}/runs response now carries all of the
above so the frontend can attribute failures per phase.
Frontend:
- MissionRunSummary type mirrors backend additions.
- MissionCanvas fetches runs alongside mission on load +
auto-refresh; indexes by mission_phase_id in a memoized Map.
- Each phase card renders a per-run row: colored status pill
(running / completed / failed), short run id, finished_at
timestamp. For failed runs, a <details> collapses the error
text — first line as summary, full 4kB in a monospace <pre> on
expand.
Directly unblocks the "phase says Failed but there's no info to
debug" report. Both research and coding phases get this — the code
path is phase-kind-agnostic.
|
||
|
Omar Sobh
|
f0dd0147f6 |
templates: 5 research team templates + category filtering
Adds the operator's five categorized research team archetypes:
1. codebase_research — code archeologist, architecture mapper,
flow tracer, vault scribe. Produces Obsidian vault entries
under Codebases/<repo>/ that make future missions faster.
2. papers_research — domain scout, paper reader, library curator.
Pulls arXiv / Semantic Scholar / conference proceedings, keeps
a structured local library under Papers/<topic>/.
3. insight_research — implementation tracker, novelty hunter,
publication drafter. Bidirectional loop that spots
publication-worthy novelty in our own implementations of
external papers.
4. continuous_research — signal harvester, ranker, digest writer.
Standing sweep of RSS + arXiv daily + GitHub trending; produces
a rolling ContinuousResearch/<date>/digest.md.
5. continuous_improvement — brain inspector, improvement proposer,
improvement evaluator. Standing self-audit that files level-up
proposals for the operator to review + measures the outcome.
Each template ships with role system_prompts + brain_seeds authored
in the same voice as the existing backend/frontend/etc templates —
evidence-first, redlines called out, no invention.
Schema + code:
- 0057_team_templates_category.sql — new column with
CHECK (research | development | security | ops). Existing rows
default to 'development'.
- team_templates::UpsertBuiltin + TeamTemplate carry category
(with default_category = 'development' fallback for
Serialize/Deserialize compatibility).
- team_template_loader reads `category = "..."` from the TOML;
absent defaults to 'development' so old templates keep working.
- Wizard step 3 filters:
Research teams panel → templates.filter(t.category==='research')
Development teams panel → templates.filter(t.category==='development')
Operator can no longer accidentally pick backend as their
"research team".
Test fixture updated with category="development".
The templates ship in the server image via the existing
`COPY templates /etc/clawmates/templates` line — no Dockerfile
change needed.
|
||
|
Omar Sobh
|
3ba0485e7d |
mission progress UI: auto-refresh + Team tab + Live events tab
Fills the biggest UX gap surfaced during the deploy walk: hosted
missions had no live-progress surface at all. Now they do.
Auto-refresh:
- MissionCanvas grows a second useEffect that polls getMission
every 3s while mission.status === 'running'. Stops immediately
on terminal state (completed / failed / cancelled). Phases,
Tasks, Artifacts, Benchmarks all update without a manual click.
Team tab (new):
- MissionTeamTab.tsx — fetches /api/teams/{id} + /api/team/claws,
shows a card per member with role slot + an "Open" pill that
calls onOpenClaw(clawId) → Dashboard flips to AGENT tier with
that claw selected, dropping the operator into the existing
ClawCommandCenter surface (WorkingOnNow, ReasoningStream, etc).
Live events tab (new):
- MissionLiveEvents.tsx — polls /api/missions/{id}/runs every 5s
for the topology_runs bound to this mission, opens one
EventSource per active run against /api/topology-runs/{id}/events,
renders as a chronological scrolling feed with per-event kind
pills + per-run short-id badges. Auto-scrolls unless the
operator scrolled up. New runs auto-attach; terminal runs
close cleanly.
Backend:
- cm-db::repo::topology_runs::list_by_mission — SELECT ... FROM
topology_runs WHERE mission_id = $1 ORDER BY created_at DESC.
Uses runtime sqlx::query (not the macro) to avoid a sqlx cache
regen just for this route.
- TopologyRunSummary gains #[derive(Serialize)] + rfc3339 codecs.
- GET /api/missions/{id}/runs — workspace-scoped, returns
{ runs: [...] }.
Dashboard wires onOpenClaw on MissionCanvas → setAgentId + setTier("claw").
Verified: cargo check --workspace + tsc --noEmit + eslint --quiet
all green.
|
||
|
Omar Sobh
|
d8c8793c4a |
ci fixes: cargo fmt, eslint entities, max-lines split
CI on
|
||
|
Omar Sobh
|
d6dbd044c8 |
herdr phase 1a: missions runtime_kind + target_node schema
First slice of the second-runtime path. Missions now carry
runtime_kind ('zeroclaw' | 'local_herdr') + target_node_id (FK to
nodes) so the mission_orchestrator + phase executors can dispatch
differently depending on where the operator wants execution.
Migration:
- 0055_missions_runtime_kind.sql — adds runtime_kind (NOT NULL
DEFAULT 'zeroclaw' + CHECK), target_node_id (nullable FK ON
DELETE SET NULL). All existing missions backfill to 'zeroclaw'
so behavior is unchanged.
- topology_runs also grows herdr_workspace_id / herdr_tab_id /
herdr_pane_id text columns so a resumed run can reattach to the
same Herdr pane instead of spawning a duplicate.
Code:
- cm-db::repo::missions — Mission + NewMission carry the two new
fields; all SELECTs updated; INSERT COALESCE-defaults
runtime_kind to 'zeroclaw' when unspecified.
- routes::missions::create — validates runtime_kind and requires
target_node_id when kind='local_herdr' (400 otherwise).
- lib/api/missions.ts — RuntimeKind type; Mission carries both;
CreateMissionRequest optional fields.
Behavior is opt-in: no path exists yet to actually create a
local_herdr mission — that lands in Phase 1c (wizard picker). This
commit just makes the schema + validation in place so Phase 1b's
fleet_herdr dispatch module can key on it.
Tests: mission_orchestrator integration test still green.
|
||
|
Omar Sobh
|
1f0117e35a |
mission canvas: add / edit / delete toolbar controls
Top-right toolbar grows three CRUD controls per your request:
- Plus (always visible) — opens MissionWizard, selects the new
mission on create
- Pencil (draft-only) — opens EditMissionModal for title +
description; PATCHes /api/missions/{id}
- Trash (always visible) — window.confirm then DELETEs; sidebar
selection clears via new onDeleted callback
Backend:
- cm-db::repo::missions::update_meta(id, ws, title?, description?)
— COALESCE-based partial patch
- cm-db::repo::missions::delete(id, ws) — hard delete, cascades
via FKs on phases/tasks/artifacts/benchmark_snapshots
- PATCH /api/missions/{id} (draft-only) + DELETE /api/missions/{id}
Frontend:
- lib/api/missions — updateMission + deleteMission clients
- MissionCanvas — three toolbar buttons, EditMissionModal
(title + textarea for description), local wizard state
- Dashboard — passes onSelect + onDeleted so sidebar reacts to
create + delete without stale selection
Edit is draft-only (backend enforces + button hidden past draft) so
in-flight missions can't have their brief mutated out from under
running agents. Delete is unconditional — operator responsibility to
Cancel first if a run is live.
|
||
|
Omar Sobh
|
854a617777 |
task #23: retire per-team ZeroClaw container coords (Option A)
Missions never populated teams.zeroclaw_container /
teams.zeroclaw_gateway_url — those were research/loops-era columns
for long-lived per-team containers. Every mission-materialized team
runs inside the SHARED runtime as claws-as-agents provisioned via
RuntimeProvisioner. Reading zeroclaw_container on a mission row
always came up NULL, making security_scan + benchmark_runner
silently fail with "mission has no team container yet."
Changes:
- migrations/0054_drop_teams_zeroclaw_columns.sql — DROP both
columns.
- cm-db/src/repo/teams.rs — delete dead helpers
team_container_coords + set_team_container_coords.
- cm-api/src/security_scan.rs — replace team_container_for_mission
with exec_target(pool, mission_id): container from env
CLAWMATES_RUNTIME_CONTAINER (default clawmates-runtime); workdir
from env CLAWMATES_MISSIONS_ROOT + /{mission_id}/repo
(same convention pdf_renderer uses); precondition that mission
must have repo_id bound.
- cm-api/src/benchmark_runner.rs — same shape.
Follow-up (not in this commit): mission_orchestrator + compose stack
still need to wire a per-mission repo checkout under
CLAWMATES_MISSIONS_ROOT before scan/bench actually produce findings.
Columns cleanup here removes the misleading silent-fail; the
missing-checkout gap is now surfaced with a clear error.
Verified: SQLX_OFFLINE=true cargo check --workspace + cargo test
-p cm-api --test mission_orchestrator both green.
Closes task #23.
|
||
|
Omar Sobh
|
fdb8cfeecc |
slice 9 cleanup: drop legacy research/loops backend + tables
Retires the legacy research/loops backend after the missions arc
(slices 1-9) fully replaced it. Frontend cutover was 4663348; this
commit finishes the job on the backend + database.
Migration:
- 0053_drop_legacy_research_loops.sql — drops the 8 legacy tables
(research_topics, research_topic_agents, research_outcomes,
research_publish_approvals, loops, loop_agents, loop_orgs,
loop_teams) and the 3 topology_runs FK columns
(research_topic_id, loop_id, iteration). parent_run_id stays;
recursive_exec still uses it.
Files deleted (11):
- crates/cm-api/src/routes/{research,loops,research_setup,
research_pipeline,wizard_repo,probe}.rs
- crates/cm-api/src/research_container.rs
- crates/cm-db/src/repo/{research_topics,research_outcomes,
research_publish_approvals,loops}.rs
- crates/cm-runtime/src/loops.rs
- crates/cm-api/tests/research_publish_role.rs
Files edited:
- crates/cm-api/src/lib.rs — dropped 20 legacy route registrations
(all /api/research/* + /api/loops/* + /webhooks/loops + probe)
and module decls
- crates/cm-api/src/topology_worker.rs — deleted legacy dispatch
(freeze_research_outcome, advance_loop_after_completion,
continue_initial_burst, maybe_transition_research_topic,
parse_reorder_rationale, per-topic/loop gateway resolver).
reap_stuck_runs now keys on mission_id (not topic_id).
Executor path unconditionally uses ZeroClawDriveExecutor::from_env
— mission_orchestrator provisions each claw as an agent inside
the shared runtime via RuntimeProvisioner, so per-team gateway
resolution is no longer applicable.
- crates/cm-api/src/routes/topology.rs — deleted container-log SSE
endpoint (research/loop-specific), dropped loop_id filter and
iteration field from ListRunsQuery/RunSummary
- crates/cm-api/src/routes/world.rs — removed
active_research_topics/active_loops/preseed_repo_paths;
World SSE no longer emits repo:{topic}/loop:{id} landmark orbs
(follow-up task #21 tracks adding mission:{id} equivalents)
- crates/cm-api/src/runtime_provision.rs — removed now-unused
mint_workspace_service_token
- crates/cm-db/src/repo/topology_runs.rs — removed 9 legacy
helpers (research_topic_id lookup, loop_id_for_run,
iteration_for_run, active_runs_for_research_topic, etc.)
- crates/cm-db/src/repo/teams.rs — removed 4 dead helpers
(team_for_loop, team_for_research_topic + setters)
- crates/cm-api/tests/topology_jobs.rs — removed loop/topic
tests, dropped enqueue_run_with_topic helper
- crates/bins/clawmates-server/src/main.rs — removed
spawn_loop_scheduler call
- crates/cm-api/src/routes/mod.rs, crates/cm-db/src/repo/mod.rs,
crates/cm-runtime/src/lib.rs — module decls stripped
sqlx cache: regenerated against post-migration schema
(71 files changed, ~+70 / -8896 net)
Test/build: SQLX_OFFLINE=true cargo check --workspace clean;
cargo test --workspace --no-run clean.
Follow-up (task #21): World view lost the in-flight-work landmarks
when repo:{topic} / loop:{id} orbs disappeared. Add mission:{id}
orbs as the missions-era replacement.
|
||
|
Omar Sobh
|
56201a6985 |
mission canvas: add Refine button + markdown-rendered description
Adds a Refine button to the left of Refresh + Launch on the mission
detail toolbar (draft-only). Clicking it POSTs to a new endpoint that
calls Gemini 2.5 Flash to rewrite the user's freeform description into
a coherent, sectioned Markdown brief (Objective / Context / Scope /
Constraints / Acceptance Criteria / Open Questions) ready for the
research + coding agents to ingest cleanly.
Backend:
- crates/cm-api/src/mission_refiner.rs — Gemini call with a
system prompt that preserves user-provided facts, avoids
invention, and emits raw markdown (not JSON).
- POST /api/missions/{id}/refine — draft-only, 400 on empty
description or non-draft state.
- cm-db::repo::missions::set_description helper.
Frontend:
- MarkdownBlock — tiny zero-dep renderer for h1/h2/h3, bullet +
numbered lists, **bold**, `code`, paragraphs. Deliberately
small; the refiner emits a bounded subset.
- MissionCanvas — Refine button (Sparkles icon, secondary style)
to the left of Refresh; description now renders through
MarkdownBlock instead of a single <p>. Disabled while
description is empty or a refine is in flight.
- lib/api/missions — refineMission client.
|
||
|
Omar SobhandClaude Opus 4.7
|
9b5e63cbb7 |
slice 8.5: per-agent + per-team level-up endpoints
Level-up analyzes an agent's brain + recent run outcomes (or a
whole team's aggregate state), calls Gemini 2.5 Flash for structured
JSON proposals, and persists them as pending level_up_proposals
rows. Reviewer approves a subset via /apply; the applier commits
only those items.
Migration 0052 adds level_up_proposals (id, workspace_id, agent_id
XOR team_id via CHECK constraint, status, payload JSONB,
applied_items[], model, created_by, approved_by, created_at,
applied_at) + workspace/pending/agent/team indexes.
Rust surface:
- cm_db::repo::level_up::{insert, get, list_pending, mark_applied,
mark_rejected}
- cm_api::level_up::{propose_agent, propose_team, apply}
Item kinds handled by apply():
identity_refinement → UPDATE agents.system_prompt
skill_add → agent_skills_ext INSERT
skill_candidate → workspace-scoped skills INSERT
(deterministic id per (workspace, name))
brain_consolidation → set_agent_md on the brain (unlike
brain_seed::ingest, this overwrites)
roster_change / mcp_bundle_change — logged as
"not auto-applied, human runs
team-wizard" (structural changes need
human review of side effects).
API:
- POST /api/claws/{id}/level-up → { proposal_id }
- POST /api/teams/{id}/level-up → { proposal_id }
- GET /api/level-up-proposals → pending list
- GET /api/level-up-proposals/{id}
- POST /api/level-up-proposals/{id}/apply { approved_item_ids }
- POST /api/level-up-proposals/{id}/reject
Uses Gemini 2.5 Flash with response_mime_type: "application/json"
so the model returns structured JSON directly (no ```json fence
stripping needed). Configurable via CLAWMATES_LEVEL_UP_MODEL.
Follow-ups:
- Frontend diff-review UI (pick items, approve/reject)
- roster_change / mcp_bundle_change appliers (currently manual)
- Anthropic + OpenAI proposer variants
- Promote workspace-scoped skills to builtin via a curator flow
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
f843c9ddb1 |
slice 7: before/after benchmark runner
Executes a benchmark harness inside the mission's team container
and records the resulting metrics as a benchmark_snapshots row keyed
on (phase_id, iteration). Baseline pass (iteration=0) captures
before_metrics; each post-iteration call captures after_metrics +
computes delta vs baseline.
Rust surface:
- cm_db::repo::missions::upsert_benchmark_snapshot / benchmark_snapshots_for
- cm_api::benchmark_runner::{baseline, after_iteration, run}
- Harness enum: Auto | Criterion | CargoBench | VitestBench |
PytestBench | Shell (each with a command() vector)
- Auto detection peeks at the repo layout inside the container
(Cargo.toml → CargoBench, package.json → VitestBench, pyproject
→ PytestBench). Falls back to a Shell echo when nothing
identifiable.
- Bencher-format line parser extracts (name, ns_per_iter,
plusminus) so criterion + `cargo bench` output become structured
samples the canvas can diff.
- compute_delta pairs samples by name, emits {before_ns, after_ns,
delta_pct, direction: improved|regressed}.
API:
- POST /api/missions/{id}/benchmark { phase_id, slot, iteration? }
triggers baseline or after run and returns the mission's full
snapshot list.
- GET /api/missions/{id} now includes `benchmarks[]` in the detail
payload.
Frontend:
- New Benchmarks tab on MissionCanvas with iteration + driver
header, plus a 4-column grid (bench / before / after / Δ%) when
delta samples are present. Improved deltas render green,
regressions red.
- TS types + triggerBenchmark() helper in lib/api/missions.ts.
Wiring notes:
- team_container_for_mission reads teams.zeroclaw_container — that's
populated by topology_worker::try_team_gateway_url on first run,
so trigger baseline AFTER the mission's first phase spawns the
container.
- Not auto-fired yet by phase execution; that's the "template phase
executor" work that spans Slices 4-8. Manual API trigger works
today; automated hook is a follow-up.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
85a97dffca |
slice 3.5d: agent_template_link + brain seed helper + skills merge
Ships the lineage layer that ties agents back to their team template
and wires the MCP skills server to actually merge template default
skills with per-agent overrides.
Migration 0050 adds `agent_template_link` (agent_id PK, template_id,
template_version, role_slot, seeded_at, created_at + indexes for
template/role lookups). Populated at agent-materialization time by
Slice 4's mission-launch orchestrator; read here by the skills MCP
server and by future level-up (Slice 8.5).
New Rust surface:
- cm_db::repo::agent_template_link (upsert / get / mark_seeded /
agents_for_template — the last is what level-up's "prompt upgrade
on template N+1" query needs)
- cm_api::brain_seed::ingest(claw_id, seed_md, identity_prompt)
opens cm_brain::ClawBrain on spawn_blocking, sets system_prompt
on first touch, writes seed as agent_md, commits. Idempotent —
skips when agent_md already populated.
- cm_api::mcp_skills::mcp_skills tools/call now resolves the caller
agent's template + role via agent_template_link and merges
template default skills with per-agent overrides (was overrides-
only in Slice 3.5b).
- cm_api::team_template_loader now binds template_role_skills after
upserting each template — looks up each declared skill by name,
attaches with pin_in_context=true for foundation skills and the
first two role skills. Missing skills log + skip.
- Boot ordering: skills load BEFORE team templates so the binding
lookup resolves.
Follow-up (Slice 4): mission-launch orchestrator calls brain_seed::ingest
+ agent_template_link::upsert when minting a team from a template.
Until that lands, the link is populated only by manual writes; the
MCP merge is silent-no-op for agents without a link (falls through
to overrides-only), which matches the pre-3.5d behavior.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar Sobh
|
5f988ce022 | fix: legacy skills::create mirrors title into new name column | ||
|
Omar SobhandClaude Opus 4.7
|
1153d72d00 |
slice 3.5a: skills catalog — the "how to think" layer
Introduce the skills catalog: the second half of the two-layer agent
model (skills teach agents HOW to think about a problem; MCP servers
give them the ABILITY to act). Delivery via MCP resources lands in
Slice 3.5b; this slice ships the data model + API surface.
Migration 0049 extends the legacy `skills` table (from 0001_init.sql,
originally a workspace catalog of markdown snippets) with the richer
typing we need — name, when_to_use, tags, source_kind, current_version
— rather than duplicating tables. Also adds:
- skill_versions (version history for level-up promotions +
rollback; back-pointer via promoted_from
JSONB records agent_id / research artifact /
brain memory that produced it)
- template_role_skills (m2m binding skills to team-template roles
with pin_in_context + order_idx)
- agent_skills_ext (per-agent overlay: include=true adds a skill
to the bundle; include=false prunes a
template default for this specific agent)
Rust surface:
- cm_db::repo::skills_catalog with typed Skill/SkillVersion/
AgentSkillBinding structs + upsert_builtin (idempotent — bumps
version + appends to skill_versions ONLY when body changes) +
list_visible/get/get_by_name reads + template + agent binding
helpers + effective_for_agent (merges template defaults with
agent overrides, applies exclude precedence, batch-fetches skill
bodies)
- cm_api::routes::skills_catalog with:
GET /api/skills — list visible
GET /api/skills/{id} — detail
GET /api/claws/{id}/skills — effective binding (accepts
template_id + role_slot as query args to merge in template
defaults)
Follow-ups:
- Slice 3.5b: clawmates_skills MCP server exposes catalog as MCP
resources, honoring pin_in_context for auto-injection
- Slice 3.5c: seed ~40-60 builtin skills across the 6 stacks
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar Sobh
|
3f77370a6e | fix: remove unused sqlx::Row imports in team_templates | ||
|
Omar SobhandClaude Opus 4.7
|
9ba5c06a1a |
slice 3: 6 team templates seeded from TOML recipes
Team templates are the canonical rosters + tool bundles that mint
concrete teams for a mission. Every builtin ships as a TOML recipe
under templates/teams/*.toml, loaded into the DB at server boot.
Migration 0048 adds:
- team_templates (id, key, name, stack, default_topology,
risk_profile, mcp_bundles, version, source,
workspace_id)
- template_roles (m2m: template_id + slot; system_prompt,
skills[], brain_seed)
- teams gets template_id + template_version for level-up lineage
Ships 6 builtins:
- rust_sdlc — planner/coder/tester/reviewer/committer for Rust
- backend — api_designer/db_engineer/coder/tester/committer
(Postgres, DuckDB, graph DBs, wire protocols)
- frontend — designer/coder/tester/committer (React + Tailwind + ShadCN)
- mobile — designer/coder/tester/committer (Expo, RN, iOS, Android)
- gpu — arch_analyst/kernel_author/bench_engineer/coder/committer
(CUDA, Metal, ROCm from Rust)
- threejs — scene_designer/coder/shader_author/perf_engineer/
committer (three.js, WebGL, WebGPU)
Each role has a versioned system_prompt + skill list + brain_seed
markdown. Skills column is a name array today; Slice 3.5a promotes it
to a typed m2m join with the real skills catalog.
Server boot:
- team_template_loader::load_builtins reads TOML from
/etc/clawmates/templates/teams (container) or templates/teams (dev),
upserts idempotently. Deterministic uuid per template key (sha256
of a fixed namespace + key) so ids are stable across boots.
- Dockerfile copies templates/ to /etc/clawmates/templates.
Read API:
- GET /api/team-templates — list all
- GET /api/team-templates/{id} — detail with roles
Wizard:
- Step 3 rewired from a raw team_id text field to a template picker
with "LLM auto-provision" as the default option + one card per
builtin, showing stack, topology, risk profile, and description.
- Mission create now passes team_template_id (not team_id) so phase
execution knows which template to mint from.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar Sobh
|
a72da9dff0 | fmt: apply cargo fmt to missions | ||
|
Omar SobhandClaude Opus 4.7
|
fbefc67878 |
slice 1: missions data model + migration
Introduce the unified `missions` tier that will replace the current
research_topics + loops split. This slice ships the data model +
backfill + skeleton REST surface; the old wizards keep working in
parallel until Slice 9's big-bang cutover.
Migration 0047 adds:
- missions (top-level workflow: template_kind + team +
schedule + status + config)
- mission_phases (ordered {research|coding|benchmark|
security_scan} phases per mission)
- mission_tasks (typed units of work, e.g. INT-XX cards,
UPSERT-keyed on (phase_id, external_id))
- mission_artifacts (MD/PDF/benchmark/security/diff files with
a pending queue for the PDF renderer worker)
- benchmark_snapshots (before/after pairs per iteration)
Backfill copies existing research_topics + loops rows into the new
tables as one-shot missions with the appropriate template_kind, so
Slice 2's UI can render the full history immediately.
New Rust surface:
- cm_domain: MissionId, MissionPhaseId, MissionTaskId, MissionArtifactId
- cm_db::repo::missions: Mission/MissionPhase/MissionTask/
MissionArtifact structs + insert (txn-wrapped)/get/list/set_status/
phases_for/set_phase_status/upsert_task/tasks_for/register_artifact/
artifacts_for/next_pdf_pending/set_pdf_result
- cm_api::routes::missions: skeleton list/create/get/set_status
routes registered at /api/missions/*
Follow-up slices layer richer behavior (template dispatch, phase
execution, task parsing, artifact rendering) on this foundation.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.7
|
49f94a5360 |
mcp door: mint workspace-owner service session for team runtime bearer
The runtime template's static clawmates_door bearer is rejected by cm_auth::authenticate() (needs an auth_sessions row). Every per-team agent was getting `unauthorized: missing or invalid bearer token` and `0 tool(s) registered from 0 server(s)`. Add AuthService::mint_service_session + users::owner_of_workspace and mint a 30d service session in try_team_gateway_url; inject it into the freshly-spawned team container's config.toml [[mcp.servers]] clawmates Authorization header via prewrite_daemon_config_with_risk (bearer arg). Follow-up: apply the same pattern to research::spawn (per-topic) and per-loop spawn paths. Co-Authored-By: Claude Opus 4.7 <[email protected]> |
||
|
Omar Sobh
|
d687a00524 |
teams: zeroclaw container coords + coding_readwrite risk profile
Slice 3a of the per-loop-team arc — prerequisites for the runtime spawn hookup that lands in 3b: 0046 migration - ALTER TABLE teams ADD zeroclaw_container TEXT - ALTER TABLE teams ADD zeroclaw_gateway_url TEXT Both NULL until the runtime's spawn_team fn (3b) provisions the container and persists its coordinates. Mirrors the shape already on research_topics (0038) so the resolver code path can generalize. cm-db - team_container_coords / set_team_container_coords: dynamic sqlx::query() readers/writers for the new columns. Runtime template (gw-04, out-of-band edit on /var/lib/clawmates-runtime-template/config.toml + shared runtime /root/clawmates-runtime/data/.zeroclaw/config.toml) - New [risk_profiles.coding_readwrite]: adds file_write + shell on top of the research_readonly baseline. Still excludes http_request / browser / composio (egress stays behind the MCP door). Slice 3b will add spawn_team (bind-mounts paired-topic repo, uses team-scoped state dir, injects team.risk_profile into the config template) and rewire topology_worker to resolve gateway URL through team_id when the loop has one. |
||
|
Omar Sobh
|
6066e93889 |
teams: per-team runtime posture (risk_profile + mcp_bundles) + FK from loops/topics
Foundation slice for letting a coding loop bring its own team instead
of reusing the paired research topic's team. Turns out the teams
table already exists (0010_teams.sql) with full CRUD — this scales
back to the minimal missing bits:
Schema (0045_teams.sql)
- ALTER TABLE teams ADD risk_profile TEXT (NULL = template default)
- ALTER TABLE teams ADD mcp_bundles JSONB DEFAULT '[]'
- ALTER TABLE loops ADD team_id UUID REFERENCES teams ON DELETE SET NULL
- ALTER TABLE research_topics ADD team_id UUID REFERENCES teams
- Two partial indexes (team_id NOT NULL) for the future cascade queries
cm-db (dynamic sqlx::query so the existing get_team's compile-time
cache doesn't need regenerating):
- TeamRuntimeConfig struct
- get_team_runtime_config / set_team_runtime_config
- team_for_loop / team_for_research_topic (resolvers)
- set_team_for_loop / set_team_for_research_topic (binders)
cm-api
- GET /api/teams/{id} now surfaces risk_profile + mcp_bundles
- PATCH /api/teams/{id}/runtime-config sets them
Not touched (comes in follow-up slices):
- Wizard picker exposing 'reuse research team' vs 'fresh coding team'
- Runtime container spawn keyed on team_id
- Migration of existing paired coding loops onto their own team
|
||
|
Omar Sobh
|
e3ef3fd056 |
research: fix Draft 'Invalid Date' + stale-error leak in pipeline card
Two cosmetic bugs surfaced by the successful v0.8.3 pipeline run: 1. **'produced Invalid Date'** — research_outcomes.created_at was an OffsetDateTime serialized by time's default array format (`[y, ordinal, hh, mm, ss, ns, tz]`), which browser's `new Date(...)` can't parse. Add `#[serde(with = "time::serde::rfc3339")]` matching the pattern already in threads.rs / routine_runs.rs. 2. **Stale error text on pipeline card** — the runs stage's `latest_error` walked every run by `created_at DESC` and returned the first non-empty error, so a topic with an earlier failed run + a later completed run kept displaying the old error next to '1 completed'. Now the error only surfaces when the MOST RECENT run itself failed. Historical failures stay in the run count but don't leak their message. |
||
|
Omar Sobh
|
f70f6c679e |
research: errored-state card + one-click rerun (no wizard re-entry)
When a topic ends up parked in 'processing' with all runs failed and nothing in flight, the sidebar card was still spinning as if progress were happening. Now: Backend - topology_runs::run_counts_by_research_topic — batch query that returns (in_flight, failed-since-last-success) per topic. Used by the list endpoint; dynamic sqlx::query() so no prepare needed. - TopicListItem DTO gains runs_in_flight + runs_failed. - start_topic status guard relaxed: allow (standby) OR (processing AND runs_in_flight == 0). Blocks accidental double-fires on a live pipeline; permits rerun on a failed one. Same request body, same behavior once accepted, so the frontend just POSTs /research/:id/start on the RotateCw click. Frontend - ResearchList detects errored: status===processing && !in_flight && failed>0. Swaps the MiniSpinner for a red AlertTriangle and changes the status text to 'error · N failed'. - New RotateCw icon button next to the delete Trash — same button cluster, one click, no wizard re-entry required. Disables while a request is in flight; error surfaces in the sidebar's shared error banner. |
||
 osobh
|
1d69866bcf | research canvas: collapsible sidebar + live topology-run logs (#6) | ||
|
Omar Sobh
|
7e0620fd08 |
research: don't advance to reviewing without an outcome + guard-before-write
Two bugs that combine to produce the 409 you get when clicking Approve:
1) notify_run_completed (topology_worker post-hook) was advancing
topic status processing → reviewing whenever the last sibling run
terminated — success OR fail. A failed run with 0 outcomes still
pushed the topic to `reviewing`, the canvas rendered the "Request
publish" affordance, and the reviewer clicked Approve on nothing.
Fixed by adding an EXISTS(research_outcomes …) clause to the
UPDATE. Topic stays in `processing` when no outcome exists; the
loop's next iteration still has a chance to produce one.
2) decide_publish was calling
research_publish_approvals::decide(approve=true)
FIRST (which flips the row to `status='approved'`) and then
running the "no outcome? 409" guard SECOND. On the 409 return,
the DB was left half-flipped: approval says approved, topic still
in reviewing, no outcome exists, and every future click to the
same approval returns 409 on the "already decided" guard —
leaving reviewers with no way forward.
Fixed by moving the outcome-existence check BEFORE the decide()
call. On 409 now nothing was written, so the reviewer can try
again cleanly once an outcome is produced.
Also unstuck the current stuck row out-of-band (SQL UPDATE to reset
the approval to pending + topic to processing) so the user isn't
forced to delete the topic to escape the 409 loop.
sqlx dynamic query — the new EXISTS clause wasn't in the offline
cache so I switched notify_run_completed to plain `sqlx::query`.
|
||
|
Omar Sobh
|
bfcdca0583 |
research-canvas: managed-by-loop UI + start_topic guard (fold cleanup)
Closes the UX gap the fold introduced: the topic canvas was still showing "Start research" for standby-state topics even when a scheduled loop already owned the runs. Clicking it would 409 (or worse: race the loop into a duplicate run). Topic status stayed at standby forever because the loop path bypassed start_topic's set_status transition. Four changes: 1. **Backend status transition** — compose_and_enqueue_iteration for kind='research' now calls set_status_if(standby, processing) on the topic before the run is enqueued. New DB helper set_status_if only advances when the current status matches the "from" arg — safe against races and re-invocations. Later iterations no-op since the topic is already past standby. 2. **has_managed_loop on TopicDetail** — get_topic hydrates a new ManagedLoop struct (loop_id, title, enabled, next_fire_at, last_run_id, schedule_summary) when a kind='research' loop is bound to the topic. summarize_schedule() derives a human string from the loop's triggers jsonb (e.g. "cron: 0 3 * * * · on new artifact", "one-shot", "manual"). New DB helper loops::research_loop_for_topic returns the row. 3. **Canvas branch** — nextAction takes a managedByLoop flag; when set + status=standby, returns null (no button). The canvas renders a "MANAGED BY LOOP" strip below the topic title showing loop name, schedule summary, next fire time, and enabled dot. Reviewer buttons (Request publish / Approve / Reject) still show normally in later states — reviewers should still promote outcomes even when a loop is producing them. 4. **start_topic guard** — refuses with 409 when a research loop already owns the topic. Closes the direct-POST hole for anyone bypassing the frontend. TS type + summarize_schedule live in the same commit so an old client hitting a new backend just ignores the extra field (no breakage), and a new client hitting an old backend renders the classic buttons (managed_by_loop is optional). |
||
|
Omar Sobh
|
51756d0e68 |
clippy: struct-bundle enqueue_iteration_with_topic + fix doc list warnings
CI's clippy stage failed with -D warnings on three classes of lint:
1. map_clone in loops::recent_reorders — .map(|a| a.clone()) is the
pattern clippy wants replaced by .cloned(). Trivial swap.
2. too_many_arguments on enqueue_iteration_with_topic (8 args, ceiling
7). Refactored the caller-side surface into a new
IterationEnqueue<'a> struct with fields for each column. Matches
the pattern research::NewTopic + loops::NewLoop already use for
the same clippy ceiling. Callers in loops.rs (routes) + the
enqueue_iteration wrapper updated to build the struct literal.
3. doc_lazy_continuation + doc_list_indentation — two doc comment
blocks used ambiguous list-like layouts:
- compose_iteration_task's ASCII-art prepended-block preview:
wrapped in a ```text fence so clippy stops parsing "RESEARCH
ARTIFACT:" etc as list continuation.
- fire_initial_burst_if_set's mention of `burst - 1`: rewrote so
"- 1" doesn't start a line and get misread as list marker.
Purely a refactor + doc pass; no behavior change.
|
||
|
Omar Sobh
|
3e42b1ea39 |
research-wizard: schedule step (once/nightly/manual) + paired coding loop
Completes the research/loop fold. The wizard now closes with a
"How should this research run?" step; picking any mode materializes a
kind='research' loop bound to the topic, and an optional checkbox
adds a paired kind='exec' loop that consumes each new artifact.
Frontend:
- ResearchWizard grows from 5 to 6 steps. Step 6 is the schedule
picker:
· Just once — initial_burst=1, no other triggers
· Nightly — initial_burst=1 + cron "0 3 * * *"
· Manual — webhook_enabled=true
Below the radios, an optional card offers "Also create a coding
loop that consumes each new artifact" — creates a paired
kind='exec' loop with on_artifact_update=true + initial_burst=1
bound to the same topic.
- createTopic API type extended with `schedule` + `create_paired_coding_loop`.
- Both fields ride the existing POST /api/research call; back-compat
is preserved when the wizard omits them.
Backend:
- CreateTopicRequest gains TopicSchedule + create_paired_coding_loop.
- After topic + agent attach, create_topic calls
materialize_topic_loops which:
1. Creates a research-kind loop titled "Research · <topic>" bound
to the topic. Triggers vary by schedule mode; next_fire_at
computed from cron for nightly. Falls back silently if
loop-create errors so the topic still lands.
2. Flips kind to 'research' via loops::set_kind (NewLoop doesn't
take kind directly — default is 'exec' for backward compat).
3. Optionally creates a coding loop titled "Coding · <topic>"
with on_artifact_update=true + initial_burst=1.
- cm_db::repo::loops::set_kind — trivial UPDATE helper used by the
materialize path.
D-answer callouts:
- D1 (fold): every runnable thing is now a loop. "Just once" is a
research loop with initial_burst=1 and no other triggers.
- D2 (inherit): both paired loops carry the same source_research_topic_id
— repo binding lives on the topic, not duplicated.
- D3 (coordinator resolves): the research iteration prompt (from the
earlier commit) instructs the team to preserve stable INT ids and
mark deprecations; coding loops' consumed lists stay valid across
versions.
Follow-ups queued:
- Kind pill on LoopsList cards (research=purple, exec=cyan) so users
can tell them apart at a glance.
- Extract start_topic's task-build so kind='research' iterations
reuse the same coordinator prompt shape as one-shot runs (they
currently use a simpler refresh-oriented prompt; that's fine for
MVP but a rich shared build would give better parity).
- Research topic sidebar shows "linked to N loops" badge.
|
||
|
Omar Sobh
|
91a51dce11 |
loops: kind='research' dispatch — research runs as loops
Delivers the research/loop fold: kind='research' loops run the research pipeline each iteration, appending a new research_outcomes version. The paired on_artifact_update fan-out then wakes up any kind='exec' loops bound to the same topic to consume new INTs. Every runnable thing is now a loop (D1). Backend — DB helpers: - cm_db::repo::loops::kind_and_binding — reads (kind, source_topic, task_template) so callers can dispatch without hydrating the whole Loop struct. - cm_db::repo::loops::enqueue_iteration_with_topic — new variant that sets research_topic_id on topology_runs alongside loop_id, so the completion hook's freeze_research_outcome writes a new outcome version for research-kind iterations. - cm_db::repo::research_topics::get_any_workspace — cross-workspace fetch used by the research task builder (the loop row is authoritative for the workspace binding via kind_and_binding). Backend — dispatch: - routes::loops::compose_research_iteration_task — builds the coordinator prompt for a research iteration: topic title + description + outcome_kind + prior version pointer + refresh instructions (survey new sources, preserve stable INT ids, mark superseded items as deprecated rather than delete). The completion hook writes the resulting synthesis as research_outcomes v(prior+1). - routes::loops::compose_and_enqueue_iteration — one-shot dispatch: reads the kind, picks compose_iteration_task (exec) or compose_research_iteration_task (research), enqueues with or without research_topic_id set. All four enqueue callsites now route through compose_and_enqueue: - create_loop (initial_burst) - run_now - webhook_receive - topology_worker::continue_initial_burst - topology_worker::freeze_research_outcome (on_artifact_update fan-out) Research-kind loops naturally form the "nightly refresh" side of a paired research + coding loop: research writes a fresh outcome version → fan-out wakes exec loops with on_artifact_update → coding loops consume the next INT (which the research loop may have just added). D2 answer (inherit repo binding): repo lives on the topic; both loops sharing the source topic id read from the same context, no duplication. D3 answer (coordinator resolves): the research iteration prompt tells the team to preserve stable INT ids and mark deprecations rather than delete, so coding loops' consumed lists stay valid across versions. Follow-up (next commit): ResearchWizard schedule step — "Just once / Nightly / Manual" that creates the paired research-kind loop with initial_burst=1 (just once) or cron 0 3 * * * (nightly) + optional paired coding loop with on_artifact_update. |
||
|
Omar Sobh
|
4ada5557f2 |
loops: kind column + initial_burst + on_artifact_update trigger fan-out
Foundation for folding research into loops as a first-class kind.
This commit ships the plumbing; the research-kind dispatch itself
lands next. Behavior for existing exec-kind loops is unchanged unless
they opt into the new trigger fields.
Migration 0044:
- kind TEXT NOT NULL DEFAULT 'exec' CHECK ('exec' | 'research'). New
research-kind will run the research pipeline each iteration (next
commit); 'exec' preserves today's behavior.
- initial_burst_remaining INT NOT NULL DEFAULT 0 — countdown for the
triggers.initial_burst quota. Decremented CAS-safely on each
completion until it hits 0.
- Two partial indexes: (kind, source_research_topic_id) for kind-
aware lookups, and (source_research_topic_id) filtered on
on_artifact_update=true + enabled=true for the fan-out hook.
Trigger schema extended with two optional fields:
- initial_burst: N — fire N iterations back-to-back at create time.
create_loop enqueues the first iteration inline (subject to
empty-roster gate), sets remaining=N-1, and the completion hook
continues the chain until exhausted.
- on_artifact_update: true — when a bound research_outcomes row is
inserted for the source topic, wake up one iteration of this loop.
Coalesced against has_active_run so a burst of rapid revisions
doesn't queue duplicates.
Backend:
- cm_db::repo::loops helpers (all dynamic sqlx, no offline cache
regen needed):
- set_initial_burst_remaining
- take_initial_burst_slot (CAS UPDATE returning prev value; 0 on
exhausted or race loss)
- loops_awaiting_topic (fan-out query: kind=exec + enabled +
on_artifact_update=true bound to the given topic)
- has_active_run (queued|running iteration existence check)
- get_any_workspace (bypasses the workspace scope guard; used by
the completion hook where the run row is authoritative)
- routes/loops::compose_iteration_task made pub so the completion
hook can build the same enriched task string as run_now.
- topology_worker::freeze_research_outcome now fans out to awakened
loops after the outcome insert, using compose_iteration_task and
coalescing on has_active_run.
- topology_worker::continue_initial_burst runs on every completion:
· take_initial_burst_slot (CAS) — no-op if already exhausted
· has_active_run coalesce guard
· re-fetches the loop via get_any_workspace + compose_iteration_task
· enqueues via loops::enqueue_iteration with parent_run_id set
Follow-ups already queued:
- kind='research' dispatch in run_job — build the research
coordinator task from the topic config, run the research pipeline
each iteration. Requires factoring start_topic's task-build.
- ResearchWizard "When should this run?" step (Just once / Nightly /
Manual) creating the topic + paired research-kind loop.
- LoopsWizard trigger UI matching the design proposal (burst count,
cron, on-artifact checkbox).
|
||
|
Omar Sobh
|
ce111273bb |
loops: reorder history collapsed under the progress pill
Completes the reorder rationale loop — the previous commit captured
REORDER: markers server-side but nothing surfaced them. Now the loops
sidebar renders a small "N reorders ▸" button under the progress pill
whenever the loop has any reorder events. Click expands to a
newest-first list of "iter <n> · <rationale>" lines so a reviewer can
see, at a glance, when the plan was adjusted and why.
Backend:
- LoopProgress DTO gains recent_reorders: Vec<Value> — newest-first,
capped at 5 so the card stays compact. Full history remains on the
loop row's reorder_events column.
- cm_db::repo::loops::recent_reorders — reads the jsonb array, returns
the last N in newest-first order.
- list_progress populates it per loop.
Frontend:
- LoopReorderEvent + recent_reorders on LoopProgress type.
- LoopsList tracks openHistoryId per-loop (one open at a time).
- Card renders history button + expanded panel styled to match the
progress pill above.
Notes:
- The event object schema is {run_id, iteration, text, ts}. Fields are
optional in the TS type so future schema tweaks don't break the
render.
- 5-item cap chosen so the sidebar card doesn't grow unbounded. If a
loop accumulates a lot of reorders, follow-up UI can render the full
history on the loop detail page.
|
||
|
Omar Sobh
|
0c17de52dd |
loops: reorder rationale extraction — REORDER: markers logged per iteration
Coordinator can now log WHY it worked on an INT-XX out of order
("REORDER: INT-05 before INT-04 because prereq X is unmet") and the
completion hook captures each rationale as an append-only event on
the loop. Sets up a reviewable timeline of when the plan was
adjusted, independent of the underlying `consumed_int_ids` advance.
Migration 0043:
- loops.reorder_events JSONB NOT NULL DEFAULT '[]'::jsonb — append-
only array of {run_id, iteration, text, ts}. Kept on the loop row
(rather than a dedicated table) so the mini-timeline is one read
away from the loop card.
Backend:
- topology_worker::parse_reorder_rationale — line matcher symmetric
with parse_completed_int_ids. Tolerates list dashes / prefixes /
markdown emphasis; case-insensitive marker match, preserves case of
the rationale text.
- cm_db::repo::loops::append_reorder_event — one INSERT-like append
per rationale, uses jsonb_build_object with postgres now() so ts is
wall-clock canonical (no client-clock skew).
- topology_runs::iteration_for_run — new helper so events carry the
iteration index.
- routes::loops::compose_iteration_task — coordinator prompt now
explicitly asks for `REORDER: <one-sentence>` at the top of the
first substantive turn when working out of order, AND spells out
that both markers must appear literally with colons (no bold, no
code fence) so the line parser doesn't miss them.
Non-loop and standalone-loop runs are unaffected — the hook only
fires when the run belongs to a source-bound loop.
Follow-up: expose reorder_events on the loops list endpoint + render
a small collapsed timeline on the LoopsList card.
|
||
|
Omar Sobh
|
4a140cb7db |
loops: parse COMPLETED: INT-XX markers to advance loop pointer (option b)
Closes the loop bridge — the missing piece from the previous commit.
Without this, `current_int_index` stayed at 0 forever and every
iteration re-worked INT-01. Now the topology_worker's completion hook
parses the run's final output for `COMPLETED: INT-<NN>` markers and
atomically advances the loop's consumed_int_ids + current_int_index.
Backend:
- topology_worker::advance_loop_after_completion — new post-terminal
hook that fires alongside freeze_research_outcome. Reads loop_id_for_run
(skips non-loop runs) + source_research_context (skips standalone
loops without a bound source topic).
- parse_completed_int_ids — forgiving parser: matches `COMPLETED: INT-01`,
`- COMPLETED: `INT-01``, `COMPLETED: INT-01, INT-02`, case-insensitive,
tolerates list dashes / backticks / markdown emphasis. De-dupes within
a single output.
- cm_db::repo::loops::advance_after_completion — atomic UPDATE that:
· appends only NEW ids to consumed_int_ids (idempotent on re-runs)
· bumps current_int_index by the count of new ids landed
Set semantics via `SELECT DISTINCT unnest(...)` so ordering-based
bugs can't accumulate duplicates.
Behavior end-to-end:
1. Loop wizard imports an integrations artifact (previous commit).
2. run_now / webhook_receive → compose_iteration_task prepends artifact
+ focus instruction ("address INT-<current+1>, log COMPLETED at end").
3. Coordinator run does the work, emits `COMPLETED: INT-<NN>`.
4. topology_worker completion hook parses the marker, advances the
loop, and the NEXT iteration sees an updated `consumed:` list +
incremented `current_int_index` in its focus instruction.
Follow-ups still queued:
- Loop card refresh button — pull latest artifact after reject-with-
revision on the source topic (right now the prepend uses the LATEST
outcome automatically, so refresh is UX only, not correctness).
- Reorder rationale extraction — coordinator emits "REORDER: INT-05
before INT-04 because prereq X is unmet"; today that's just prose
in the output, not indexed.
|
||
|
Omar Sobh
|
ce73abe5ab |
loops: bridge research artifact into loop iterations (option C + b)
The bridge lets a coding loop "consume" an integrations research artifact one INT-XX item per iteration. Options b (order-sequential iteration) and C (snapshot in task_template + save the pointer for future refresh) from the design discussion. Migration 0042 — three new loops columns: - source_research_topic_id — nullable pointer to research_topics. - consumed_int_ids TEXT[] — INT-XX ids the loop has completed. Advances when topology_worker parses "COMPLETED: INT-<NN>" markers from the run's final output (wired in a follow-up commit). - current_int_index INT — monotonic pointer for order-sequential iteration. Coordinator addresses INT-<current+1> unless prereqs are unmet, in which case it works on the smallest unblocking INT-XX and logs the reorder rationale. Backend: - cm_db::repo::loops::set_source_research_topic — bind/unbind pointer. - cm_db::repo::loops::source_research_context — read pointer + state. - routes::loops::compose_iteration_task — new caller-side helper that reads the pointer, fetches the topic's latest research_outcome, and prepends the artifact + focus instruction to task_template. - run_now + webhook_receive both pass task_template through compose_iteration_task before enqueue. Standalone loops (no pointer) behave identically to before. - CreateLoopRequest accepts `source_research_topic_id`, ownership- checked via research_topics::get before persist. Frontend: - New ResearchArtifactPicker modal — lists published topics, fetches the artifact on pick, returns (topic_id, markdown) to caller. - LoopsWizard task_template step gains "Import from research artifact" button (right-aligned). Click opens the picker. On pick: task populates with the artifact markdown, pointer saved, textarea expands to 8 rows, small info strip shows "Loop is bound to topic <id>. Each iteration will focus on the next unconsumed INT-XX." - Unlink button reverts to standalone loop mode. Follow-up (next commit): - topology_worker completion hook — parse "COMPLETED: INT-<NN>" out of the run's final output + update consumed_int_ids + current_int_index atomically. Without this, current_int_index stays at 0 forever and every iteration works on the same INT. - Loop card refresh button — re-read source topic's latest outcome (useful after a reject-with-revision cycle on the source topic). |
||
|
Omar Sobh
|
44d6e95022 |
fmt: apply rustfmt across the P2 arc
CI's rustfmt check flagged the multi-line sqlx::query() calls I introduced in P2 (loop_id_for_run, zeroclaw_gateway_url, etc.). No behavior change — pure formatting. |
||
|
Omar Sobh
|
03f1830d1f |
loops: Path B container isolation (P2)
Symmetric with the research pipeline: every enabled loop can now have its own per-loop team container so scheduled runs don't share state with other loops or with research. Same daemon image, same clawmates network, deterministic name loop-<id>-team. Backend surface: - Migration 0040 adds nullable `zeroclaw_container` + `zeroclaw_gateway_url` columns to loops (parallel to research_topics). - research_container.rs grows loop_container_name_for(), spawn_loop() (state-only mount, no repo), and teardown_loop(). Kept in the same module to share the docker connect() + inherited_env() plumbing; each pattern gets its own labels (clawmates.role=loop-team) so ps filters can tell them apart. - cm_db::repo::loops gains set_zeroclaw_container() + zeroclaw_gateway_url() (dynamic sqlx queries — no offline cache regen needed). - cm_db::repo::topology_runs gets loop_id_for_run(): mirror of research_topic_id, used by the worker. Wiring: - routes/loops::run_now + webhook_receive call ensure_loop_container() before enqueuing an iteration. Idempotent: an already-running container is just reattached. Failures are logged and do NOT block the enqueue — topology_worker falls back to the workspace gateway when the URL isn't set on the loop. - routes/loops::disable_loop + delete_loop both fire teardown_loop() so paused / deleted loops don't hold a docker slot. - topology_worker's per-run URL resolution: existing research fast path unchanged; when it doesn't hit, the worker now looks up loop_id and reads the loop's gateway URL. Deploy step (required on gw-04 for state to persist across container restarts): add a `/var/lib/clawmates-loops:/var/lib/clawmates-loops` bind mount + `CLAWMATES_LOOPS_STATE_ROOT=/var/lib/clawmates-loops` env var to clawmates_server_1 in the compose. Without it, loops still run — the state dir lives inside the API container's filesystem so persistence is limited to that container's lifetime. Follow-up: - Scheduler-tick fires (cron-driven, not run_now) — they call enqueue_iteration in cm-scheduler and don't yet go through ensure_loop_container. Add a symmetric spawn there so cron fires also land on the isolated daemon. - Compose file reconciliation — deploy/compose/docker-compose.yml in the repo has drifted from prod; when we sync it, add the loops mount at the same time. |
||
|
Omar Sobh
|
e3011ed025 |
research: reject-with-revision loop (R2)
Before: reviewer rejected a publish → audit log flipped, topic stayed in reviewing, no way to feed the critique back into the run pipeline. Reviewers with revision notes had to eat them or hand-message the coordinator. Now: reject accepts an optional `notes` field. When present: - Persisted on the research_publish_approvals row (migration 0039). - Topic flips `reviewing → standby` so the next `start_topic` is legal. - `start_topic` reads the most recent rejected-approval notes for the topic and prepends "PRIOR REVIEW NOTES (address these in this revision):\n<notes>\n---" to the coordinator task. Loop closes through the same run pipeline — no new spawn code path, which means the reviewer's guidance flows through the same topology_worker, run_events, outcome-writer chain and lands as a fresh research_outcomes row (versioned, prior drafts preserved). No notes on reject = legacy behavior (topic stays in reviewing, publish requests still allowed). Migration 0039 adds nullable `notes TEXT` to research_publish_approvals. `decide()` gains a `notes: Option<&str>` parameter (only one caller, updated inline). New `latest_rejection_notes(pool, topic_id)` helper for start_topic. Frontend: - rejectPublish(id, notes?) now sends a JSON body when notes are provided. - ResearchCanvas reject button opens an inline form with a textarea + Cancel/"Send back for revision" pair. Empty notes → plain reject. - Button label switches: "Send back for revision" when notes present, "Reject without notes" when empty. Follow-up: - Notes shown in the review UI on the resulting draft so the next reviewer sees what changed. - Multiple rejection rounds — currently only the LATEST rejection's notes surface. Accumulating history is a schema-only tweak. |
||
|
Omar Sobh
|
acd2a0f287 |
structure polish: post-reify nav + ensure-chain + TeamWizard auto-parent
Two small quality-of-life fixes on top of the reify commit:
Post-reify navigation
OrphanMigrationDialog already returned team_id in its result;
Dashboard now pushes /?team=<team_id> before router.refresh() so
the user lands on the freshly-materialized team and sees exactly
where their agents just moved. Previously they had to hunt for it
in the newly-rebuilt sidebar.
Wizard auto-materialize (POST /api/structure/ensure-chain)
cm-db: ensure_chain(pool, ws, fallback_org, fallback_company) —
fast path returns coordinates of the first org+company already
bound in this workspace (workspace's oldest org, oldest company
under it). Slow path inserts a new org+company with the
fallback names ("My Workspace" / "General") + binds them via
org_companies. Returns { org_id, company_id, created }. Small
txn — leaves the workspace consistent whether it was already
wired or not.
cm-api: POST /api/structure/ensure-chain accepts optional
fallback_org_name and fallback_company_name in the body (trimmed,
else default). Returns the ids.
CreateTeamRequest gains an optional attach_to_company_id. When
set, after build_team() completes, we look up the company
(workspace ownership check enforced by companies::get), count
its existing teams for a stable n_i node id, and insert a
company_teams binding — so the team lands under the parent
atomically instead of a follow-up round-trip.
TeamWizard now calls ensure-chain before POST /api/teams and
passes the returned company_id in attach_to_company_id. Both
calls are best-effort — if ensure-chain fails (network etc.)
we still try to create the team, and the migration dialog stays
available as the fallback UX. Wizard flow now: fresh workspace's
first team is fully wired from the moment it appears in the
tree — no synthetic "My Workspace" scaffolding ever gets
rendered around it.
The Team/Company create paths not touched here (create_team_from_claws,
company create, org create, MasterPlannerModal scaffold) still
work as before — they just won't auto-parent yet. Later commits
can wire them the same way.
|
||
|
Omar Sobh
|
8b789beec0 |
structure: reify-orphans endpoint + "give these a home" dialog
Turns the four synthetic tree containers into a real migration path.
Clicking any of them ("My Workspace", "Teams", "Direct",
"Ungrouped") opens a dialog that creates a real
org → company → team chain and re-parents every orphan into it, all
in one DB transaction.
cm-db (new module structure_reify)
- orphan_agents / orphan_teams / orphan_companies: workspace-scoped
SELECTs of entities without a parent binding in team_members /
company_teams / org_companies. Used both by the dialog's counter
and internally by the migration.
- count_orphans: cheap combined-count via three subqueries in a
single SELECT so the dialog only round-trips once for the header.
- reify_orphans(pool, ws, org_name, company_name, team_name):
1. begins a tx
2. inserts a new org + company + team (all `flat`, empty graphs
— user can shape them later via the existing PATCH endpoints)
3. binds company under org (org_companies "n0")
4. binds team under company (company_teams "n0")
5. inserts team_members rows for every orphan agent (n1, n2, …)
6. inserts company_teams rows for every orphan team
7. inserts org_companies rows for every orphan company
8. commits, returns the created ids + moved counts
cm-api (routes/structure)
- GET /api/structure/orphan-counts → { agents, teams, companies }
- POST /api/structure/reify-orphans → { org_id, company_id, team_id,
moved_* }. Trims + rejects any empty name; validates before
starting the transaction so a 400 never rolls anything back.
Frontend
- New OrphanMigrationDialog: fetches counts on open, three name
fields (defaults: Organization "My Workspace", Company "General",
Team "Everyone"), POSTs on save. "Nothing to migrate" state
disables the save button when the workspace is already fully
wired. Copy explicitly notes that everything is renameable in the
sidebar afterward.
- Dashboard: onTreeSelect now branches on SYNTHETIC_TREE_IDS —
clicking a synthetic node opens the dialog instead of falling
through to the (nonexistent) selection. On successful reify,
router.refresh() so the sidebar + world viz reflect the new real
chain.
What this doesn't do yet (next commit)
- Wizard auto-materialize: when creating a team/company via wizard,
auto-create parent placeholders if they don't exist. Deferred so
this commit stays focused.
|
||
|
Omar Sobh
|
99e5207e69 |
sidebar: click-to-rename org/company/team + strip synthetics from world viz
Two related pieces of the "kill My Workspace" cleanup, landed
together because they share the same file:
Backend
- Three tiny inline-rename endpoints:
PATCH /api/orgs/{id}/name
PATCH /api/companies/{id}/name
PATCH /api/teams/{id}/name
Each takes { name: string }, trims + rejects empty, returns 204.
Backed by rename_org / rename_company / rename_team in cm-db —
single-row UPDATEs scoped to the caller's workspace, NotFound if
the id isn't visible.
- Registered next to the existing PATCH /:id (topology) routes so
they don't collide.
Frontend
- StructureTree accepts an optional onRename and canRename.
TreeRow: click on the label text of a renamable node → the span
becomes an <input>, focus + select-all, save on Enter or blur,
cancel on Escape. The rest of the row (row chevron / row body)
still navigates + selects as before, so single-click behaviour
is preserved for everything except the name text itself.
react-hooks/set-state-in-effect avoided by resetting the draft
in the enterEdit() click handler instead of inside a useEffect.
- Dashboard passes canRename={item.level !== "claw" && !synthetic}
(claws don't have a rename endpoint yet; synthetic scaffolding
gets reified into real rows in the next commit — the wizard
auto-materialize + orphan-migration dialog).
onRename fires the corresponding PATCH and calls router.refresh()
so the label lands in every consumer of the tree.
- World viz seed: new stripSynthetics(roots) helper walks the tree
and lifts children of any synthetic container up to their
grandparent's level. worldCanvasRoots feeds through this before
narrowRoots(). Result: the Live viz no longer shows "My Workspace"
or "Teams" nodes — real agents orbit the world root directly
(which is what you were asking for). Sidebar tree still shows
them so orphaned agents remain visible until the migration lands.
|