d23f30e92950c7b67fa24bad1e06882be61c044d
453
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar Sobh
|
6504ed6062 | fmt: single-line if condition in mcp bearer rewriter | ||
|
Omar SobhandClaude Opus 4.7
|
fd3ebb628a |
clippy: replace is_some+unwrap with if let in mcp bearer rewriter
Co-Authored-By: Claude Opus 4.7 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.7
|
cbb709e31d |
fix: avoid let-chain (needs edition 2024) in mcp bearer rewriter
CI's fmt check runs on stable toolchain that rejects let chains in if conditions. Nest the `if let` inside the block instead. Co-Authored-By: Claude Opus 4.7 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.7
|
49f94a5360 |
mcp door: mint workspace-owner service session for team runtime bearer
The runtime template's static clawmates_door bearer is rejected by cm_auth::authenticate() (needs an auth_sessions row). Every per-team agent was getting `unauthorized: missing or invalid bearer token` and `0 tool(s) registered from 0 server(s)`. Add AuthService::mint_service_session + users::owner_of_workspace and mint a 30d service session in try_team_gateway_url; inject it into the freshly-spawned team container's config.toml [[mcp.servers]] clawmates Authorization header via prewrite_daemon_config_with_risk (bearer arg). Follow-up: apply the same pattern to research::spawn (per-topic) and per-loop spawn paths. Co-Authored-By: Claude Opus 4.7 <[email protected]> |
||
|
Omar Sobh
|
8cd0c7bd01 |
runtime_provision: migrate provider_alias_for to v0.8.3 families
Auto-provision (and every other build_team caller) hit 'dangling_reference: claude_cli is not a known provider family' because provider_alias_for still returned v0.7.x aliases — claude_cli.default / claude_cli.glm / claude_cli.glm5 / kimi_cli.default — all deleted upstream when providers.models schema was restructured. Now the mapping resolves to real configured aliases: - 'claude'* prefix → anthropic.default - 'gemini'* prefix → gemini.default - 'llama'* / 'groq' → groq.default - glm* + kimi* → anthropic.default (fallback until glm.default / moonshot.default provider tables land in the runtime template) - unknown → anthropic.default Covers the model picker's full roster (claude-sonnet-5 / opus-4-8 / haiku-4-5 / sonnet-4-6 / glm-4.6 / glm-5.2 / kimi-k2 / gemini-2.0-flash / llama-3.3-70b-versatile) plus every legacy shorthand. Tests updated to assert the new mappings including the model picker's ids. |
||
|
Omar Sobh
|
6e37802f73 |
claws: PATCH /api/claws/{id}/model — swap the runtime-bound model
Auto-provisioned agents (via the research + loops team wizard) show
up on the Agents page (same agents::insert → agents::roster path),
and PATCH /api/claws/{id} already handled name/job_title/system_prompt/
avatar/accent/wallpaper. The one thing that was NOT reachable from the
Agents page: swapping the model.
Add a dedicated endpoint:
- PATCH /api/claws/{id}/model { model: string }
- Persists via agents::set_model_binding (DB)
- Best-effort runtime rebind via RuntimeProvisioner::provision_claw
(idempotent — overwrites agents.<alias>.model_provider on the
shared ZeroClaw config)
- Audit-logged as 'agent.model_changed' with the new model in payload
Now an operator can open the Agents page, click a claw that was
auto-provisioned by the team wizard, and swap its model
(claude-sonnet-5 → glm-5.2 → whatever) without recreating the team.
Same DB row, same claw_id, same brain — just a new provider on the
next turn.
Frontend affordance not shipped in this commit — the endpoint is
usable via curl/psql/scripts today; a UI 'Model' picker on the claw
detail card can land in the next Agents-page pass.
|
||
|
Omar Sobh
|
ac8c689f50 |
logs + team parity: pretty step/container renderers; quota + audit on team creation
Two bundled changes:
── LiveRunLogs prettification ──────────────────────────────────
The Steps + Container tabs were plain mono lines with a single
color per event. Now they get structured layout:
Steps:
- Color-hashed actor pill (stable palette so [Distiller] and
[Novelty Analyst] each get their own hue across the session).
- Phase pill (plan=cyan, work=green, synth=amber, aggregate=purple).
- Token count pill formatted 1.2k / 14.3k / etc.
- Gated-action warning pill in amber when > 0.
- Left-border color strip keyed to the actor for at-a-glance
visual grouping.
- Long outputs collapse to their first 300 chars with a '+ N more'
toggle to expand the full text.
- 'done' events get a green (or red for error) border strip +
pill instead of blending into the stream.
Container:
- Splits '[actor] action (outcome) · msg' into colored spans —
actor pill (deterministic color), action in dim, outcome pill
green/red/dim by state.
- Non-line events (info/error/done) get their own left-border
strip so bash echoes and stack traces don't drown in the daemon
chatter.
- Timestamps switch to HH:MM:SS.mmm — dense but scannable.
Small palette (LOG constants) keeps the color budget bounded — no
new UI vocabulary, just cleaner reads of what was already there.
── Team-wizard governance parity ───────────────────────────────
build_team_with_lifecycle now matches POST /api/claws' governance:
- enforce_new_agent quota check per member (previously bypassed
workspace agent quotas entirely for team/auto-provision paths).
- audit::append('agent.created', ..., {source: 'team_wizard'}) per
member so team-created claws appear in the same audit trail as
individually-created ones. Adding a 'source' key distinguishes
provenance without changing consumers.
.brain (h5) handling was already consistent between the two paths —
both use the lazy on-first-access load_brain hook seeded from
agents.system_prompt. No change there.
|
||
|
Omar Sobh
|
956be2cf4f |
wizard: in-place auto-provision team from topic (LLM-derived, sonnet-5)
Step 5 of ResearchWizard was 'assign agents from workspace roster'.
When the roster was empty, the wizard body was hard-swapped for
NoAgentsGate — you couldn't reach step 5 at all.
Now step 5 shows a 'Team' panel:
- Big cyan card: 'Auto-provision team from this topic'. One click
runs an LLM plan pass, gets 3-5 role slots + system prompts back,
materializes claws via the existing build_team pipeline, stamps
runtime posture, returns a shape that drops straight into the
submit body's agents[]. Card flips green with the derived roster.
- Below that: the classic roster picker, but only when the workspace
actually has ≥1 claw AND auto-provision hasn't landed. Otherwise
hidden — no dead empty-state affordance.
Every gate that required agents.length > 0 to render the wizard body
or the footer is gone. canNext gains a step-5 clause: allow Next when
EITHER auto-team is ready OR the user handpicked from a non-empty
roster.
Backend
- POST /api/teams/auto-provision — accepts {title, description,
outcome_kind, topology_kind?, model?, risk_profile?, mcp_bundles?}.
Derives topology from outcome_kind (integrations → pipeline; else
hub_spoke). LLM plan pass yields a JSON roster of 3-5 roles
(role_slot, name, system_prompt). Materializes team + claws via
build_team, stamps risk_profile (default research_web_readonly) +
mcp_bundles (default [clawmates_door, gitea_forge]). Response
carries team_id + agents[] in the shape /api/research already
expects.
- Every provisioned claw runs on claude-sonnet-5 by default;
overridable via the model field.
Follow-ups (not in this slice):
- Same picker in LoopsWizard (slice C — parallel change, same API).
- Post-create 'Team' section on ResearchCanvas / LoopsCanvas so
users can rebind after the fact (slice D).
- Full Teams tier UI + Agents-page deprecation (slice E).
|
||
|
Omar Sobh
|
cde196dbdc |
runtime: bake tea + gitea-mcp binaries + let GITEA_TOKEN pass through
Prep for the SDLC coding team. Adds two upstream Gitea binaries to the clawmates-runtime image so team-scoped agents can drive git.redclaw.dev without custom MCP code: - tea v0.14.2 (shell CLI: clone/push/pr checkout, git remote helper) - gitea-mcp v1.3.0 (native MCP server: list_repo_pull_requests, create_pull_request, create_file, update_file, create_branch, create_issue, get_file_content, etc.) Both installed by arch (amd64 / arm64) at image build time; also adds git to the runtime layer since it was missing (needed by tea's shell delegates). research_container::inherited_env now propagates GITEA_ prefixed vars so a team container inherits GITEA_TOKEN (+ optional GITEA_HOST) from the server env. Team-scoped daemons can then start gitea-mcp via stdio with --token-env GITEA_TOKEN. Followup on gw-04: 1. rebuild clawmates-runtime image against the new Dockerfile 2. add [mcp_bundles.gitea_forge] to the runtime template 3. set GITEA_TOKEN in the server compose env |
||
|
Omar Sobh
|
0d0bb5ffaa |
teams: runtime spawn hookup for per-team containers (slice 3b)
Wires the per-loop-team arc end-to-end. When a loop with team_id fires an iteration, the worker now spawns/reattaches a dedicated team container, mounts the paired research topic's repo at /workspace/repo (rw), and stamps the team's risk_profile into every [agents.*] binding on the freshly-written config.toml. Legacy loops with team_id = NULL keep taking the per-topic / per-loop path unchanged. research_container.rs - team_container_name_for(team_id) = 'team-<uuid>-container' - team_state_root(team_id) — /var/lib/clawmates-team-state/<uuid>/state (overridable via CLAWMATES_TEAM_STATE_ROOT) - prewrite_daemon_config_with_risk: line-based sed that swaps only the risk_profile line inside each [agents.<name>] block. Avoids the regex-eats-array-literal trap that bricked the shared runtime config on the earlier out-of-band edit. - spawn_team: full-shape idempotent spawner. Same mount + env + label pattern as spawn/spawn_loop; additionally supports Claude settings bind-mount + external-bridge attach. topology_worker.rs - try_team_gateway_url resolver runs BEFORE the existing per-topic and per-loop lookups. Cold path: reads team runtime config + paired research topic repo path, spawns the container, persists coords back to teams.zeroclaw_container/zeroclaw_gateway_url. Any failure logs + returns None so the caller falls through to the legacy shared-container path — team spawn can never brick a run that could otherwise complete. Not shipped in this slice: - Wizard 'existing team' picker (currently just fresh vs reuse) - Teams tier UI to list/edit/delete teams - Auto-teardown for stale team containers (piggyback on existing reaper is a follow-up) |
||
|
Omar Sobh
|
0b7f247b0e |
wizard: 'fresh coding team' picker for paired coding loop
Second slice of the per-loop-team arc. The paired-coding-loop checkbox
in ResearchWizard step 6 now exposes a two-option picker:
⦿ Provision a dedicated coding team (default when the loop is on)
— fresh 'Coding · <topic>' team row, risk_profile =
coding_readwrite, clawmates_door in mcp_bundles. Loop's
team_id is bound at wizard-submit time.
○ Reuse the research team (legacy) — no team_id bound; coding
iterations spawn against the research topic's container.
Frontend
- New codingTeamMode state, radio picker rendered under the checkbox.
- research.ts createTopic body gains paired_coding_team_mode?: 'fresh'|
'reuse'.
Backend
- CreateTopicRequest gains paired_coding_team_mode: Option<String>.
- materialize_topic_loops takes it through and, when 'fresh', calls
the new provision_fresh_coding_team helper — inserts a teams row
via the existing insert_team_with_lifecycle (pipeline kind, same
graph as the loop), sets its runtime-config via
set_team_runtime_config, then binds loop.team_id.
- All operations best-effort with stderr logging — a team-provision
failure leaves the loop functional under the legacy fallback.
Not shipped in this slice (deferred to runtime hookup slice):
- research_container::spawn keyed on team_id → per-team container
- Config template rewrite injecting the team's risk_profile
- Migration of existing paired loops onto their own teams
The plumbing lands now so the wizard's intent is recorded; the
runtime honors it in the next PR.
|
||
|
Omar Sobh
|
6066e93889 |
teams: per-team runtime posture (risk_profile + mcp_bundles) + FK from loops/topics
Foundation slice for letting a coding loop bring its own team instead
of reusing the paired research topic's team. Turns out the teams
table already exists (0010_teams.sql) with full CRUD — this scales
back to the minimal missing bits:
Schema (0045_teams.sql)
- ALTER TABLE teams ADD risk_profile TEXT (NULL = template default)
- ALTER TABLE teams ADD mcp_bundles JSONB DEFAULT '[]'
- ALTER TABLE loops ADD team_id UUID REFERENCES teams ON DELETE SET NULL
- ALTER TABLE research_topics ADD team_id UUID REFERENCES teams
- Two partial indexes (team_id NOT NULL) for the future cascade queries
cm-db (dynamic sqlx::query so the existing get_team's compile-time
cache doesn't need regenerating):
- TeamRuntimeConfig struct
- get_team_runtime_config / set_team_runtime_config
- team_for_loop / team_for_research_topic (resolvers)
- set_team_for_loop / set_team_for_research_topic (binders)
cm-api
- GET /api/teams/{id} now surfaces risk_profile + mcp_bundles
- PATCH /api/teams/{id}/runtime-config sets them
Not touched (comes in follow-up slices):
- Wizard picker exposing 'reuse research team' vs 'fresh coding team'
- Runtime container spawn keyed on team_id
- Migration of existing paired coding loops onto their own team
|
||
|
Omar Sobh
|
2ba40b03b7 |
loops: per-iteration live logs (Steps + Container tabs), collapse graph JSON
The Loops canvas' IterationsTimeline was a flat status-pill list —
no way to see WHAT an iteration was actually doing. Meanwhile the
Research canvas had full LiveRunLogs with Steps + Container tabs
against the same underlying topology_run SSE endpoints. This
factors LiveRunLogs so both canvases share it.
Frontend
- LiveRunLogs gains a directRunId?: string prop. In this mode it
skips the topic-scoped active-runs + pipeline-state polls and
pins activeRun to the given id. topicId stays optional (topic
mode unchanged from ResearchCanvas' perspective).
- LoopsCanvas IterationsTimeline: each row is now a click-to-
expand card. On expand, renders <LiveRunLogs directRunId={...}/>
right below the header — Steps + Container tabs, full SSE tail,
same 320px terminal.
- Auto-opens the newest running/queued iteration so a click on
'Run now' immediately exposes the live pane.
- New CollapsibleSection helper wraps the graph JSON block so it
starts closed. Reference material stays one click away without
cluttering the canvas.
Backend
- run_container_log_sse now resolves the tail target via
loop.source_research_topic_id when the run itself has no
research_topic_id. Paired coding loops (kind=exec with a
source_research_topic_id) reuse the paired topic's team
container, so we tail its docker logs. Pure loop runs still
error with a clearer message.
|
||
|
Omar Sobh
|
e3ef3fd056 |
research: fix Draft 'Invalid Date' + stale-error leak in pipeline card
Two cosmetic bugs surfaced by the successful v0.8.3 pipeline run: 1. **'produced Invalid Date'** — research_outcomes.created_at was an OffsetDateTime serialized by time's default array format (`[y, ordinal, hh, mm, ss, ns, tz]`), which browser's `new Date(...)` can't parse. Add `#[serde(with = "time::serde::rfc3339")]` matching the pattern already in threads.rs / routine_runs.rs. 2. **Stale error text on pipeline card** — the runs stage's `latest_error` walked every run by `created_at DESC` and returned the first non-empty error, so a topic with an earlier failed run + a later completed run kept displaying the old error next to '1 completed'. Now the error only surfaces when the MOST RECENT run itself failed. Historical failures stay in the run count but don't leak their message. |
||
|
Omar Sobh
|
ae113dd319 |
live-run-logs: add Container tab that tails filtered daemon logs
Steps summaries only fire AFTER each topology step completes — so a
stalled first turn was completely dark. Add a second tab that
streams the team runtime container's daemon log live via a new SSE
endpoint.
Backend
- GET /api/topology-runs/:id/container-log — workspace-scoped SSE
around bollard's docker.logs(follow=true, tail=200). Buffers on
newline so partial mux chunks don't truncate a log line.
- compact_container_log: parse a zeroclaw daemon line
('[actor] ... zc_action=X zc_outcome=Y ... msg') into
'[actor] action (outcome) · msg'. Framing-only continuations are
dropped; non-zc lines (bash echoes, backtraces) pass through as-is
so nothing interesting is lost. ANSI escapes stripped.
- Everything funnels through one async_stream! so early exits
(workspace check / docker connect / no bound topic) yield an
'error' event and return without breaking Sse::new's single stream
type.
Frontend
- LiveRunLogs gets a sub-tabs strip: Steps · Container.
- New useContainerLog(runId, active) hook — gated by tab so we don't
hold two open SSE streams when the operator isn't looking.
- Same terminal widget renders each container line with a level
color (info/done grey, line default, error red). Sub-tab pill
shows count + status live.
|
||
|
Omar Sobh
|
36a9fbe81f |
research/start: allow rerun on loop-owned topics too
D1-fold loop guard 409'd even the rerun path if the topic was scheduled — a failed iteration couldn't be restarted until the loop's next scheduled fire. Now the loop guard only fires for the fresh 'standby' start; the rerun path (status=='processing' + orphan cancel) works whether or not a loop owns the topic. Loop binding is preserved either way. |
||
|
Omar Sobh
|
6af9d509b0 |
research: rerun cancels orphan runs first so 409 doesn't block restart
The previous rerun guard (standby OR (processing AND in_flight==0)) still 409'd when a lingering queued/running row existed — typically an orphan from a server restart mid-pipeline, before the reaper or stale-checkpoint requeuer had marked it failed. Now: allow rerun on any 'processing' topic; before enqueuing the fresh run, batch-cancel every in-flight run for the topic so the new run isn't racing them. Terminal states (reviewing / publishing / published) still 409 as before. |
||
|
Omar Sobh
|
671d3ac4f0 | clippy: fix doc_lazy_continuation on TopicListItem.runs_failed | ||
|
Omar Sobh
|
f70f6c679e |
research: errored-state card + one-click rerun (no wizard re-entry)
When a topic ends up parked in 'processing' with all runs failed and nothing in flight, the sidebar card was still spinning as if progress were happening. Now: Backend - topology_runs::run_counts_by_research_topic — batch query that returns (in_flight, failed-since-last-success) per topic. Used by the list endpoint; dynamic sqlx::query() so no prepare needed. - TopicListItem DTO gains runs_in_flight + runs_failed. - start_topic status guard relaxed: allow (standby) OR (processing AND runs_in_flight == 0). Blocks accidental double-fires on a live pipeline; permits rerun on a failed one. Same request body, same behavior once accepted, so the frontend just POSTs /research/:id/start on the RotateCw click. Frontend - ResearchList detects errored: status===processing && !in_flight && failed>0. Swaps the MiniSpinner for a red AlertTriangle and changes the status text to 'error · N failed'. - New RotateCw icon button next to the delete Trash — same button cluster, one click, no wizard re-entry required. Disables while a request is in flight; error surfaces in the sidebar's shared error banner. |
||
|
Omar Sobh
|
067bebd071 |
wedged-run fix: pre-approve claude perms + verbose step log + reaper
Root cause: claude CLI in the per-topic research container runs as
uid=0(root). ZeroClaw's claude_cli provider passes
--dangerously-skip-permissions which Claude CLI rejects under root
for security — so the CLI hangs waiting for interactive permission
approval that never arrives, hitting the 600s provider timeout with
zero step records journaled.
Three-part fix:
1. Claude settings bind-mount (research_container.rs):
Optional CLAWMATES_CLAUDE_SETTINGS_PATH env — when set, mount the
host file at /root/.claude/settings.json (read-only) in every
spawned team container. deploy/claude-settings.json ships the
canonical config (permissions.defaultMode = bypassPermissions +
hasCompletedOnboarding). CLI accepts requests immediately with no
--dangerously-skip-permissions flag needed.
2. Verbose per-step log line (topology_worker.rs):
Every checkpoint now writes to stderr:
topology_worker::step run_id=X step=N node=Y role=Z phase=W
output_bytes=B tokens=T gated=G
Visible in docker logs clawmates_server_1 — gives us live
'topology is flowing' signal without opening the canvas, and
makes it obvious when a topology_kind is skipping stages it
shouldn't.
3. Stuck-container reaper (topology_worker.rs):
New 60s-tick loop reap_stuck_runs: for any research topology_run
older than 15 min with zero checkpoint.records, docker-stop its
container and mark the run failed with a diagnostic error. Only
reaps research-bound runs (non-research runs don't own a
container). The existing 180s stale-checkpoint requeuer stays
in place for other failure modes.
Deploy: gw-04 needs
ln -sf /path/to/repo/deploy/claude-settings.json /opt/clawmates/claude-settings.json
CLAWMATES_CLAUDE_SETTINGS_PATH=/opt/clawmates/claude-settings.json
in the server env, plus the timeout lowered from 600 -> 120 in
compose. Both handled in the deploy step outside this commit.
|
||
 osobh
|
1d69866bcf | research canvas: collapsible sidebar + live topology-run logs (#6) | ||
|
Omar Sobh
|
13ead5cb9f | fmt: sort routes/mod.rs pub mod entries | ||
|
osobh
|
149ad992bb | wizard: materialize picked repo across clawstor fleet at step-2-next (#5) | ||
|
osobh
|
1cb643142c | research/publish: gate approve+reject on Owner role (#4) | ||
|
Omar Sobh
|
dd791061ee |
test(topology_jobs): seed research_outcome so transition test matches new invariant
Commit
|
||
|
Omar Sobh
|
212e68f6b1 |
research/probe: end-to-end smoke test endpoint (skip topics/loops/spawn)
New: POST /api/research/probe — one-shot pipeline probe against the
workspace's shared ZeroClaw gateway. Drives a trivial 'reply OK'
turn via ZeroClawDriveExecutor::drive and reports per-step timings.
Purpose: stop guessing what's broken in the wizard flow by exercising
JUST the executor→daemon→claude→response path. If probe succeeds
within a few seconds, we know:
- ZEROCLAW_TOKEN + gateway URL config is correct
- Daemon can reach and authenticate against claude
- The full ws round-trip works
…and every other failure we've been chasing (turn timed out, LLM
request failed, ws connect DNS error, etc.) is spawn-config or
prompt-size specific.
Request body (both optional):
{ "prompt": "Reply OK", "agent": "coordinator" }
Response:
{
"verdict": "ok" | "fail",
"total_duration_ms": …,
"prompt_len": …,
"response_preview": "OK",
"steps": [
{ name: "build_executor", duration_ms: …, status: "ok" },
{ name: "drive_turn", duration_ms: …, status: "ok", detail: "tokens=N, output_len=M" }
]
}
Wire-up:
- new routes/probe.rs
- pub mod probe in routes/mod.rs
- POST /api/research/probe registered in lib.rs router
- ZeroClawDriveExecutor::drive promoted from private to pub so the
probe handler can call it (behavior unchanged, other callers were
all inside the same struct).
Usage from anywhere (curl, browser dev tools, etc.):
curl -X POST https://clawmates.work/api/research/probe \
-H "Content-Type: application/json" \
-H "Cookie: <session cookie>" \
-d '{}'
Follow-up: a small frontend button (e.g. bottom of ResearchList) that
POSTs this and renders the response inline, so users don't need to
curl. Skipping in this commit to ship the useful part first.
|
||
|
Omar Sobh
|
3373f57da0 |
topology_exec: bump TURN_TIMEOUT to 700s to outlast daemon claude_cli
Verbose daemon logs revealed the real turn failure: the daemon's own claude_cli provider was timing out at 180s while running a 10-15k token coordinator prompt. The daemon reads its per-provider timeout from ZEROCLAW_providers__models__claude_cli__default__timeout_ms — we set that to 600000 (10 min) on gw-04 via a compose patch. Now the executor's TURN_TIMEOUT (was 300s) must exceed the daemon's own limit, or we kill the ws before the daemon can reply. New sequence: daemon has up to 600s to invoke claude and return a response; executor waits up to 700s (100s headroom) for the ws event stream to drain. If claude takes 500s, both survive. If the daemon really does hang past 600s, its own timeout fires first and we get a proper "provider timed out" error instead of a phantom executor timeout. Compose env applied on gw-04 in the same session: - ZEROCLAW_providers__models__claude_cli__default__timeout_ms=600000 - ZEROCLAW_providers__models__claude_cli__door__timeout_ms=600000 (backup: /opt/clawmates/docker-compose.yml.bak-timeout) Server container recreated to pick them up; env verified. Follow-up: the coordinator prompt is legitimately huge (autonomy contract + roster + description + repo tree + integration-plan template + operator notes = 10-15k tokens). We should consider either shrinking it or breaking the work into multiple smaller turns so the daemon isn't gambling on a single call taking 3-8 minutes. |
||
|
Omar Sobh
|
7e0620fd08 |
research: don't advance to reviewing without an outcome + guard-before-write
Two bugs that combine to produce the 409 you get when clicking Approve:
1) notify_run_completed (topology_worker post-hook) was advancing
topic status processing → reviewing whenever the last sibling run
terminated — success OR fail. A failed run with 0 outcomes still
pushed the topic to `reviewing`, the canvas rendered the "Request
publish" affordance, and the reviewer clicked Approve on nothing.
Fixed by adding an EXISTS(research_outcomes …) clause to the
UPDATE. Topic stays in `processing` when no outcome exists; the
loop's next iteration still has a chance to produce one.
2) decide_publish was calling
research_publish_approvals::decide(approve=true)
FIRST (which flips the row to `status='approved'`) and then
running the "no outcome? 409" guard SECOND. On the 409 return,
the DB was left half-flipped: approval says approved, topic still
in reviewing, no outcome exists, and every future click to the
same approval returns 409 on the "already decided" guard —
leaving reviewers with no way forward.
Fixed by moving the outcome-existence check BEFORE the decide()
call. On 409 now nothing was written, so the reviewer can try
again cleanly once an outcome is produced.
Also unstuck the current stuck row out-of-band (SQL UPDATE to reset
the approval to pending + topic to processing) so the user isn't
forced to delete the topic to escape the 409 loop.
sqlx dynamic query — the new EXISTS clause wasn't in the offline
cache so I switched notify_run_completed to plain `sqlx::query`.
|
||
|
Omar Sobh
|
ee05037095 |
research-pipeline-diag: state-aware statuses (waiting != failing)
The diagnostic was calling any 0-outcome + 0-completed-runs state a
FAIL — red dot + 'No outcome produced yet — check the runs stage for
the failure reason'. That fires the moment a wizard-materialized
topic lands, before its very first turn even completes, and stays
red for the whole 2-3 min a legitimate coordinator turn runs. Result:
users see 'FAIL' on every fresh topic and can't tell a real failure
from a normal in-flight state.
Backend fix — introduce a `waiting` status (blue/pulsing in UI):
Runs stage:
0 runs -> skip ('No runs yet — pipeline hasn't fired')
any running/queued -> waiting ('N in flight, M completed')
all failed -> fail (with error text)
some failed -> warn
all completed no fails -> ok
Outcomes stage:
outcome_count > 0 -> ok
0 outcomes + 0 runs -> skip ('No outcome yet (pipeline hasn't fired)')
0 outcomes + any running -> waiting ('Waiting for the current run to finish…')
0 outcomes + any failed -> fail (the actual silent-bug case)
0 outcomes + all done ok -> warn (weird — completed but wrote nothing)
Also suppresses the run stage's `latest_error` detail when the run
status is `waiting` or `skip` — reporting a stale error next to an
actively-running job is what made users think the current run had
failed.
Frontend:
- PipelineStage['status'] union grows a 'waiting' arm.
- Pill color: cyan (#5ec8d8) with a pulsing scale/opacity animation
(new cm-pulse keyframe in motion.css).
- Strip summary line: 'Pipeline in flight — waiting for run to finish…'
when there's any waiting stage and no failures.
- Border tint: cyan border when waiting, coral when failing, neutral
otherwise.
Zero backend semantic changes to the outcome-write path — this is
purely UI truth-telling.
|
||
|
Omar Sobh
|
6e06bcf136 |
research_container: spawn team daemons with --verbose
Without --verbose the daemon only emits boot + fatal-crash lines to stderr, so 'LLM request failed' 500s (webhook, /ws/chat) surface as opaque errors with no way to diagnose from `docker logs`. --verbose turns on per-request traces via the runtime's structured logger. Immediately paid off: recreating the current stuck team container with --verbose revealed 'agents.actor.model_provider resolves to a model_provider entry with no `model` set' which points at a missing config override (the shared runtime injects model = ... via ZEROCLAW_providers__models__groq__default__model on its own env; per-team containers don't get that env because it lives on the shared runtime container, not on clawmates_server_1). That's the next fix. Same --verbose is added to spawn_loop's per-loop cmd for symmetry. |
||
|
Omar Sobh
|
27b74e13d4 |
research_container: propagate ZAI_ + KIMI_ env into per-team runtime
Shared runtime's config points several providers at Z.AI's Anthropic proxy (`ANTHROPIC_AUTH_TOKEN = \"\$ZAI_API_KEY\"`) and the Kimi provider block needs KIMI_API_KEY. inherited_env only propagated ZEROCLAW_/OPENAI_/ANTHROPIC_/GEMINI_/GROQ_ prefixes — ZAI_ and KIMI_ were silently dropped, so per-team daemons booted with empty substitutions and every Z.AI-routed call died with 'LLM request failed' (500 from the daemon, executor timeout at 300s). Adds both to the prefix allowlist. Server container already has ZAI_API_KEY set from the compose env_file; KIMI_ will flow through too when we add it. Doesn't fully close today's incident — the current stuck run's webhook still 500s even with ZAI_API_KEY injected, so there's a second daemon-internal issue (need to boot the team daemon with --verbose to surface it). But the propagation fix is a real bug on its own and would silently break the moment the workspace's configured role calls into a ZAI-routed provider. |
||
|
Omar Sobh
|
bfcdca0583 |
research-canvas: managed-by-loop UI + start_topic guard (fold cleanup)
Closes the UX gap the fold introduced: the topic canvas was still showing "Start research" for standby-state topics even when a scheduled loop already owned the runs. Clicking it would 409 (or worse: race the loop into a duplicate run). Topic status stayed at standby forever because the loop path bypassed start_topic's set_status transition. Four changes: 1. **Backend status transition** — compose_and_enqueue_iteration for kind='research' now calls set_status_if(standby, processing) on the topic before the run is enqueued. New DB helper set_status_if only advances when the current status matches the "from" arg — safe against races and re-invocations. Later iterations no-op since the topic is already past standby. 2. **has_managed_loop on TopicDetail** — get_topic hydrates a new ManagedLoop struct (loop_id, title, enabled, next_fire_at, last_run_id, schedule_summary) when a kind='research' loop is bound to the topic. summarize_schedule() derives a human string from the loop's triggers jsonb (e.g. "cron: 0 3 * * * · on new artifact", "one-shot", "manual"). New DB helper loops::research_loop_for_topic returns the row. 3. **Canvas branch** — nextAction takes a managedByLoop flag; when set + status=standby, returns null (no button). The canvas renders a "MANAGED BY LOOP" strip below the topic title showing loop name, schedule summary, next fire time, and enabled dot. Reviewer buttons (Request publish / Approve / Reject) still show normally in later states — reviewers should still promote outcomes even when a loop is producing them. 4. **start_topic guard** — refuses with 409 when a research loop already owns the topic. Closes the direct-POST hole for anyone bypassing the frontend. TS type + summarize_schedule live in the same commit so an old client hitting a new backend just ignores the extra field (no breakage), and a new client hitting an old backend renders the classic buttons (managed_by_loop is optional). |
||
|
Omar Sobh
|
f910771bbb |
research-prompt: add AUTONOMY CONTRACT + relax citation rigidity
Inspection of a stuck 500s+ research run showed both agents were
correctly picking up the topic AND producing rich, structured plans —
then stalling at 'Should I proceed?' and 'Which approach?'. No human
to answer = infinite spin until stale-sweep + retry, forever.
Two prompt changes to compose_research_iteration_task:
1. AUTONOMY CONTRACT at the top:
- Explicit "no human will answer you"
- Explicit "do NOT ask for confirmation"
- Explicit "you MUST emit the completed artifact"
- Framed as a contract, at the very top, before the task itself.
2. Softer citation rule:
- Was: 'Cite every claim; never fabricate sources or repo paths.'
- Now: 'Cite what you can verify. Use [claim needs verification]
inline when you can't. A written v(N) with rough citations
beats a blocked v(N) waiting for approval. Do not fabricate
concrete titles/authors/DOIs.'
The absolute anti-fabrication rule made agents refuse to write
anything unless they could be sure. Coupled with no user available,
the whole loop stalled — turns completed successfully but no output
converged.
Follow-up if this repeats:
- Even shorter prompt (the current one is 300+ words)
- Explicit output-shape enforcement ('respond with only the markdown
artifact, no preamble')
- Cap iteration count so a broken prompt doesn't burn tokens forever
|
||
|
Omar Sobh
|
63305689be |
topology_exec: bump TURN_TIMEOUT to 300s
Turn timeout was 90s. Research coordinator turns run 60-120s routinely (long prompt + cold tool selection) — 90s was tripping legitimate turns while the daemon was still working, throwing away completed inference. 300s gives real turns room while still bounding worst-case at a walk-away limit. Latest wizard-created topic hit this precisely: bridge attached, handshake worked, claude auth verified in the container, but the run's status went to failed with 'turn executor failed: turn timed out' after exactly ~90s. The topology worker's stale-run sweep at 180s covers the case where the worker itself dies mid-turn — a 300s turn still checkpoints every step so a stuck worker will get requeued. |
||
|
Omar Sobh
|
a0fb64ac47 |
clippy: use bollard::models::NetworkConnectRequest (deprecation)
CI's clippy step failed on commit
|
||
|
Omar Sobh
|
194d63ef36 |
research_container: attach bridge network so per-team egress works
Per-team containers were spawned onto clawmates_core only. That
network is Internal=true on gw-04 (no default gateway to the host's
default route), so any egress attempt — including the daemon's
own claude/gemini/groq API calls — fails with FailedToOpenSocket
and the turn times out.
The shared clawmates-runtime is attached to BOTH clawmates_core AND
the default bridge (that's how it can hit api.anthropic.com); the
per-team containers were missing the second network.
Fix: after start_container succeeds, best-effort connect the
container to `bridge` too. Both spawn() and spawn_loop() call the
same helper. Idempotent — a 403 from Docker on repeat-attach ("already
on network") is silently ignored.
Verified out-of-band on the current stuck research team container:
- Manually `docker network connect bridge research-<id>-team`
- `docker exec ... claude --print "reply only: ok"` → returned "ok"
- Auth + egress both working, so next iteration should complete.
Sequence of pipeline fixes finally converging:
1. materialize_topic_loops didn't fire burst → fixed by
fire_initial_burst_if_set
2. research iteration skipped clone/spawn → fixed by
prepare_topic_runtime
3. graph parse failed → fixed by build_topic_graph_json
4. bind-mount perms wrong → chown 65532:65532
5. server image missing git → debian:12-slim base
6. daemon required pairing → prewrite_daemon_config
7. daemon rejected unknown agents → template config from shared runtime
8. THIS: no external egress → attach bridge post-start
|
||
|
Omar Sobh
|
fd82fe6762 |
research_container: template daemon config from shared runtime
Per-team daemon boots with require_pairing=false but no
`[agents.*]` sections. Server's ws connect authenticates fine and
then trips over "Unknown agent `coordinator` — no [agents.coordinator]
entry configured." (400).
Fix: prewrite_daemon_config now reads a template config from
CLAWMATES_RUNTIME_TEMPLATE_CONFIG (default
/var/lib/clawmates-runtime-template/config.toml) which mirrors the
shared clawmates-runtime container's config with all agent + provider
sections. We strip the template's [gateway] block (its paired_tokens
list is encrypted with the shared runtime's key and un-decryptable
per-team) and prepend a fresh [gateway] require_pairing = false.
Falls back to a minimal pairing-off config with a loud eprintln when
the template isn't readable — the log line makes the misconfig
visible instead of silently 400-ing.
Verified out-of-band on the current stuck team container:
- Restarted with the shared config + rewritten [gateway] section
- Daemon boots cleanly, logs "Pairing: DISABLED (all requests accepted)"
- ws /ws/chat handshake returns 101 Switching Protocols + session_start
message (auth working end-to-end)
gw-04 deploy step (already applied):
1. sudo mkdir -p /var/lib/clawmates-runtime-template
2. sudo cp /root/clawmates-runtime/data/.zeroclaw/config.toml \
/var/lib/clawmates-runtime-template/config.toml
3. sudo chown -R 65532:65532 /var/lib/clawmates-runtime-template
4. Compose: added
/var/lib/clawmates-runtime-template:/var/lib/clawmates-runtime-template:ro
+ CLAWMATES_RUNTIME_TEMPLATE_CONFIG env
to /opt/clawmates/docker-compose.yml
(backup: docker-compose.yml.bak-template)
Follow-up: expose a "reload template" endpoint or re-copy the shared
config on each server boot so we don't drift when the shared runtime
adds a new agent.
|
||
|
Omar Sobh
|
4be3e43f6f |
research_container: prewrite daemon config with require_pairing=false
Per-topic and per-loop team containers spawn from clawmates-runtime with an empty daemon config, so they boot with require_pairing=true and an empty paired_tokens store. Every incoming ws connect from the API server got 401 Unauthorized because the server's ZEROCLAW_TOKEN wasn't in that store. The shared clawmates-runtime container has a paired_tokens list maintained out-of-band (encrypted enc2:… entries in /root/clawmates-runtime/data/.zeroclaw/config.toml on gw-04). That list isn't portable to freshly-spawned per-team containers — the tokens are encrypted with a key we don't share, and pairing new tokens requires a pairing code we don't generate. Simplest correct answer: per-team containers are ephemeral, live on the private clawmates_core docker network, and only accept traffic from the API server. Disabling pairing there closes zero security holes. New helper prewrite_daemon_config(state_root): - creates <state_root>/.zeroclaw/ - writes config.toml with schema_version=3 + [gateway] require_pairing=false - skips write when config.toml already exists so a manually-paired team container survives re-spawn Called from both spawn() (per-topic) and spawn_loop() (per-loop) right after ensuring the host state dir. Existing paired containers are unaffected; new ones come up open-network to the compose stack. Verified out-of-band by writing the same config into the current stuck team container and restarting it — daemon health flipped require_pairing from true to false. |
||
|
Omar Sobh
|
2df9dd04df |
research: build real topology graph in materialize_topic_loops
The wizard-created research loop's first iteration failed with
"missing or invalid graph" — materialize_topic_loops was writing
`{nodes: [], edges: []}` as a placeholder, which the topology worker
rejects. Also explains why prepare_topic_runtime hadn't cloned the
repo or spawned the container: the run failed before
compose_and_enqueue_iteration got to call it.
Fix: extract build_topic_graph_json() into research_setup.rs — same
shape start_topic uses (hydrate roster, promote a role_slot-tagged
coordinator to index 0 for hub_spoke/hierarchical/star_moe, build
via cm_topology::build, serialize via cm_topology::to_json). Called
from materialize_topic_loops instead of the empty placeholder.
Fully best-effort. Any DB/topology failure falls back to a
single-node hub graph so the loop still runs (degraded, but not
silently broken).
The next wizard-created topic should now:
1. Materialize the research loop with a valid graph
2. Fire the initial burst
3. compose_and_enqueue_iteration calls prepare_topic_runtime →
clones the repo, spawns the container
4. Enqueues a run whose graph parse succeeds
5. Run drives to completion, produces an outcome
|
||
|
Omar Sobh
|
51756d0e68 |
clippy: struct-bundle enqueue_iteration_with_topic + fix doc list warnings
CI's clippy stage failed with -D warnings on three classes of lint:
1. map_clone in loops::recent_reorders — .map(|a| a.clone()) is the
pattern clippy wants replaced by .cloned(). Trivial swap.
2. too_many_arguments on enqueue_iteration_with_topic (8 args, ceiling
7). Refactored the caller-side surface into a new
IterationEnqueue<'a> struct with fields for each column. Matches
the pattern research::NewTopic + loops::NewLoop already use for
the same clippy ceiling. Callers in loops.rs (routes) + the
enqueue_iteration wrapper updated to build the struct literal.
3. doc_lazy_continuation + doc_list_indentation — two doc comment
blocks used ambiguous list-like layouts:
- compose_iteration_task's ASCII-art prepended-block preview:
wrapped in a ```text fence so clippy stops parsing "RESEARCH
ARTIFACT:" etc as list continuation.
- fire_initial_burst_if_set's mention of `burst - 1`: rewrote so
"- 1" doesn't start a line and get misread as list marker.
Purely a refactor + doc pass; no behavior change.
|
||
|
Omar Sobh
|
c0c5fd7104 |
research: extract setup helpers into research_setup.rs (fixes file-size gate)
research.rs hit 1446 lines with the wizard-fold + regression-fix
commits, tripping the CI file-size gate (limit 1250). Pure extraction
into a sibling module; no behavior change.
Moved to routes/research_setup.rs:
- RepoContext (now pub — used by build_coordinator_task)
- TopicSchedule (now pub — request body sub-struct)
- research_workspace_root (now pub)
- prepare_topic_runtime (already pub — used by loops iteration path)
- ensure_repo_workspace (now pub — used by start_topic + prepare)
- materialize_topic_loops (now pub — used by create_topic)
research.rs re-imports them via `use crate::routes::research_setup::{...}`
so the calling code reads identically.
New line counts:
research.rs 1127 lines (was 1446, limit 1250)
research_setup.rs 321 lines (new)
Also updated the loop-iteration callsite in routes/loops.rs to point
at the new path (crate::routes::research_setup::prepare_topic_runtime).
No API changes; migrations unaffected.
|
||
|
Omar Sobh
|
80c23e57ed |
research: fix wizard loops never firing + missing clone/spawn (regression)
Two bugs surfaced by the first end-to-end wizard run — the pipeline diagnostic showed "0 run(s)", "Repo bound but never cloned", and "Container not spawned" for a topic that had been created with a nightly research loop. Bug 1: materialize_topic_loops bypassed initial_burst firing. The wizard-materialized loops path calls cm_db::repo::loops::create directly (a plain INSERT). The initial_burst-fires-first-iteration logic lived inside the create_loop HTTP handler, so wizard loops landed in the DB but never fired their initial iteration. Fix: extract routes::loops::fire_initial_burst_if_set as a pub helper that read triggers, ensures the loop container, and calls the kind-aware compose_and_enqueue_iteration. Both create_loop and materialize_topic_loops now call it. Bug 2: research-kind loop iterations skipped clone/spawn. compose_research_iteration_task only built the coordinator prompt; the repo clone and topic container spawn lived only in start_topic. So the first research iteration ran against a nonexistent clone directory and a stale gateway, and every run failed. Fix: extract routes::research::prepare_topic_runtime as a pub helper that runs ensure_repo_workspace + research_container::spawn. Idempotent — second iteration reattaches. Called from compose_and_enqueue_iteration before enqueuing a research iteration. Topics without a repo bound are a no-op. Both fixes ship as one commit because they surface together on the same user path (wizard → research loop → first iteration) — you can't hit one without the other manifesting. Follow-up: start_topic still runs its own inline clone/spawn code (now duplicated with prepare_topic_runtime). Next commit collapses start_topic to just call prepare_topic_runtime + build_task like the loop path does, so the one-shot and loop paths agree on setup. |
||
|
Omar Sobh
|
3e42b1ea39 |
research-wizard: schedule step (once/nightly/manual) + paired coding loop
Completes the research/loop fold. The wizard now closes with a
"How should this research run?" step; picking any mode materializes a
kind='research' loop bound to the topic, and an optional checkbox
adds a paired kind='exec' loop that consumes each new artifact.
Frontend:
- ResearchWizard grows from 5 to 6 steps. Step 6 is the schedule
picker:
· Just once — initial_burst=1, no other triggers
· Nightly — initial_burst=1 + cron "0 3 * * *"
· Manual — webhook_enabled=true
Below the radios, an optional card offers "Also create a coding
loop that consumes each new artifact" — creates a paired
kind='exec' loop with on_artifact_update=true + initial_burst=1
bound to the same topic.
- createTopic API type extended with `schedule` + `create_paired_coding_loop`.
- Both fields ride the existing POST /api/research call; back-compat
is preserved when the wizard omits them.
Backend:
- CreateTopicRequest gains TopicSchedule + create_paired_coding_loop.
- After topic + agent attach, create_topic calls
materialize_topic_loops which:
1. Creates a research-kind loop titled "Research · <topic>" bound
to the topic. Triggers vary by schedule mode; next_fire_at
computed from cron for nightly. Falls back silently if
loop-create errors so the topic still lands.
2. Flips kind to 'research' via loops::set_kind (NewLoop doesn't
take kind directly — default is 'exec' for backward compat).
3. Optionally creates a coding loop titled "Coding · <topic>"
with on_artifact_update=true + initial_burst=1.
- cm_db::repo::loops::set_kind — trivial UPDATE helper used by the
materialize path.
D-answer callouts:
- D1 (fold): every runnable thing is now a loop. "Just once" is a
research loop with initial_burst=1 and no other triggers.
- D2 (inherit): both paired loops carry the same source_research_topic_id
— repo binding lives on the topic, not duplicated.
- D3 (coordinator resolves): the research iteration prompt (from the
earlier commit) instructs the team to preserve stable INT ids and
mark deprecations; coding loops' consumed lists stay valid across
versions.
Follow-ups queued:
- Kind pill on LoopsList cards (research=purple, exec=cyan) so users
can tell them apart at a glance.
- Extract start_topic's task-build so kind='research' iterations
reuse the same coordinator prompt shape as one-shot runs (they
currently use a simpler refresh-oriented prompt; that's fine for
MVP but a rich shared build would give better parity).
- Research topic sidebar shows "linked to N loops" badge.
|
||
|
Omar Sobh
|
91a51dce11 |
loops: kind='research' dispatch — research runs as loops
Delivers the research/loop fold: kind='research' loops run the research pipeline each iteration, appending a new research_outcomes version. The paired on_artifact_update fan-out then wakes up any kind='exec' loops bound to the same topic to consume new INTs. Every runnable thing is now a loop (D1). Backend — DB helpers: - cm_db::repo::loops::kind_and_binding — reads (kind, source_topic, task_template) so callers can dispatch without hydrating the whole Loop struct. - cm_db::repo::loops::enqueue_iteration_with_topic — new variant that sets research_topic_id on topology_runs alongside loop_id, so the completion hook's freeze_research_outcome writes a new outcome version for research-kind iterations. - cm_db::repo::research_topics::get_any_workspace — cross-workspace fetch used by the research task builder (the loop row is authoritative for the workspace binding via kind_and_binding). Backend — dispatch: - routes::loops::compose_research_iteration_task — builds the coordinator prompt for a research iteration: topic title + description + outcome_kind + prior version pointer + refresh instructions (survey new sources, preserve stable INT ids, mark superseded items as deprecated rather than delete). The completion hook writes the resulting synthesis as research_outcomes v(prior+1). - routes::loops::compose_and_enqueue_iteration — one-shot dispatch: reads the kind, picks compose_iteration_task (exec) or compose_research_iteration_task (research), enqueues with or without research_topic_id set. All four enqueue callsites now route through compose_and_enqueue: - create_loop (initial_burst) - run_now - webhook_receive - topology_worker::continue_initial_burst - topology_worker::freeze_research_outcome (on_artifact_update fan-out) Research-kind loops naturally form the "nightly refresh" side of a paired research + coding loop: research writes a fresh outcome version → fan-out wakes exec loops with on_artifact_update → coding loops consume the next INT (which the research loop may have just added). D2 answer (inherit repo binding): repo lives on the topic; both loops sharing the source topic id read from the same context, no duplication. D3 answer (coordinator resolves): the research iteration prompt tells the team to preserve stable INT ids and mark deprecations rather than delete, so coding loops' consumed lists stay valid across versions. Follow-up (next commit): ResearchWizard schedule step — "Just once / Nightly / Manual" that creates the paired research-kind loop with initial_burst=1 (just once) or cron 0 3 * * * (nightly) + optional paired coding loop with on_artifact_update. |
||
|
Omar Sobh
|
4ada5557f2 |
loops: kind column + initial_burst + on_artifact_update trigger fan-out
Foundation for folding research into loops as a first-class kind.
This commit ships the plumbing; the research-kind dispatch itself
lands next. Behavior for existing exec-kind loops is unchanged unless
they opt into the new trigger fields.
Migration 0044:
- kind TEXT NOT NULL DEFAULT 'exec' CHECK ('exec' | 'research'). New
research-kind will run the research pipeline each iteration (next
commit); 'exec' preserves today's behavior.
- initial_burst_remaining INT NOT NULL DEFAULT 0 — countdown for the
triggers.initial_burst quota. Decremented CAS-safely on each
completion until it hits 0.
- Two partial indexes: (kind, source_research_topic_id) for kind-
aware lookups, and (source_research_topic_id) filtered on
on_artifact_update=true + enabled=true for the fan-out hook.
Trigger schema extended with two optional fields:
- initial_burst: N — fire N iterations back-to-back at create time.
create_loop enqueues the first iteration inline (subject to
empty-roster gate), sets remaining=N-1, and the completion hook
continues the chain until exhausted.
- on_artifact_update: true — when a bound research_outcomes row is
inserted for the source topic, wake up one iteration of this loop.
Coalesced against has_active_run so a burst of rapid revisions
doesn't queue duplicates.
Backend:
- cm_db::repo::loops helpers (all dynamic sqlx, no offline cache
regen needed):
- set_initial_burst_remaining
- take_initial_burst_slot (CAS UPDATE returning prev value; 0 on
exhausted or race loss)
- loops_awaiting_topic (fan-out query: kind=exec + enabled +
on_artifact_update=true bound to the given topic)
- has_active_run (queued|running iteration existence check)
- get_any_workspace (bypasses the workspace scope guard; used by
the completion hook where the run row is authoritative)
- routes/loops::compose_iteration_task made pub so the completion
hook can build the same enriched task string as run_now.
- topology_worker::freeze_research_outcome now fans out to awakened
loops after the outcome insert, using compose_iteration_task and
coalescing on has_active_run.
- topology_worker::continue_initial_burst runs on every completion:
· take_initial_burst_slot (CAS) — no-op if already exhausted
· has_active_run coalesce guard
· re-fetches the loop via get_any_workspace + compose_iteration_task
· enqueues via loops::enqueue_iteration with parent_run_id set
Follow-ups already queued:
- kind='research' dispatch in run_job — build the research
coordinator task from the topic config, run the research pipeline
each iteration. Requires factoring start_topic's task-build.
- ResearchWizard "When should this run?" step (Just once / Nightly /
Manual) creating the topic + paired research-kind loop.
- LoopsWizard trigger UI matching the design proposal (burst count,
cron, on-artifact checkbox).
|
||
|
Omar Sobh
|
ce111273bb |
loops: reorder history collapsed under the progress pill
Completes the reorder rationale loop — the previous commit captured
REORDER: markers server-side but nothing surfaced them. Now the loops
sidebar renders a small "N reorders ▸" button under the progress pill
whenever the loop has any reorder events. Click expands to a
newest-first list of "iter <n> · <rationale>" lines so a reviewer can
see, at a glance, when the plan was adjusted and why.
Backend:
- LoopProgress DTO gains recent_reorders: Vec<Value> — newest-first,
capped at 5 so the card stays compact. Full history remains on the
loop row's reorder_events column.
- cm_db::repo::loops::recent_reorders — reads the jsonb array, returns
the last N in newest-first order.
- list_progress populates it per loop.
Frontend:
- LoopReorderEvent + recent_reorders on LoopProgress type.
- LoopsList tracks openHistoryId per-loop (one open at a time).
- Card renders history button + expanded panel styled to match the
progress pill above.
Notes:
- The event object schema is {run_id, iteration, text, ts}. Fields are
optional in the TS type so future schema tweaks don't break the
render.
- 5-item cap chosen so the sidebar card doesn't grow unbounded. If a
loop accumulates a lot of reorders, follow-up UI can render the full
history on the loop detail page.
|
||
|
Omar Sobh
|
0c17de52dd |
loops: reorder rationale extraction — REORDER: markers logged per iteration
Coordinator can now log WHY it worked on an INT-XX out of order
("REORDER: INT-05 before INT-04 because prereq X is unmet") and the
completion hook captures each rationale as an append-only event on
the loop. Sets up a reviewable timeline of when the plan was
adjusted, independent of the underlying `consumed_int_ids` advance.
Migration 0043:
- loops.reorder_events JSONB NOT NULL DEFAULT '[]'::jsonb — append-
only array of {run_id, iteration, text, ts}. Kept on the loop row
(rather than a dedicated table) so the mini-timeline is one read
away from the loop card.
Backend:
- topology_worker::parse_reorder_rationale — line matcher symmetric
with parse_completed_int_ids. Tolerates list dashes / prefixes /
markdown emphasis; case-insensitive marker match, preserves case of
the rationale text.
- cm_db::repo::loops::append_reorder_event — one INSERT-like append
per rationale, uses jsonb_build_object with postgres now() so ts is
wall-clock canonical (no client-clock skew).
- topology_runs::iteration_for_run — new helper so events carry the
iteration index.
- routes::loops::compose_iteration_task — coordinator prompt now
explicitly asks for `REORDER: <one-sentence>` at the top of the
first substantive turn when working out of order, AND spells out
that both markers must appear literally with colons (no bold, no
code fence) so the line parser doesn't miss them.
Non-loop and standalone-loop runs are unaffected — the hook only
fires when the run belongs to a source-bound loop.
Follow-up: expose reorder_events on the loops list endpoint + render
a small collapsed timeline on the LoopsList card.
|
||
|
Omar Sobh
|
a34be33261 |
loops: N/M INTs progress pill on source-bound loop cards
Users couldn't see how a research-bound loop was progressing through
its integration plan — the consumed_int_ids state existed in the DB
but nothing surfaced it. Now each loop card in the sidebar shows a
cyan pill "3/8 INTs" + source topic title + a thin progress bar, when
the loop is bound to a research topic.
Backend:
- GET /api/loops/progress — bulk read for every loop with a source
topic. Returns {loop_id, source_topic_id, source_topic_title,
source_outcome_version, consumed_count, total_int_count,
current_int_index} per loop. Standalone loops are omitted.
- count_int_ids parses unique INT-<number> ids out of the source
outcome's markdown — same permissive matcher as the completion
hook, so what the pill counts matches what the loop can advance.
- Memoized by topic_id inside the endpoint so N loops sharing 1
source topic only fetch the outcome once.
Frontend:
- listLoopProgress helper + LoopProgress type in the loops API.
- LoopsList fetches loops + progress in parallel on mount.
- Each card looks up progress by loop_id and, when found, renders
under the schedule line: pill "3/8 INTs · Topic title" with a
3px cyan progress bar. Title hover shows artifact version.
Follow-ups:
- Refresh button on the card — re-snapshot artifact into
task_template (cosmetic; the enqueue path already reads latest).
- Reorder rationale extraction — parse "REORDER: <text>" out of run
output, index as a per-loop event log for a mini-timeline UI.
|
||
|
Omar Sobh
|
4a140cb7db |
loops: parse COMPLETED: INT-XX markers to advance loop pointer (option b)
Closes the loop bridge — the missing piece from the previous commit.
Without this, `current_int_index` stayed at 0 forever and every
iteration re-worked INT-01. Now the topology_worker's completion hook
parses the run's final output for `COMPLETED: INT-<NN>` markers and
atomically advances the loop's consumed_int_ids + current_int_index.
Backend:
- topology_worker::advance_loop_after_completion — new post-terminal
hook that fires alongside freeze_research_outcome. Reads loop_id_for_run
(skips non-loop runs) + source_research_context (skips standalone
loops without a bound source topic).
- parse_completed_int_ids — forgiving parser: matches `COMPLETED: INT-01`,
`- COMPLETED: `INT-01``, `COMPLETED: INT-01, INT-02`, case-insensitive,
tolerates list dashes / backticks / markdown emphasis. De-dupes within
a single output.
- cm_db::repo::loops::advance_after_completion — atomic UPDATE that:
· appends only NEW ids to consumed_int_ids (idempotent on re-runs)
· bumps current_int_index by the count of new ids landed
Set semantics via `SELECT DISTINCT unnest(...)` so ordering-based
bugs can't accumulate duplicates.
Behavior end-to-end:
1. Loop wizard imports an integrations artifact (previous commit).
2. run_now / webhook_receive → compose_iteration_task prepends artifact
+ focus instruction ("address INT-<current+1>, log COMPLETED at end").
3. Coordinator run does the work, emits `COMPLETED: INT-<NN>`.
4. topology_worker completion hook parses the marker, advances the
loop, and the NEXT iteration sees an updated `consumed:` list +
incremented `current_int_index` in its focus instruction.
Follow-ups still queued:
- Loop card refresh button — pull latest artifact after reject-with-
revision on the source topic (right now the prepend uses the LATEST
outcome automatically, so refresh is UX only, not correctness).
- Reorder rationale extraction — coordinator emits "REORDER: INT-05
before INT-04 because prereq X is unmet"; today that's just prose
in the output, not indexed.
|
||
|
Omar Sobh
|
ce73abe5ab |
loops: bridge research artifact into loop iterations (option C + b)
The bridge lets a coding loop "consume" an integrations research artifact one INT-XX item per iteration. Options b (order-sequential iteration) and C (snapshot in task_template + save the pointer for future refresh) from the design discussion. Migration 0042 — three new loops columns: - source_research_topic_id — nullable pointer to research_topics. - consumed_int_ids TEXT[] — INT-XX ids the loop has completed. Advances when topology_worker parses "COMPLETED: INT-<NN>" markers from the run's final output (wired in a follow-up commit). - current_int_index INT — monotonic pointer for order-sequential iteration. Coordinator addresses INT-<current+1> unless prereqs are unmet, in which case it works on the smallest unblocking INT-XX and logs the reorder rationale. Backend: - cm_db::repo::loops::set_source_research_topic — bind/unbind pointer. - cm_db::repo::loops::source_research_context — read pointer + state. - routes::loops::compose_iteration_task — new caller-side helper that reads the pointer, fetches the topic's latest research_outcome, and prepends the artifact + focus instruction to task_template. - run_now + webhook_receive both pass task_template through compose_iteration_task before enqueue. Standalone loops (no pointer) behave identically to before. - CreateLoopRequest accepts `source_research_topic_id`, ownership- checked via research_topics::get before persist. Frontend: - New ResearchArtifactPicker modal — lists published topics, fetches the artifact on pick, returns (topic_id, markdown) to caller. - LoopsWizard task_template step gains "Import from research artifact" button (right-aligned). Click opens the picker. On pick: task populates with the artifact markdown, pointer saved, textarea expands to 8 rows, small info strip shows "Loop is bound to topic <id>. Each iteration will focus on the next unconsumed INT-XX." - Unlink button reverts to standalone loop mode. Follow-up (next commit): - topology_worker completion hook — parse "COMPLETED: INT-<NN>" out of the run's final output + update consumed_int_ids + current_int_index atomically. Without this, current_int_index stays at 0 forever and every iteration works on the same INT. - Loop card refresh button — re-read source topic's latest outcome (useful after a reject-with-revision cycle on the source topic). |