8 Commits

Include renames

Author	SHA1	Message	Date
Omar Sobh	0c17de52dd	loops: reorder rationale extraction — REORDER: markers logged per iteration ... Coordinator can now log WHY it worked on an INT-XX out of order ("REORDER: INT-05 before INT-04 because prereq X is unmet") and the completion hook captures each rationale as an append-only event on the loop. Sets up a reviewable timeline of when the plan was adjusted, independent of the underlying `consumed_int_ids` advance. Migration 0043: - loops.reorder_events JSONB NOT NULL DEFAULT '[]'::jsonb — append- only array of {run_id, iteration, text, ts}. Kept on the loop row (rather than a dedicated table) so the mini-timeline is one read away from the loop card. Backend: - topology_worker::parse_reorder_rationale — line matcher symmetric with parse_completed_int_ids. Tolerates list dashes / prefixes / markdown emphasis; case-insensitive marker match, preserves case of the rationale text. - cm_db::repo::loops::append_reorder_event — one INSERT-like append per rationale, uses jsonb_build_object with postgres now() so ts is wall-clock canonical (no client-clock skew). - topology_runs::iteration_for_run — new helper so events carry the iteration index. - routes::loops::compose_iteration_task — coordinator prompt now explicitly asks for `REORDER: <one-sentence>` at the top of the first substantive turn when working out of order, AND spells out that both markers must appear literally with colons (no bold, no code fence) so the line parser doesn't miss them. Non-loop and standalone-loop runs are unaffected — the hook only fires when the run belongs to a source-bound loop. Follow-up: expose reorder_events on the loops list endpoint + render a small collapsed timeline on the LoopsList card.	2026-07-09 18:58:55 -07:00
Omar Sobh	4a140cb7db	loops: parse COMPLETED: INT-XX markers to advance loop pointer (option b) ... Closes the loop bridge — the missing piece from the previous commit. Without this, `current_int_index` stayed at 0 forever and every iteration re-worked INT-01. Now the topology_worker's completion hook parses the run's final output for `COMPLETED: INT-<NN>` markers and atomically advances the loop's consumed_int_ids + current_int_index. Backend: - topology_worker::advance_loop_after_completion — new post-terminal hook that fires alongside freeze_research_outcome. Reads loop_id_for_run (skips non-loop runs) + source_research_context (skips standalone loops without a bound source topic). - parse_completed_int_ids — forgiving parser: matches `COMPLETED: INT-01`, `- COMPLETED: `INT-01``, `COMPLETED: INT-01, INT-02`, case-insensitive, tolerates list dashes / backticks / markdown emphasis. De-dupes within a single output. - cm_db::repo::loops::advance_after_completion — atomic UPDATE that: · appends only NEW ids to consumed_int_ids (idempotent on re-runs) · bumps current_int_index by the count of new ids landed Set semantics via `SELECT DISTINCT unnest(...)` so ordering-based bugs can't accumulate duplicates. Behavior end-to-end: 1. Loop wizard imports an integrations artifact (previous commit). 2. run_now / webhook_receive → compose_iteration_task prepends artifact + focus instruction ("address INT-<current+1>, log COMPLETED at end"). 3. Coordinator run does the work, emits `COMPLETED: INT-<NN>`. 4. topology_worker completion hook parses the marker, advances the loop, and the NEXT iteration sees an updated `consumed:` list + incremented `current_int_index` in its focus instruction. Follow-ups still queued: - Loop card refresh button — pull latest artifact after reject-with- revision on the source topic (right now the prepend uses the LATEST outcome automatically, so refresh is UX only, not correctness). - Reorder rationale extraction — coordinator emits "REORDER: INT-05 before INT-04 because prereq X is unmet"; today that's just prose in the output, not indexed.	2026-07-09 18:48:34 -07:00
Omar Sobh	ce73abe5ab	loops: bridge research artifact into loop iterations (option C + b) ... The bridge lets a coding loop "consume" an integrations research artifact one INT-XX item per iteration. Options b (order-sequential iteration) and C (snapshot in task_template + save the pointer for future refresh) from the design discussion. Migration 0042 — three new loops columns: - source_research_topic_id — nullable pointer to research_topics. - consumed_int_ids TEXT[] — INT-XX ids the loop has completed. Advances when topology_worker parses "COMPLETED: INT-<NN>" markers from the run's final output (wired in a follow-up commit). - current_int_index INT — monotonic pointer for order-sequential iteration. Coordinator addresses INT-<current+1> unless prereqs are unmet, in which case it works on the smallest unblocking INT-XX and logs the reorder rationale. Backend: - cm_db::repo::loops::set_source_research_topic — bind/unbind pointer. - cm_db::repo::loops::source_research_context — read pointer + state. - routes::loops::compose_iteration_task — new caller-side helper that reads the pointer, fetches the topic's latest research_outcome, and prepends the artifact + focus instruction to task_template. - run_now + webhook_receive both pass task_template through compose_iteration_task before enqueue. Standalone loops (no pointer) behave identically to before. - CreateLoopRequest accepts `source_research_topic_id`, ownership- checked via research_topics::get before persist. Frontend: - New ResearchArtifactPicker modal — lists published topics, fetches the artifact on pick, returns (topic_id, markdown) to caller. - LoopsWizard task_template step gains "Import from research artifact" button (right-aligned). Click opens the picker. On pick: task populates with the artifact markdown, pointer saved, textarea expands to 8 rows, small info strip shows "Loop is bound to topic <id>. Each iteration will focus on the next unconsumed INT-XX." - Unlink button reverts to standalone loop mode. Follow-up (next commit): - topology_worker completion hook — parse "COMPLETED: INT-<NN>" out of the run's final output + update consumed_int_ids + current_int_index atomically. Without this, current_int_index stays at 0 forever and every iteration works on the same INT. - Loop card refresh button — re-read source topic's latest outcome (useful after a reject-with-revision cycle on the source topic).	2026-07-09 18:47:17 -07:00
Omar Sobh	44d6e95022	fmt: apply rustfmt across the P2 arc ... CI's rustfmt check flagged the multi-line sqlx::query() calls I introduced in P2 (loop_id_for_run, zeroclaw_gateway_url, etc.). No behavior change — pure formatting.	2026-07-09 16:43:05 -07:00
Omar Sobh	03f1830d1f	loops: Path B container isolation (P2) ... Symmetric with the research pipeline: every enabled loop can now have its own per-loop team container so scheduled runs don't share state with other loops or with research. Same daemon image, same clawmates network, deterministic name loop-<id>-team. Backend surface: - Migration 0040 adds nullable `zeroclaw_container` + `zeroclaw_gateway_url` columns to loops (parallel to research_topics). - research_container.rs grows loop_container_name_for(), spawn_loop() (state-only mount, no repo), and teardown_loop(). Kept in the same module to share the docker connect() + inherited_env() plumbing; each pattern gets its own labels (clawmates.role=loop-team) so ps filters can tell them apart. - cm_db::repo::loops gains set_zeroclaw_container() + zeroclaw_gateway_url() (dynamic sqlx queries — no offline cache regen needed). - cm_db::repo::topology_runs gets loop_id_for_run(): mirror of research_topic_id, used by the worker. Wiring: - routes/loops::run_now + webhook_receive call ensure_loop_container() before enqueuing an iteration. Idempotent: an already-running container is just reattached. Failures are logged and do NOT block the enqueue — topology_worker falls back to the workspace gateway when the URL isn't set on the loop. - routes/loops::disable_loop + delete_loop both fire teardown_loop() so paused / deleted loops don't hold a docker slot. - topology_worker's per-run URL resolution: existing research fast path unchanged; when it doesn't hit, the worker now looks up loop_id and reads the loop's gateway URL. Deploy step (required on gw-04 for state to persist across container restarts): add a `/var/lib/clawmates-loops:/var/lib/clawmates-loops` bind mount + `CLAWMATES_LOOPS_STATE_ROOT=/var/lib/clawmates-loops` env var to clawmates_server_1 in the compose. Without it, loops still run — the state dir lives inside the API container's filesystem so persistence is limited to that container's lifetime. Follow-up: - Scheduler-tick fires (cron-driven, not run_now) — they call enqueue_iteration in cm-scheduler and don't yet go through ensure_loop_container. Add a symmetric spawn there so cron fires also land on the isolated daemon. - Compose file reconciliation — deploy/compose/docker-compose.yml in the repo has drifted from prod; when we sync it, add the loops mount at the same time.	2026-07-09 16:02:08 -07:00
Omar Sobh	c063d4bbdd	loops: cargo fmt --all (unblock CI)	2026-07-08 05:45:06 -07:00
Omar Sobh	a6da19430f	loops: repo picker + agent/team/org staffing + sidebar edit/delete ... Adds the missing pieces the wizard needed and the sidebar controls around it: - LoopsWizard is now a 6-step flow (identity → repo → task/topology → triggers → repeat → assign agents) plus the existing secrets card. ResearchWizard picks up the same repo step and a hard gate when the workspace has zero agents. - New LoopStaffingStep with three tabs — Individual / Team / Organization — that mix freely per loop; selections persist via new loop_agents / loop_teams / loop_orgs join tables (0035 migration), each cascading on loop_id so hard-delete stays a single-row DELETE. - Backend CreateLoopRequest / UpdateLoopRequest accept the three lists and apply_staffing does a transactional replace-all; list_loops / get_loop hydrate the lists via a flattened LoopWithStaffing response. - LoopsList sidebar gains per-row enable/disable, edit (reopens the wizard prefilled with the current loop, PATCHes on submit), and delete with an inline confirm. - NoAgentsGate blocks launching a loop or research topic from a workspace with no roster; the sidebar `+` buttons also disable with a tooltip pointing at the TEAM tier. Not yet wired: the run driver still fills role slots from the workspace-wide pool; teaching enqueue_iteration to prefer loop_agents/loop_teams/loop_orgs is a follow-up.	2026-07-07 22:09:06 -07:00
Omar Sobh	4b48c521eb	loops: backend routes + repo + cron scheduler + HMAC webhook ... Third commit of the Research + Loops arc. Lights up loops as durable recurring topology executions: GET /api/loops list workspace's loops POST /api/loops create — returns webhook_token + signing_key ONCE when webhook trigger is enabled; never exposed again GET /api/loops/:id detail PATCH /api/loops/:id update definition DELETE /api/loops/:id delete POST /api/loops/:id/run trigger one iteration NOW POST /api/loops/:id/enable set enabled=true POST /api/loops/:id/disable set enabled=false POST /webhooks/loops/:token public; HMAC-SHA256-verified Scheduler (cm_runtime::spawn_loop_scheduler) wakes every 10s, queries the partial index on (next_fire_at) for due loops, enqueues one topology_runs row per fire with loop_id + iteration + parent_run_id chained back to the previous iteration. Uses croner via the existing scheduling::next_occurrence helper. Missed windows fire ONCE and skip the backlog — next_fire_at is always computed strictly AFTER now(), so a late scheduler doesn't drain a buildup. Webhook signatures follow the same pattern as the Stripe billing webhook (HMAC-SHA256 with constant-time hex compare). Token + signing key are 24-byte OS-RNG values; the URL uses base64-url for the token, and the signing key is base64-std. Both surface exactly once at create time. All three fire paths (scheduler, immediate-run, webhook) funnel through `cm_db::repo::loops::enqueue_iteration` so the invariants stay in one place. `iters` repeat policy is enforced by the scheduler tick; `until` and `on_completion` land with the orchestrator hook in commit 4. Adds cm-llm as a direct cm-api dep, getrandom for the webhook material generator, and wires the scheduler spawn into the server binary alongside the resume sweeper and outbox drainer.	2026-07-06 04:39:27 -07:00