loops: kind column + initial_burst + on_artifact_update trigger fan-out
ci / gates (push) Successful in 6s
ci / frontend (push) Successful in 28s
ci / rust (push) Failing after 1m0s
ci / e2e (push) Has been skipped
ci / publish (push) Has been skipped

Foundation for folding research into loops as a first-class kind.
This commit ships the plumbing; the research-kind dispatch itself
lands next. Behavior for existing exec-kind loops is unchanged unless
they opt into the new trigger fields.

Migration 0044:
- kind TEXT NOT NULL DEFAULT 'exec' CHECK ('exec' | 'research'). New
  research-kind will run the research pipeline each iteration (next
  commit); 'exec' preserves today's behavior.
- initial_burst_remaining INT NOT NULL DEFAULT 0 — countdown for the
  triggers.initial_burst quota. Decremented CAS-safely on each
  completion until it hits 0.
- Two partial indexes: (kind, source_research_topic_id) for kind-
  aware lookups, and (source_research_topic_id) filtered on
  on_artifact_update=true + enabled=true for the fan-out hook.

Trigger schema extended with two optional fields:
- initial_burst: N — fire N iterations back-to-back at create time.
  create_loop enqueues the first iteration inline (subject to
  empty-roster gate), sets remaining=N-1, and the completion hook
  continues the chain until exhausted.
- on_artifact_update: true — when a bound research_outcomes row is
  inserted for the source topic, wake up one iteration of this loop.
  Coalesced against has_active_run so a burst of rapid revisions
  doesn't queue duplicates.

Backend:
- cm_db::repo::loops helpers (all dynamic sqlx, no offline cache
  regen needed):
  - set_initial_burst_remaining
  - take_initial_burst_slot (CAS UPDATE returning prev value; 0 on
    exhausted or race loss)
  - loops_awaiting_topic (fan-out query: kind=exec + enabled +
    on_artifact_update=true bound to the given topic)
  - has_active_run (queued|running iteration existence check)
  - get_any_workspace (bypasses the workspace scope guard; used by
    the completion hook where the run row is authoritative)
- routes/loops::compose_iteration_task made pub so the completion
  hook can build the same enriched task string as run_now.
- topology_worker::freeze_research_outcome now fans out to awakened
  loops after the outcome insert, using compose_iteration_task and
  coalescing on has_active_run.
- topology_worker::continue_initial_burst runs on every completion:
  · take_initial_burst_slot (CAS) — no-op if already exhausted
  · has_active_run coalesce guard
  · re-fetches the loop via get_any_workspace + compose_iteration_task
  · enqueues via loops::enqueue_iteration with parent_run_id set

Follow-ups already queued:
- kind='research' dispatch in run_job — build the research
  coordinator task from the topic config, run the research pipeline
  each iteration. Requires factoring start_topic's task-build.
- ResearchWizard "When should this run?" step (Just once / Nightly /
  Manual) creating the topic + paired research-kind loop.
- LoopsWizard trigger UI matching the design proposal (burst count,
  cron, on-artifact checkbox).
This commit is contained in:
Omar Sobh
2026-07-09 22:50:01 -07:00
parent ce111273bb
commit 4ada5557f2
4 changed files with 321 additions and 1 deletions
+128
View File
@@ -104,6 +104,44 @@ pub async fn list(pool: &PgPool, workspace_id: Uuid) -> Result<Vec<Loop>, DbErro
Ok(rows)
}
/// Cross-workspace fetch used by internal callers (topology_worker
/// completion hooks) where the run row is authoritative for the
/// workspace binding — no need for a second scoping check. Returns
/// None if the loop was deleted between run enqueue and completion.
pub async fn get_any_workspace(pool: &PgPool, id: Uuid) -> Result<Option<Loop>, DbError> {
// Dynamic query so this callsite doesn't require an offline sqlx
// cache regen — used from the completion hook, not on the hot path.
use sqlx::Row;
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
"SELECT id, workspace_id, title, description, graph, task_template,
triggers, repeat_policy, enabled, next_fire_at, last_run_id,
webhook_token, webhook_signing_key, created_by, created_at, updated_at
FROM loops
WHERE id = $1",
)
.bind(id)
.fetch_optional(pool)
.await?;
Ok(row.map(|r| Loop {
id: r.get("id"),
workspace_id: r.get("workspace_id"),
title: r.get("title"),
description: r.get("description"),
graph: r.get("graph"),
task_template: r.get("task_template"),
triggers: r.get("triggers"),
repeat_policy: r.get("repeat_policy"),
enabled: r.get("enabled"),
next_fire_at: r.try_get("next_fire_at").ok().flatten(),
last_run_id: r.try_get("last_run_id").ok().flatten(),
webhook_token: r.try_get("webhook_token").ok().flatten(),
webhook_signing_key: r.try_get("webhook_signing_key").ok().flatten(),
created_by: r.get("created_by"),
created_at: r.get("created_at"),
updated_at: r.get("updated_at"),
}))
}
pub async fn get(pool: &PgPool, id: Uuid, workspace_id: Uuid) -> Result<Option<Loop>, DbError> {
let row = sqlx::query_as!(
Loop,
@@ -254,6 +292,96 @@ pub async fn set_zeroclaw_container(
/// downstream mini-timeline can show WHEN the plan was adjusted and
/// WHY. Idempotent: appending a duplicate text/run_id combo is allowed
/// (rare — indicates the parser matched twice on the same line).
/// Set the countdown for the triggers.initial_burst quota. On
/// create_loop we set this to `burst - 1` after firing the first
/// iteration inline; on each subsequent completion we decrement and,
/// while it's > 0, enqueue another iteration. Dynamic query so the
/// new column doesn't need an offline sqlx cache regen.
pub async fn set_initial_burst_remaining(
pool: &PgPool,
loop_id: Uuid,
remaining: i32,
) -> Result<(), DbError> {
sqlx::query("UPDATE loops SET initial_burst_remaining = $2 WHERE id = $1")
.bind(loop_id)
.bind(remaining)
.execute(pool)
.await?;
Ok(())
}
/// Atomically decrement initial_burst_remaining, returning the value
/// BEFORE decrement. Zero is a no-op (returns 0). Used by the
/// completion hook: caller enqueues a new iteration if the returned
/// value is > 0. CAS-safe: two workers can't race and both enqueue.
pub async fn take_initial_burst_slot(pool: &PgPool, loop_id: Uuid) -> Result<i32, DbError> {
use sqlx::Row;
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
"UPDATE loops
SET initial_burst_remaining = GREATEST(initial_burst_remaining - 1, 0)
WHERE id = $1 AND initial_burst_remaining > 0
RETURNING initial_burst_remaining + 1 AS prev",
)
.bind(loop_id)
.fetch_optional(pool)
.await?;
Ok(row
.and_then(|r| r.try_get::<i32, _>("prev").ok())
.unwrap_or(0))
}
/// Enumerate loops that should wake up when a research topic gets a
/// new outcome. Filters to kind='exec', enabled, and
/// triggers.on_artifact_update = true. Called by the completion hook
/// after freeze_research_outcome inserts a new row. Returns
/// (loop_id, workspace_id, task_template, graph) so the caller can
/// enqueue directly without a second fetch.
pub async fn loops_awaiting_topic(
pool: &PgPool,
topic_id: Uuid,
) -> Result<Vec<(Uuid, Uuid, String, Value)>, DbError> {
use sqlx::Row;
let rows = sqlx::query(
"SELECT id, workspace_id, task_template, graph
FROM loops
WHERE source_research_topic_id = $1
AND kind = 'exec'
AND enabled = true
AND (triggers ->> 'on_artifact_update')::boolean = true",
)
.bind(topic_id)
.fetch_all(pool)
.await?;
Ok(rows
.into_iter()
.map(|r| {
(
r.get::<Uuid, _>("id"),
r.get::<Uuid, _>("workspace_id"),
r.get::<String, _>("task_template"),
r.get::<Value, _>("graph"),
)
})
.collect())
}
/// Truthy when the loop currently has a queued or running iteration.
/// Used by triggers (on_artifact_update, on_completion chain) to
/// coalesce — no point enqueuing another iteration while one is
/// already pending.
pub async fn has_active_run(pool: &PgPool, loop_id: Uuid) -> Result<bool, DbError> {
use sqlx::Row;
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
"SELECT 1 AS one FROM topology_runs
WHERE loop_id = $1 AND status IN ('queued', 'running')
LIMIT 1",
)
.bind(loop_id)
.fetch_optional(pool)
.await?;
Ok(row.is_some())
}
/// Read the reorder_events array for a loop, newest-first, capped at
/// `limit`. Used by the progress endpoint to surface a compact recent
/// history on the sidebar card. Empty array for standalone loops or