loops: kind column + initial_burst + on_artifact_update trigger fan-out
Foundation for folding research into loops as a first-class kind.
This commit ships the plumbing; the research-kind dispatch itself
lands next. Behavior for existing exec-kind loops is unchanged unless
they opt into the new trigger fields.
Migration 0044:
- kind TEXT NOT NULL DEFAULT 'exec' CHECK ('exec' | 'research'). New
research-kind will run the research pipeline each iteration (next
commit); 'exec' preserves today's behavior.
- initial_burst_remaining INT NOT NULL DEFAULT 0 — countdown for the
triggers.initial_burst quota. Decremented CAS-safely on each
completion until it hits 0.
- Two partial indexes: (kind, source_research_topic_id) for kind-
aware lookups, and (source_research_topic_id) filtered on
on_artifact_update=true + enabled=true for the fan-out hook.
Trigger schema extended with two optional fields:
- initial_burst: N — fire N iterations back-to-back at create time.
create_loop enqueues the first iteration inline (subject to
empty-roster gate), sets remaining=N-1, and the completion hook
continues the chain until exhausted.
- on_artifact_update: true — when a bound research_outcomes row is
inserted for the source topic, wake up one iteration of this loop.
Coalesced against has_active_run so a burst of rapid revisions
doesn't queue duplicates.
Backend:
- cm_db::repo::loops helpers (all dynamic sqlx, no offline cache
regen needed):
- set_initial_burst_remaining
- take_initial_burst_slot (CAS UPDATE returning prev value; 0 on
exhausted or race loss)
- loops_awaiting_topic (fan-out query: kind=exec + enabled +
on_artifact_update=true bound to the given topic)
- has_active_run (queued|running iteration existence check)
- get_any_workspace (bypasses the workspace scope guard; used by
the completion hook where the run row is authoritative)
- routes/loops::compose_iteration_task made pub so the completion
hook can build the same enriched task string as run_now.
- topology_worker::freeze_research_outcome now fans out to awakened
loops after the outcome insert, using compose_iteration_task and
coalescing on has_active_run.
- topology_worker::continue_initial_burst runs on every completion:
· take_initial_burst_slot (CAS) — no-op if already exhausted
· has_active_run coalesce guard
· re-fetches the loop via get_any_workspace + compose_iteration_task
· enqueues via loops::enqueue_iteration with parent_run_id set
Follow-ups already queued:
- kind='research' dispatch in run_job — build the research
coordinator task from the topic config, run the research pipeline
each iteration. Requires factoring start_topic's task-build.
- ResearchWizard "When should this run?" step (Just once / Nightly /
Manual) creating the topic + paired research-kind loop.
- LoopsWizard trigger UI matching the design proposal (burst count,
cron, on-artifact checkbox).
This commit is contained in:
@@ -104,6 +104,44 @@ pub async fn list(pool: &PgPool, workspace_id: Uuid) -> Result<Vec<Loop>, DbErro
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
/// Cross-workspace fetch used by internal callers (topology_worker
|
||||
/// completion hooks) where the run row is authoritative for the
|
||||
/// workspace binding — no need for a second scoping check. Returns
|
||||
/// None if the loop was deleted between run enqueue and completion.
|
||||
pub async fn get_any_workspace(pool: &PgPool, id: Uuid) -> Result<Option<Loop>, DbError> {
|
||||
// Dynamic query so this callsite doesn't require an offline sqlx
|
||||
// cache regen — used from the completion hook, not on the hot path.
|
||||
use sqlx::Row;
|
||||
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
|
||||
"SELECT id, workspace_id, title, description, graph, task_template,
|
||||
triggers, repeat_policy, enabled, next_fire_at, last_run_id,
|
||||
webhook_token, webhook_signing_key, created_by, created_at, updated_at
|
||||
FROM loops
|
||||
WHERE id = $1",
|
||||
)
|
||||
.bind(id)
|
||||
.fetch_optional(pool)
|
||||
.await?;
|
||||
Ok(row.map(|r| Loop {
|
||||
id: r.get("id"),
|
||||
workspace_id: r.get("workspace_id"),
|
||||
title: r.get("title"),
|
||||
description: r.get("description"),
|
||||
graph: r.get("graph"),
|
||||
task_template: r.get("task_template"),
|
||||
triggers: r.get("triggers"),
|
||||
repeat_policy: r.get("repeat_policy"),
|
||||
enabled: r.get("enabled"),
|
||||
next_fire_at: r.try_get("next_fire_at").ok().flatten(),
|
||||
last_run_id: r.try_get("last_run_id").ok().flatten(),
|
||||
webhook_token: r.try_get("webhook_token").ok().flatten(),
|
||||
webhook_signing_key: r.try_get("webhook_signing_key").ok().flatten(),
|
||||
created_by: r.get("created_by"),
|
||||
created_at: r.get("created_at"),
|
||||
updated_at: r.get("updated_at"),
|
||||
}))
|
||||
}
|
||||
|
||||
pub async fn get(pool: &PgPool, id: Uuid, workspace_id: Uuid) -> Result<Option<Loop>, DbError> {
|
||||
let row = sqlx::query_as!(
|
||||
Loop,
|
||||
@@ -254,6 +292,96 @@ pub async fn set_zeroclaw_container(
|
||||
/// downstream mini-timeline can show WHEN the plan was adjusted and
|
||||
/// WHY. Idempotent: appending a duplicate text/run_id combo is allowed
|
||||
/// (rare — indicates the parser matched twice on the same line).
|
||||
/// Set the countdown for the triggers.initial_burst quota. On
|
||||
/// create_loop we set this to `burst - 1` after firing the first
|
||||
/// iteration inline; on each subsequent completion we decrement and,
|
||||
/// while it's > 0, enqueue another iteration. Dynamic query so the
|
||||
/// new column doesn't need an offline sqlx cache regen.
|
||||
pub async fn set_initial_burst_remaining(
|
||||
pool: &PgPool,
|
||||
loop_id: Uuid,
|
||||
remaining: i32,
|
||||
) -> Result<(), DbError> {
|
||||
sqlx::query("UPDATE loops SET initial_burst_remaining = $2 WHERE id = $1")
|
||||
.bind(loop_id)
|
||||
.bind(remaining)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Atomically decrement initial_burst_remaining, returning the value
|
||||
/// BEFORE decrement. Zero is a no-op (returns 0). Used by the
|
||||
/// completion hook: caller enqueues a new iteration if the returned
|
||||
/// value is > 0. CAS-safe: two workers can't race and both enqueue.
|
||||
pub async fn take_initial_burst_slot(pool: &PgPool, loop_id: Uuid) -> Result<i32, DbError> {
|
||||
use sqlx::Row;
|
||||
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
|
||||
"UPDATE loops
|
||||
SET initial_burst_remaining = GREATEST(initial_burst_remaining - 1, 0)
|
||||
WHERE id = $1 AND initial_burst_remaining > 0
|
||||
RETURNING initial_burst_remaining + 1 AS prev",
|
||||
)
|
||||
.bind(loop_id)
|
||||
.fetch_optional(pool)
|
||||
.await?;
|
||||
Ok(row
|
||||
.and_then(|r| r.try_get::<i32, _>("prev").ok())
|
||||
.unwrap_or(0))
|
||||
}
|
||||
|
||||
/// Enumerate loops that should wake up when a research topic gets a
|
||||
/// new outcome. Filters to kind='exec', enabled, and
|
||||
/// triggers.on_artifact_update = true. Called by the completion hook
|
||||
/// after freeze_research_outcome inserts a new row. Returns
|
||||
/// (loop_id, workspace_id, task_template, graph) so the caller can
|
||||
/// enqueue directly without a second fetch.
|
||||
pub async fn loops_awaiting_topic(
|
||||
pool: &PgPool,
|
||||
topic_id: Uuid,
|
||||
) -> Result<Vec<(Uuid, Uuid, String, Value)>, DbError> {
|
||||
use sqlx::Row;
|
||||
let rows = sqlx::query(
|
||||
"SELECT id, workspace_id, task_template, graph
|
||||
FROM loops
|
||||
WHERE source_research_topic_id = $1
|
||||
AND kind = 'exec'
|
||||
AND enabled = true
|
||||
AND (triggers ->> 'on_artifact_update')::boolean = true",
|
||||
)
|
||||
.bind(topic_id)
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
Ok(rows
|
||||
.into_iter()
|
||||
.map(|r| {
|
||||
(
|
||||
r.get::<Uuid, _>("id"),
|
||||
r.get::<Uuid, _>("workspace_id"),
|
||||
r.get::<String, _>("task_template"),
|
||||
r.get::<Value, _>("graph"),
|
||||
)
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
/// Truthy when the loop currently has a queued or running iteration.
|
||||
/// Used by triggers (on_artifact_update, on_completion chain) to
|
||||
/// coalesce — no point enqueuing another iteration while one is
|
||||
/// already pending.
|
||||
pub async fn has_active_run(pool: &PgPool, loop_id: Uuid) -> Result<bool, DbError> {
|
||||
use sqlx::Row;
|
||||
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
|
||||
"SELECT 1 AS one FROM topology_runs
|
||||
WHERE loop_id = $1 AND status IN ('queued', 'running')
|
||||
LIMIT 1",
|
||||
)
|
||||
.bind(loop_id)
|
||||
.fetch_optional(pool)
|
||||
.await?;
|
||||
Ok(row.is_some())
|
||||
}
|
||||
|
||||
/// Read the reorder_events array for a loop, newest-first, capped at
|
||||
/// `limit`. Used by the progress endpoint to surface a compact recent
|
||||
/// history on the sidebar card. Empty array for standalone loops or
|
||||
|
||||
Reference in New Issue
Block a user