loops: kind='research' dispatch — research runs as loops
ci / gates (push) Successful in 5s
ci / frontend (push) Successful in 30s
ci / rust (push) Failing after 48s
ci / e2e (push) Has been skipped
ci / publish (push) Has been skipped

Delivers the research/loop fold: kind='research' loops run the
research pipeline each iteration, appending a new research_outcomes
version. The paired on_artifact_update fan-out then wakes up any
kind='exec' loops bound to the same topic to consume new INTs. Every
runnable thing is now a loop (D1).

Backend — DB helpers:
- cm_db::repo::loops::kind_and_binding — reads (kind, source_topic,
  task_template) so callers can dispatch without hydrating the whole
  Loop struct.
- cm_db::repo::loops::enqueue_iteration_with_topic — new variant that
  sets research_topic_id on topology_runs alongside loop_id, so the
  completion hook's freeze_research_outcome writes a new outcome
  version for research-kind iterations.
- cm_db::repo::research_topics::get_any_workspace — cross-workspace
  fetch used by the research task builder (the loop row is
  authoritative for the workspace binding via kind_and_binding).

Backend — dispatch:
- routes::loops::compose_research_iteration_task — builds the
  coordinator prompt for a research iteration: topic title +
  description + outcome_kind + prior version pointer + refresh
  instructions (survey new sources, preserve stable INT ids, mark
  superseded items as deprecated rather than delete). The completion
  hook writes the resulting synthesis as research_outcomes v(prior+1).
- routes::loops::compose_and_enqueue_iteration — one-shot dispatch:
  reads the kind, picks compose_iteration_task (exec) or
  compose_research_iteration_task (research), enqueues with or
  without research_topic_id set.

All four enqueue callsites now route through compose_and_enqueue:
- create_loop (initial_burst)
- run_now
- webhook_receive
- topology_worker::continue_initial_burst
- topology_worker::freeze_research_outcome (on_artifact_update fan-out)

Research-kind loops naturally form the "nightly refresh" side of a
paired research + coding loop: research writes a fresh outcome
version → fan-out wakes exec loops with on_artifact_update →
coding loops consume the next INT (which the research loop may have
just added). D2 answer (inherit repo binding): repo lives on the
topic; both loops sharing the source topic id read from the same
context, no duplication. D3 answer (coordinator resolves): the
research iteration prompt tells the team to preserve stable INT ids
and mark deprecations rather than delete, so coding loops' consumed
lists stay valid across versions.

Follow-up (next commit): ResearchWizard schedule step — "Just once /
Nightly / Manual" that creates the paired research-kind loop with
initial_burst=1 (just once) or cron 0 3 * * * (nightly) + optional
paired coding loop with on_artifact_update.
This commit is contained in:
Omar Sobh
2026-07-09 22:56:19 -07:00
parent 5aa2de7f30
commit 91a51dce11
4 changed files with 239 additions and 49 deletions
+71 -9
View File
@@ -104,6 +104,36 @@ pub async fn list(pool: &PgPool, workspace_id: Uuid) -> Result<Vec<Loop>, DbErro
Ok(rows)
}
/// Kind + source_research_topic_id + task_template — the minimum a
/// caller needs to compose the right iteration for a loop without
/// hydrating the whole Loop struct. Kind='exec' preserves today's
/// behavior; kind='research' builds a research prompt bound to the
/// source topic so freeze_research_outcome writes a new outcome
/// version.
pub async fn kind_and_binding(
pool: &PgPool,
loop_id: Uuid,
) -> Result<Option<(String, Option<Uuid>, String)>, DbError> {
use sqlx::Row;
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
"SELECT kind, source_research_topic_id, task_template
FROM loops
WHERE id = $1",
)
.bind(loop_id)
.fetch_optional(pool)
.await?;
Ok(row.map(|r| {
(
r.get::<String, _>("kind"),
r.try_get::<Option<Uuid>, _>("source_research_topic_id")
.ok()
.flatten(),
r.get::<String, _>("task_template"),
)
}))
}
/// Cross-workspace fetch used by internal callers (topology_worker
/// completion hooks) where the run row is authoritative for the
/// workspace binding — no need for a second scoping check. Returns
@@ -370,7 +400,6 @@ pub async fn loops_awaiting_topic(
/// coalesce — no point enqueuing another iteration while one is
/// already pending.
pub async fn has_active_run(pool: &PgPool, loop_id: Uuid) -> Result<bool, DbError> {
use sqlx::Row;
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
"SELECT 1 AS one FROM topology_runs
WHERE loop_id = $1 AND status IN ('queued', 'running')
@@ -718,20 +747,53 @@ pub async fn enqueue_iteration(
iteration: i32,
parent_run_id: Option<Uuid>,
) -> Result<Uuid, DbError> {
let run_id = Uuid::now_v7();
sqlx::query!(
"INSERT INTO topology_runs
(id, workspace_id, task, kind, status, graph, tier,
loop_id, iteration, parent_run_id)
VALUES ($1, $2, $3, 'run', 'queued', $4, 'team', $5, $6, $7)",
run_id,
enqueue_iteration_with_topic(
pool,
loop_id,
workspace_id,
task,
graph,
loop_id,
iteration,
parent_run_id,
None,
)
.await
}
/// Variant of `enqueue_iteration` that also sets `research_topic_id` on
/// the topology_runs row. Used by kind='research' loops so
/// `freeze_research_outcome` writes a new outcome version each
/// iteration, and by any future flow that binds a run to both a loop
/// and a research topic.
pub async fn enqueue_iteration_with_topic(
pool: &PgPool,
loop_id: Uuid,
workspace_id: Uuid,
task: &str,
graph: &Value,
iteration: i32,
parent_run_id: Option<Uuid>,
research_topic_id: Option<Uuid>,
) -> Result<Uuid, DbError> {
let run_id = Uuid::now_v7();
// Dynamic query so the new column combination (loop_id +
// research_topic_id on the same row) doesn't require an offline
// sqlx cache regen — the enqueue path only runs on user actions,
// not the tight worker loop.
sqlx::query(
"INSERT INTO topology_runs
(id, workspace_id, task, kind, status, graph, tier,
loop_id, iteration, parent_run_id, research_topic_id)
VALUES ($1, $2, $3, 'run', 'queued', $4, 'team', $5, $6, $7, $8)",
)
.bind(run_id)
.bind(workspace_id)
.bind(task)
.bind(graph)
.bind(loop_id)
.bind(iteration)
.bind(parent_run_id)
.bind(research_topic_id)
.execute(pool)
.await?;
Ok(run_id)
+37
View File
@@ -153,6 +153,43 @@ pub async fn list(pool: &PgPool, workspace_id: Uuid) -> Result<Vec<ResearchTopic
Ok(rows)
}
/// Cross-workspace fetch used by internal callers (topology_worker
/// completion hooks, kind='research' loop iteration builders) where
/// the caller already has an authoritative workspace binding from the
/// linked loop row. Skip the workspace scope filter to avoid a second
/// hop.
pub async fn get_any_workspace(pool: &PgPool, id: Uuid) -> Result<Option<ResearchTopic>, DbError> {
use sqlx::Row;
let row: Option<sqlx::postgres::PgRow> = sqlx::query(
"SELECT id, workspace_id, title, description, outcome_kind, status,
created_by, created_at, updated_at, published_at, topology_kind,
repo_id, repo_workspace_path,
zeroclaw_container_name, zeroclaw_gateway_url
FROM research_topics
WHERE id = $1",
)
.bind(id)
.fetch_optional(pool)
.await?;
Ok(row.map(|r| ResearchTopic {
id: r.get("id"),
workspace_id: r.get("workspace_id"),
title: r.get("title"),
description: r.get("description"),
outcome_kind: r.get("outcome_kind"),
status: r.get("status"),
created_by: r.get("created_by"),
created_at: r.get("created_at"),
updated_at: r.get("updated_at"),
published_at: r.try_get("published_at").ok().flatten(),
topology_kind: r.get("topology_kind"),
repo_id: r.try_get("repo_id").ok().flatten(),
repo_workspace_path: r.try_get("repo_workspace_path").ok().flatten(),
zeroclaw_container_name: r.try_get("zeroclaw_container_name").ok().flatten(),
zeroclaw_gateway_url: r.try_get("zeroclaw_gateway_url").ok().flatten(),
}))
}
pub async fn get(
pool: &PgPool,
id: Uuid,