loops: kind='research' dispatch — research runs as loops
Delivers the research/loop fold: kind='research' loops run the research pipeline each iteration, appending a new research_outcomes version. The paired on_artifact_update fan-out then wakes up any kind='exec' loops bound to the same topic to consume new INTs. Every runnable thing is now a loop (D1). Backend — DB helpers: - cm_db::repo::loops::kind_and_binding — reads (kind, source_topic, task_template) so callers can dispatch without hydrating the whole Loop struct. - cm_db::repo::loops::enqueue_iteration_with_topic — new variant that sets research_topic_id on topology_runs alongside loop_id, so the completion hook's freeze_research_outcome writes a new outcome version for research-kind iterations. - cm_db::repo::research_topics::get_any_workspace — cross-workspace fetch used by the research task builder (the loop row is authoritative for the workspace binding via kind_and_binding). Backend — dispatch: - routes::loops::compose_research_iteration_task — builds the coordinator prompt for a research iteration: topic title + description + outcome_kind + prior version pointer + refresh instructions (survey new sources, preserve stable INT ids, mark superseded items as deprecated rather than delete). The completion hook writes the resulting synthesis as research_outcomes v(prior+1). - routes::loops::compose_and_enqueue_iteration — one-shot dispatch: reads the kind, picks compose_iteration_task (exec) or compose_research_iteration_task (research), enqueues with or without research_topic_id set. All four enqueue callsites now route through compose_and_enqueue: - create_loop (initial_burst) - run_now - webhook_receive - topology_worker::continue_initial_burst - topology_worker::freeze_research_outcome (on_artifact_update fan-out) Research-kind loops naturally form the "nightly refresh" side of a paired research + coding loop: research writes a fresh outcome version → fan-out wakes exec loops with on_artifact_update → coding loops consume the next INT (which the research loop may have just added). D2 answer (inherit repo binding): repo lives on the topic; both loops sharing the source topic id read from the same context, no duplication. D3 answer (coordinator resolves): the research iteration prompt tells the team to preserve stable INT ids and mark deprecations rather than delete, so coding loops' consumed lists stay valid across versions. Follow-up (next commit): ResearchWizard schedule step — "Just once / Nightly / Manual" that creates the paired research-kind loop with initial_burst=1 (just once) or cron 0 3 * * * (nightly) + optional paired coding loop with on_artifact_update.
This commit is contained in:
@@ -55,6 +55,112 @@ fn loop_state_root() -> std::path::PathBuf {
|
||||
/// <task_template>
|
||||
/// The topology_worker's completion hook (P3) parses the COMPLETED
|
||||
/// marker to update `consumed_int_ids`.
|
||||
/// One-shot compose + enqueue for a loop iteration. Reads the loop's
|
||||
/// kind from the DB and dispatches: kind='exec' uses
|
||||
/// compose_iteration_task (INT-consumption prepend); kind='research'
|
||||
/// uses compose_research_iteration_task AND sets research_topic_id on
|
||||
/// the topology_run so freeze_research_outcome writes a new outcome
|
||||
/// version at completion. Returns the run id. Standalone exec loops
|
||||
/// (no source topic) still work — the compose helper returns the
|
||||
/// task_template verbatim.
|
||||
pub async fn compose_and_enqueue_iteration(
|
||||
pool: &sqlx::PgPool,
|
||||
loop_id: Uuid,
|
||||
workspace_id: Uuid,
|
||||
graph: &Value,
|
||||
parent_run_id: Option<Uuid>,
|
||||
task_template_override: Option<&str>,
|
||||
) -> Result<Uuid, cm_db::DbError> {
|
||||
// Kind is the source of truth — task_template alone isn't enough
|
||||
// to know whether to write to research_outcomes.
|
||||
let (kind, source_topic, template) =
|
||||
match cm_db::repo::loops::kind_and_binding(pool, loop_id).await? {
|
||||
Some(t) => t,
|
||||
None => return Err(cm_db::DbError::NotFound),
|
||||
};
|
||||
let template_ref = task_template_override.unwrap_or(&template);
|
||||
let iter = cm_db::repo::loops::next_iteration(pool, loop_id).await?;
|
||||
if kind == "research" {
|
||||
// Research-kind requires a bound topic (schema-level constraint
|
||||
// isn't enforced yet — surface the misconfiguration explicitly).
|
||||
let Some(topic_id) = source_topic else {
|
||||
return Err(cm_db::DbError::NotFound);
|
||||
};
|
||||
let task = compose_research_iteration_task(pool, topic_id, template_ref).await;
|
||||
cm_db::repo::loops::enqueue_iteration_with_topic(
|
||||
pool,
|
||||
loop_id,
|
||||
workspace_id,
|
||||
&task,
|
||||
graph,
|
||||
iter,
|
||||
parent_run_id,
|
||||
Some(topic_id),
|
||||
)
|
||||
.await
|
||||
} else {
|
||||
let task = compose_iteration_task(pool, loop_id, template_ref).await;
|
||||
cm_db::repo::loops::enqueue_iteration(
|
||||
pool,
|
||||
loop_id,
|
||||
workspace_id,
|
||||
&task,
|
||||
graph,
|
||||
iter,
|
||||
parent_run_id,
|
||||
)
|
||||
.await
|
||||
}
|
||||
}
|
||||
|
||||
/// Build the coordinator prompt for a kind='research' loop iteration.
|
||||
/// Wraps the topic's description + outcome_kind + prior artifact
|
||||
/// version pointer into an instruction that asks the team to refresh
|
||||
/// the plan (survey new sources, revise existing INTs, add new ones)
|
||||
/// and emit the updated artifact using the same section shape. The
|
||||
/// completion hook's `freeze_research_outcome` will insert a new
|
||||
/// versioned row automatically because the topology_run carries
|
||||
/// research_topic_id.
|
||||
pub async fn compose_research_iteration_task(
|
||||
pool: &PgPool,
|
||||
topic_id: Uuid,
|
||||
task_template: &str,
|
||||
) -> String {
|
||||
let (title, description, outcome_kind, prior_version) =
|
||||
match cm_db::repo::research_topics::get_any_workspace(pool, topic_id).await {
|
||||
Ok(Some(t)) => {
|
||||
let prior = cm_db::repo::research_outcomes::latest(pool, topic_id)
|
||||
.await
|
||||
.unwrap_or(None)
|
||||
.map(|o| o.version)
|
||||
.unwrap_or(0);
|
||||
(t.title, t.description, t.outcome_kind, prior)
|
||||
}
|
||||
_ => return task_template.to_string(),
|
||||
};
|
||||
format!(
|
||||
"RESEARCH LOOP ITERATION\n\
|
||||
=======================\n\
|
||||
Topic: {title}\n\
|
||||
Outcome kind: {outcome_kind}\n\
|
||||
Prior artifact version: v{prior_version} (0 = fresh)\n\n\
|
||||
DESCRIPTION:\n{description}\n\n\
|
||||
YOUR JOB THIS ITERATION:\n\
|
||||
- Refresh the research — pull in any new papers / findings since v{prior_version}.\n\
|
||||
- Update the artifact using the SAME section structure the outcome_kind\n\
|
||||
requires (e.g. integrations kind = executive summary + INT-XX cards).\n\
|
||||
- Preserve stable ids (INT-01 stays INT-01 across versions). If an item\n\
|
||||
is superseded, mark it {{deprecated: <reason>}} rather than deleting so\n\
|
||||
downstream coding loops that already consumed it don't lose context.\n\
|
||||
- Add NEW items with new ids continuing from the last used number.\n\
|
||||
- Cite every claim; never fabricate sources or repo paths.\n\n\
|
||||
The workspace's final synthesis is captured as research_outcomes v{}. \
|
||||
Downstream on_artifact_update loops will wake up on this write.\n\n\
|
||||
LOOP OPERATOR NOTES:\n{task_template}\n",
|
||||
prior_version + 1
|
||||
)
|
||||
}
|
||||
|
||||
pub async fn compose_iteration_task(pool: &PgPool, loop_id: Uuid, task_template: &str) -> String {
|
||||
let ctx = cm_db::repo::loops::source_research_context(pool, loop_id)
|
||||
.await
|
||||
@@ -306,20 +412,16 @@ pub async fn create_loop(
|
||||
let chain_on_completion = parsed.as_ref().map(|t| t.on_completion).unwrap_or(false);
|
||||
if initial_burst > 0 {
|
||||
ensure_loop_container(&state.pool, user.workspace_id.as_uuid(), id).await;
|
||||
let iter = cm_db::repo::loops::next_iteration(&state.pool, id)
|
||||
.await
|
||||
.unwrap_or(0);
|
||||
let task = compose_iteration_task(&state.pool, id, body.task_template.trim()).await;
|
||||
// Best-effort — we don't want to abort the loop-create response
|
||||
// just because Docker didn't answer the daemon health check.
|
||||
match cm_db::repo::loops::enqueue_iteration(
|
||||
// Kind-aware compose + enqueue — research-kind loops get a
|
||||
// research prompt and research_topic_id set on the run.
|
||||
let template = body.task_template.trim();
|
||||
match compose_and_enqueue_iteration(
|
||||
&state.pool,
|
||||
id,
|
||||
user.workspace_id.as_uuid(),
|
||||
&task,
|
||||
&body.graph,
|
||||
iter,
|
||||
None,
|
||||
Some(template),
|
||||
)
|
||||
.await
|
||||
{
|
||||
@@ -629,18 +731,18 @@ pub async fn run_now(
|
||||
// resolves its gateway URL when it picks up the run. Best-effort;
|
||||
// never blocks the enqueue on Docker being unreachable.
|
||||
ensure_loop_container(&state.pool, l.workspace_id, l.id).await;
|
||||
// Kind-aware — research loops write to research_outcomes.
|
||||
let iter = cm_db::repo::loops::next_iteration(&state.pool, l.id).await?;
|
||||
let task = compose_iteration_task(&state.pool, l.id, &l.task_template).await;
|
||||
let run_id = cm_db::repo::loops::enqueue_iteration(
|
||||
let run_id = compose_and_enqueue_iteration(
|
||||
&state.pool,
|
||||
l.id,
|
||||
l.workspace_id,
|
||||
&task,
|
||||
&l.graph,
|
||||
iter,
|
||||
l.last_run_id,
|
||||
Some(&l.task_template),
|
||||
)
|
||||
.await?;
|
||||
.await
|
||||
.map_err(|_| ApiError::Internal)?;
|
||||
cm_db::repo::loops::mark_fired(&state.pool, l.id, run_id, l.next_fire_at).await?;
|
||||
Ok(Json(RunTriggered {
|
||||
run_id,
|
||||
@@ -681,15 +783,13 @@ pub async fn webhook_receive(
|
||||
Ok(n) => n,
|
||||
Err(_) => return (StatusCode::INTERNAL_SERVER_ERROR, Json(Value::Null)),
|
||||
};
|
||||
let task = compose_iteration_task(&state.pool, l.id, &l.task_template).await;
|
||||
let run_id = match cm_db::repo::loops::enqueue_iteration(
|
||||
let run_id = match compose_and_enqueue_iteration(
|
||||
&state.pool,
|
||||
l.id,
|
||||
l.workspace_id,
|
||||
&task,
|
||||
&l.graph,
|
||||
iter,
|
||||
l.last_run_id,
|
||||
Some(&l.task_template),
|
||||
)
|
||||
.await
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user