research pipeline v2: topology-aware start + persisted draft
ci / gates (push) Successful in 7s
ci / frontend (push) Successful in 37s
ci / rust (push) Successful in 2m42s
ci / e2e (push) Has been skipped
ci / publish (push) Successful in 4m2s

Three connected changes that turn "Start research" from a status
flip into a real pipeline that produces a reviewable artifact:

- Migration 0036: adds research_topics.topology_kind (default
  'hub_spoke') and a new research_outcomes table
  (id, topic_id, version DESC, body_md, produced_by_run_id, created_at)
  so each run's final synthesis is versioned and persistent.

- Wizard now has a topology picker in the Outcome step —
  hub_spoke / pipeline / hierarchical / star_moe — with copy that
  steers users to the right shape (Pipeline for research → distill
  → analyze → implement rosters, hub_spoke for the coordinator-
  and-specialists default).

- start_topic reads the chosen topology_kind, parses it into a
  cm_topology::TopologyKind, and dispatches a per-shape coordinator
  prompt via build_coordinator_task. Pipeline explicitly tells
  stage 1 not to write the final artifact and propagates a
  "final stage MUST emit a complete markdown document with
  measurable acceptance criteria" instruction downstream. The
  graph builder is called with the topology the user actually
  picked instead of hard-coded HubSpoke.

- topology_worker::freeze_research_outcome fires after every
  successful complete(). It looks up research_topic_id on the run;
  if set and final_output is non-empty, it inserts a new
  research_outcomes row (version auto-derived server-side via
  coalesce(max(version), 0) + 1). Best-effort — a DB hiccup logs
  but doesn't fail the run.

- TopicDetail now includes topology_kind and latest_outcome.
  ResearchCanvas swaps in the outcome's body_md (rendered as
  pre-wrap markdown, versioned header, produced-at timestamp)
  whenever an outcome exists; the original prompt collapses into
  an "Original prompt" <details> below so it's still one click
  away. Pre-run topics still show the description as before.

Follow-ups still open: reject-with-revision loop feeding the
coordinator, publishing → published transition + real artifact
export (md / pdf), and an approvals inbox surface for reviewers.
This commit is contained in:
Omar Sobh
2026-07-08 17:13:46 -07:00
parent 7e2b02d8bb
commit a2d3d85ebe
17 changed files with 656 additions and 94 deletions
+160 -33
View File
@@ -42,10 +42,101 @@ pub struct CreateTopicRequest {
pub title: String,
pub description: String,
pub outcome_kind: String,
/// hub_spoke | pipeline | hierarchical | star_moe. Defaults to hub_spoke.
#[serde(default)]
pub topology_kind: Option<String>,
#[serde(default)]
pub agents: Vec<AgentSlotInput>,
}
/// Topology kinds the wizard exposes for research. Every string here must
/// also be a valid `cm_topology::TopologyKind` — start_topic passes it
/// through to the builder verbatim.
const VALID_TOPOLOGY_KINDS: &[&str] = &["hub_spoke", "pipeline", "hierarchical", "star_moe"];
/// Assemble the task prompt fed to the graph's head node. Shape branches on
/// topology so the head's instructions actually match how the graph will
/// run: hub_spoke → central coordinator delegates + synthesizes; pipeline →
/// stage-1 opens and each stage owns its handoff, the last stage produces
/// the artifact; hierarchical → root plans, children work parallel, root
/// synthesizes; star_moe → router classifies + dispatches to the domain
/// expert best-suited to each subtask.
fn build_coordinator_task(
title: &str,
outcome: &str,
description: &str,
topo: &cm_topology::TopologyKind,
roster: &str,
) -> String {
use cm_topology::TopologyKind::*;
let framing = format!(
"RESEARCH TOPIC: {title}\n\
OUTCOME KIND: {outcome} (spec / prod_plan / roadmap / paper)\n\n\
DESCRIPTION:\n{description}\n\n\
TEAM:\n{roster}\n\n"
);
let body = match topo {
HubSpoke => {
"SHAPE: hub_spoke. You are the central coordinator (hub). \
Every teammate is a spoke you can address per turn.\n\n\
YOUR JOB:\n\
1. Break the topic into concrete sub-tasks and assign each to the best-fit spoke.\n\
2. Delegate turn-by-turn: each spoke's response feeds your next dispatch.\n\
3. Synthesize their outputs into a single artifact that satisfies the description.\n\
4. Cite each spoke's contribution where it lands in the final document.\n\
5. Structure the final document with clear sections — problem, evidence, \
proposal, and measurable acceptance criteria — for every improvement area \
implied by the description."
}
Pipeline => {
"SHAPE: pipeline. You are stage 1. Each teammate is the next \
stage in a linear handoff — your output is the next stage's only input, and so \
on until the last stage produces the final artifact.\n\n\
YOUR JOB (stage 1):\n\
1. Do YOUR stage's specific work as described in your role.\n\
2. Structure your handoff so the next stage can act on it directly — cite \
sources, name the units you produced, and enumerate anything the next stage \
must inspect.\n\
3. Do NOT try to write the final artifact yourself; that's the last stage's job.\n\
4. Keep the topic's outcome_kind in mind — the pipeline will produce a single \
document of that shape when the last stage synthesizes.\n\n\
LAST-STAGE INSTRUCTION (propagate this in your handoff):\n\
The final stage MUST emit the complete artifact as a single markdown document, \
structured by section, with measurable acceptance criteria for every \
recommendation and inline citations to any evidence collected upstream."
}
Hierarchical => {
"SHAPE: hierarchical. You are the root. Your direct children \
work in parallel with your plan as their context, then you synthesize their \
outputs.\n\n\
YOUR JOB:\n\
1. Decompose the topic into distinct sub-problems, one per child, chosen so \
they can run in parallel without cross-dependencies.\n\
2. Fan out: state the sub-problem, constraints, and expected output shape for \
each child's independent work.\n\
3. Collect their outputs. Synthesize into a single artifact structured by \
section, resolving any conflicts explicitly.\n\
4. Attribute each section to the contributing child."
}
StarMoe => {
"SHAPE: star_moe (mixture-of-experts). You are the router. Each \
teammate is a domain expert. Route subtasks to whichever expert best matches \
the domain of the subtask.\n\n\
YOUR JOB:\n\
1. Analyze the topic's description and enumerate the distinct domains it touches.\n\
2. For each domain, address the best-fit expert (by role / job title) with a \
scoped question. Never broadcast — routing beats fan-out here.\n\
3. Collect expert answers and produce a single artifact structured by section, \
one per domain, citing the routed expert."
}
_ => {
"Coordinate your teammates to produce a single artifact satisfying \
the description."
}
};
format!("{framing}{body}")
}
#[derive(Deserialize)]
pub struct AgentSlotInput {
pub agent_id: Uuid,
@@ -70,6 +161,10 @@ pub async fn create_topic(
return Err(ApiError::BadRequest);
}
check_outcome(&body.outcome_kind)?;
let topology_kind = body.topology_kind.as_deref().unwrap_or("hub_spoke");
if !VALID_TOPOLOGY_KINDS.contains(&topology_kind) {
return Err(ApiError::BadRequest);
}
// Ownership check before any writes: every agent must be in the caller's
// workspace. Refuses to leak "agent exists" if it isn't visible.
@@ -86,6 +181,7 @@ pub async fn create_topic(
body.title.trim(),
body.description.trim(),
&body.outcome_kind,
topology_kind,
user.user_id.as_uuid(),
)
.await?;
@@ -143,6 +239,12 @@ pub struct TopicDetail {
/// "Awaiting reviewer approval" instead — avoids the 409 the user
/// gets from double-clicking.
pub has_pending_publish_request: bool,
/// The most recent `research_outcomes` row for this topic — the draft
/// the pipeline produced. Present once a run has completed; the canvas
/// renders `body_md` in place of `description` when in `reviewing` and
/// beyond so reviewers see what actually needs approval.
#[serde(skip_serializing_if = "Option::is_none")]
pub latest_outcome: Option<cm_db::repo::research_outcomes::Outcome>,
}
pub async fn get_topic(
@@ -158,10 +260,12 @@ pub async fn get_topic(
cm_db::repo::research_publish_approvals::pending_for_topic(&state.pool, id)
.await?
.is_some();
let latest_outcome = cm_db::repo::research_outcomes::latest(&state.pool, id).await?;
Ok(Json(TopicDetail {
topic,
agents,
has_pending_publish_request,
latest_outcome,
}))
}
@@ -266,51 +370,82 @@ pub async fn start_topic(
// Coordinator = first slot whose role_slot mentions "coordinator" (case-
// insensitive), else the first slot. The coordinator becomes the hub of
// the hub_spoke graph, so it can talk to every other agent per-turn.
let coord_ix = roster
.iter()
.position(|(s, _)| {
s.role_slot
.as_deref()
.map(|r| r.to_ascii_lowercase().contains("coordinator"))
.unwrap_or(false)
})
.unwrap_or(0);
if coord_ix != 0 {
roster.swap(0, coord_ix);
// Parse the topic's chosen topology; default to hub_spoke on unknown
// strings (should be unreachable — create_topic validates the enum).
let topo: cm_topology::TopologyKind =
serde_json::from_value(serde_json::json!(topic.topology_kind.as_str()))
.unwrap_or(cm_topology::TopologyKind::HubSpoke);
let is_pipeline = matches!(topo, cm_topology::TopologyKind::Pipeline);
// hub_spoke / hierarchical / star_moe all put a coordinator at index 0;
// pipeline puts a first-stage worker there (the roster's original order
// *is* the pipeline order). If a slot is explicitly tagged "coordinator"
// and we're not in pipeline mode, promote it to index 0.
if !is_pipeline {
let coord_ix = roster
.iter()
.position(|(s, _)| {
s.role_slot
.as_deref()
.map(|r| r.to_ascii_lowercase().contains("coordinator"))
.unwrap_or(false)
})
.unwrap_or(0);
if coord_ix != 0 {
roster.swap(0, coord_ix);
}
}
// Build a role list where element 0 is the coordinator (hub) and the rest
// are spokes. cm_topology's hub_spoke builder wires edges hub↔every spoke.
let head_label = if is_pipeline {
"stage 1"
} else {
"coordinator"
};
let roles: Vec<String> = roster
.iter()
.enumerate()
.map(|(i, (s, a))| {
if i == 0 {
"coordinator".to_string()
head_label.to_string()
} else if let Some(r) = &s.role_slot {
r.clone()
} else if !a.job_title.is_empty() {
a.job_title.clone()
} else if is_pipeline {
format!("stage {}", i + 1)
} else {
"specialist".to_string()
}
})
.collect();
let role_refs: Vec<&str> = roles.iter().map(|s| s.as_str()).collect();
let graph = cm_topology::build(cm_topology::TopologyKind::HubSpoke, &role_refs)
.map_err(|_| ApiError::BadRequest)?;
let graph = cm_topology::build(topo, &role_refs).map_err(|_| ApiError::BadRequest)?;
let graph_json_str = cm_topology::to_json(&graph).map_err(|_| ApiError::BadRequest)?;
let graph_value: serde_json::Value =
serde_json::from_str(&graph_json_str).map_err(|_| ApiError::BadRequest)?;
// Coordinator prompt: topic framing + roster + delegation instruction.
// Roster listing, formatted for the prompt.
let roster_lines = roster
.iter()
.enumerate()
.map(|(i, (s, a))| {
let role = if i == 0 {
let role = if i == 0 && !is_pipeline {
"coordinator (you)".to_string()
} else if i == 0 && is_pipeline {
format!(
"{} (you — stage 1)",
s.role_slot
.as_deref()
.unwrap_or(if !a.job_title.is_empty() {
a.job_title.as_str()
} else {
"opener"
})
)
} else if let Some(r) = &s.role_slot {
r.clone()
if is_pipeline {
format!("{} (stage {})", r, i + 1)
} else {
r.clone()
}
} else if !a.job_title.is_empty() {
a.job_title.clone()
} else {
@@ -320,20 +455,12 @@ pub async fn start_topic(
})
.collect::<Vec<_>>()
.join("\n");
let task = format!(
"RESEARCH TOPIC: {title}\n\
OUTCOME KIND: {outcome} (spec / prod_plan / roadmap / paper)\n\n\
DESCRIPTION:\n{description}\n\n\
TEAM (hub_spoke — you are the hub, the rest are spokes you can address per-turn):\n{roster}\n\n\
YOUR JOB (coordinator):\n\
1. Break the topic into concrete sub-tasks and assign each to the best-fit spoke.\n\
2. Delegate turn-by-turn: each spoke's response feeds your next dispatch.\n\
3. Synthesize their outputs into a single {outcome} that satisfies the description.\n\
4. Cite each spoke's contribution where it lands in the final artifact.",
title = topic.title,
outcome = topic.outcome_kind,
description = topic.description,
roster = roster_lines,
let task = build_coordinator_task(
&topic.title,
&topic.outcome_kind,
&topic.description,
&topo,
&roster_lines,
);
let run_id = uuid::Uuid::now_v7();