research pipeline v2: topology-aware start + persisted draft
Three connected changes that turn "Start research" from a status flip into a real pipeline that produces a reviewable artifact: - Migration 0036: adds research_topics.topology_kind (default 'hub_spoke') and a new research_outcomes table (id, topic_id, version DESC, body_md, produced_by_run_id, created_at) so each run's final synthesis is versioned and persistent. - Wizard now has a topology picker in the Outcome step — hub_spoke / pipeline / hierarchical / star_moe — with copy that steers users to the right shape (Pipeline for research → distill → analyze → implement rosters, hub_spoke for the coordinator- and-specialists default). - start_topic reads the chosen topology_kind, parses it into a cm_topology::TopologyKind, and dispatches a per-shape coordinator prompt via build_coordinator_task. Pipeline explicitly tells stage 1 not to write the final artifact and propagates a "final stage MUST emit a complete markdown document with measurable acceptance criteria" instruction downstream. The graph builder is called with the topology the user actually picked instead of hard-coded HubSpoke. - topology_worker::freeze_research_outcome fires after every successful complete(). It looks up research_topic_id on the run; if set and final_output is non-empty, it inserts a new research_outcomes row (version auto-derived server-side via coalesce(max(version), 0) + 1). Best-effort — a DB hiccup logs but doesn't fail the run. - TopicDetail now includes topology_kind and latest_outcome. ResearchCanvas swaps in the outcome's body_md (rendered as pre-wrap markdown, versioned header, produced-at timestamp) whenever an outcome exists; the original prompt collapses into an "Original prompt" <details> below so it's still one click away. Pre-run topics still show the description as before. Follow-ups still open: reject-with-revision loop feeding the coordinator, publishing → published transition + real artifact export (md / pdf), and an approvals inbox surface for reviewers.
This commit is contained in:
@@ -42,10 +42,101 @@ pub struct CreateTopicRequest {
|
||||
pub title: String,
|
||||
pub description: String,
|
||||
pub outcome_kind: String,
|
||||
/// hub_spoke | pipeline | hierarchical | star_moe. Defaults to hub_spoke.
|
||||
#[serde(default)]
|
||||
pub topology_kind: Option<String>,
|
||||
#[serde(default)]
|
||||
pub agents: Vec<AgentSlotInput>,
|
||||
}
|
||||
|
||||
/// Topology kinds the wizard exposes for research. Every string here must
|
||||
/// also be a valid `cm_topology::TopologyKind` — start_topic passes it
|
||||
/// through to the builder verbatim.
|
||||
const VALID_TOPOLOGY_KINDS: &[&str] = &["hub_spoke", "pipeline", "hierarchical", "star_moe"];
|
||||
|
||||
/// Assemble the task prompt fed to the graph's head node. Shape branches on
|
||||
/// topology so the head's instructions actually match how the graph will
|
||||
/// run: hub_spoke → central coordinator delegates + synthesizes; pipeline →
|
||||
/// stage-1 opens and each stage owns its handoff, the last stage produces
|
||||
/// the artifact; hierarchical → root plans, children work parallel, root
|
||||
/// synthesizes; star_moe → router classifies + dispatches to the domain
|
||||
/// expert best-suited to each subtask.
|
||||
fn build_coordinator_task(
|
||||
title: &str,
|
||||
outcome: &str,
|
||||
description: &str,
|
||||
topo: &cm_topology::TopologyKind,
|
||||
roster: &str,
|
||||
) -> String {
|
||||
use cm_topology::TopologyKind::*;
|
||||
let framing = format!(
|
||||
"RESEARCH TOPIC: {title}\n\
|
||||
OUTCOME KIND: {outcome} (spec / prod_plan / roadmap / paper)\n\n\
|
||||
DESCRIPTION:\n{description}\n\n\
|
||||
TEAM:\n{roster}\n\n"
|
||||
);
|
||||
let body = match topo {
|
||||
HubSpoke => {
|
||||
"SHAPE: hub_spoke. You are the central coordinator (hub). \
|
||||
Every teammate is a spoke you can address per turn.\n\n\
|
||||
YOUR JOB:\n\
|
||||
1. Break the topic into concrete sub-tasks and assign each to the best-fit spoke.\n\
|
||||
2. Delegate turn-by-turn: each spoke's response feeds your next dispatch.\n\
|
||||
3. Synthesize their outputs into a single artifact that satisfies the description.\n\
|
||||
4. Cite each spoke's contribution where it lands in the final document.\n\
|
||||
5. Structure the final document with clear sections — problem, evidence, \
|
||||
proposal, and measurable acceptance criteria — for every improvement area \
|
||||
implied by the description."
|
||||
}
|
||||
Pipeline => {
|
||||
"SHAPE: pipeline. You are stage 1. Each teammate is the next \
|
||||
stage in a linear handoff — your output is the next stage's only input, and so \
|
||||
on until the last stage produces the final artifact.\n\n\
|
||||
YOUR JOB (stage 1):\n\
|
||||
1. Do YOUR stage's specific work as described in your role.\n\
|
||||
2. Structure your handoff so the next stage can act on it directly — cite \
|
||||
sources, name the units you produced, and enumerate anything the next stage \
|
||||
must inspect.\n\
|
||||
3. Do NOT try to write the final artifact yourself; that's the last stage's job.\n\
|
||||
4. Keep the topic's outcome_kind in mind — the pipeline will produce a single \
|
||||
document of that shape when the last stage synthesizes.\n\n\
|
||||
LAST-STAGE INSTRUCTION (propagate this in your handoff):\n\
|
||||
The final stage MUST emit the complete artifact as a single markdown document, \
|
||||
structured by section, with measurable acceptance criteria for every \
|
||||
recommendation and inline citations to any evidence collected upstream."
|
||||
}
|
||||
Hierarchical => {
|
||||
"SHAPE: hierarchical. You are the root. Your direct children \
|
||||
work in parallel with your plan as their context, then you synthesize their \
|
||||
outputs.\n\n\
|
||||
YOUR JOB:\n\
|
||||
1. Decompose the topic into distinct sub-problems, one per child, chosen so \
|
||||
they can run in parallel without cross-dependencies.\n\
|
||||
2. Fan out: state the sub-problem, constraints, and expected output shape for \
|
||||
each child's independent work.\n\
|
||||
3. Collect their outputs. Synthesize into a single artifact structured by \
|
||||
section, resolving any conflicts explicitly.\n\
|
||||
4. Attribute each section to the contributing child."
|
||||
}
|
||||
StarMoe => {
|
||||
"SHAPE: star_moe (mixture-of-experts). You are the router. Each \
|
||||
teammate is a domain expert. Route subtasks to whichever expert best matches \
|
||||
the domain of the subtask.\n\n\
|
||||
YOUR JOB:\n\
|
||||
1. Analyze the topic's description and enumerate the distinct domains it touches.\n\
|
||||
2. For each domain, address the best-fit expert (by role / job title) with a \
|
||||
scoped question. Never broadcast — routing beats fan-out here.\n\
|
||||
3. Collect expert answers and produce a single artifact structured by section, \
|
||||
one per domain, citing the routed expert."
|
||||
}
|
||||
_ => {
|
||||
"Coordinate your teammates to produce a single artifact satisfying \
|
||||
the description."
|
||||
}
|
||||
};
|
||||
format!("{framing}{body}")
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
pub struct AgentSlotInput {
|
||||
pub agent_id: Uuid,
|
||||
@@ -70,6 +161,10 @@ pub async fn create_topic(
|
||||
return Err(ApiError::BadRequest);
|
||||
}
|
||||
check_outcome(&body.outcome_kind)?;
|
||||
let topology_kind = body.topology_kind.as_deref().unwrap_or("hub_spoke");
|
||||
if !VALID_TOPOLOGY_KINDS.contains(&topology_kind) {
|
||||
return Err(ApiError::BadRequest);
|
||||
}
|
||||
|
||||
// Ownership check before any writes: every agent must be in the caller's
|
||||
// workspace. Refuses to leak "agent exists" if it isn't visible.
|
||||
@@ -86,6 +181,7 @@ pub async fn create_topic(
|
||||
body.title.trim(),
|
||||
body.description.trim(),
|
||||
&body.outcome_kind,
|
||||
topology_kind,
|
||||
user.user_id.as_uuid(),
|
||||
)
|
||||
.await?;
|
||||
@@ -143,6 +239,12 @@ pub struct TopicDetail {
|
||||
/// "Awaiting reviewer approval" instead — avoids the 409 the user
|
||||
/// gets from double-clicking.
|
||||
pub has_pending_publish_request: bool,
|
||||
/// The most recent `research_outcomes` row for this topic — the draft
|
||||
/// the pipeline produced. Present once a run has completed; the canvas
|
||||
/// renders `body_md` in place of `description` when in `reviewing` and
|
||||
/// beyond so reviewers see what actually needs approval.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub latest_outcome: Option<cm_db::repo::research_outcomes::Outcome>,
|
||||
}
|
||||
|
||||
pub async fn get_topic(
|
||||
@@ -158,10 +260,12 @@ pub async fn get_topic(
|
||||
cm_db::repo::research_publish_approvals::pending_for_topic(&state.pool, id)
|
||||
.await?
|
||||
.is_some();
|
||||
let latest_outcome = cm_db::repo::research_outcomes::latest(&state.pool, id).await?;
|
||||
Ok(Json(TopicDetail {
|
||||
topic,
|
||||
agents,
|
||||
has_pending_publish_request,
|
||||
latest_outcome,
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -266,51 +370,82 @@ pub async fn start_topic(
|
||||
// Coordinator = first slot whose role_slot mentions "coordinator" (case-
|
||||
// insensitive), else the first slot. The coordinator becomes the hub of
|
||||
// the hub_spoke graph, so it can talk to every other agent per-turn.
|
||||
let coord_ix = roster
|
||||
.iter()
|
||||
.position(|(s, _)| {
|
||||
s.role_slot
|
||||
.as_deref()
|
||||
.map(|r| r.to_ascii_lowercase().contains("coordinator"))
|
||||
.unwrap_or(false)
|
||||
})
|
||||
.unwrap_or(0);
|
||||
if coord_ix != 0 {
|
||||
roster.swap(0, coord_ix);
|
||||
// Parse the topic's chosen topology; default to hub_spoke on unknown
|
||||
// strings (should be unreachable — create_topic validates the enum).
|
||||
let topo: cm_topology::TopologyKind =
|
||||
serde_json::from_value(serde_json::json!(topic.topology_kind.as_str()))
|
||||
.unwrap_or(cm_topology::TopologyKind::HubSpoke);
|
||||
let is_pipeline = matches!(topo, cm_topology::TopologyKind::Pipeline);
|
||||
// hub_spoke / hierarchical / star_moe all put a coordinator at index 0;
|
||||
// pipeline puts a first-stage worker there (the roster's original order
|
||||
// *is* the pipeline order). If a slot is explicitly tagged "coordinator"
|
||||
// and we're not in pipeline mode, promote it to index 0.
|
||||
if !is_pipeline {
|
||||
let coord_ix = roster
|
||||
.iter()
|
||||
.position(|(s, _)| {
|
||||
s.role_slot
|
||||
.as_deref()
|
||||
.map(|r| r.to_ascii_lowercase().contains("coordinator"))
|
||||
.unwrap_or(false)
|
||||
})
|
||||
.unwrap_or(0);
|
||||
if coord_ix != 0 {
|
||||
roster.swap(0, coord_ix);
|
||||
}
|
||||
}
|
||||
// Build a role list where element 0 is the coordinator (hub) and the rest
|
||||
// are spokes. cm_topology's hub_spoke builder wires edges hub↔every spoke.
|
||||
let head_label = if is_pipeline {
|
||||
"stage 1"
|
||||
} else {
|
||||
"coordinator"
|
||||
};
|
||||
let roles: Vec<String> = roster
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, (s, a))| {
|
||||
if i == 0 {
|
||||
"coordinator".to_string()
|
||||
head_label.to_string()
|
||||
} else if let Some(r) = &s.role_slot {
|
||||
r.clone()
|
||||
} else if !a.job_title.is_empty() {
|
||||
a.job_title.clone()
|
||||
} else if is_pipeline {
|
||||
format!("stage {}", i + 1)
|
||||
} else {
|
||||
"specialist".to_string()
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
let role_refs: Vec<&str> = roles.iter().map(|s| s.as_str()).collect();
|
||||
let graph = cm_topology::build(cm_topology::TopologyKind::HubSpoke, &role_refs)
|
||||
.map_err(|_| ApiError::BadRequest)?;
|
||||
let graph = cm_topology::build(topo, &role_refs).map_err(|_| ApiError::BadRequest)?;
|
||||
let graph_json_str = cm_topology::to_json(&graph).map_err(|_| ApiError::BadRequest)?;
|
||||
let graph_value: serde_json::Value =
|
||||
serde_json::from_str(&graph_json_str).map_err(|_| ApiError::BadRequest)?;
|
||||
|
||||
// Coordinator prompt: topic framing + roster + delegation instruction.
|
||||
// Roster listing, formatted for the prompt.
|
||||
let roster_lines = roster
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, (s, a))| {
|
||||
let role = if i == 0 {
|
||||
let role = if i == 0 && !is_pipeline {
|
||||
"coordinator (you)".to_string()
|
||||
} else if i == 0 && is_pipeline {
|
||||
format!(
|
||||
"{} (you — stage 1)",
|
||||
s.role_slot
|
||||
.as_deref()
|
||||
.unwrap_or(if !a.job_title.is_empty() {
|
||||
a.job_title.as_str()
|
||||
} else {
|
||||
"opener"
|
||||
})
|
||||
)
|
||||
} else if let Some(r) = &s.role_slot {
|
||||
r.clone()
|
||||
if is_pipeline {
|
||||
format!("{} (stage {})", r, i + 1)
|
||||
} else {
|
||||
r.clone()
|
||||
}
|
||||
} else if !a.job_title.is_empty() {
|
||||
a.job_title.clone()
|
||||
} else {
|
||||
@@ -320,20 +455,12 @@ pub async fn start_topic(
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n");
|
||||
let task = format!(
|
||||
"RESEARCH TOPIC: {title}\n\
|
||||
OUTCOME KIND: {outcome} (spec / prod_plan / roadmap / paper)\n\n\
|
||||
DESCRIPTION:\n{description}\n\n\
|
||||
TEAM (hub_spoke — you are the hub, the rest are spokes you can address per-turn):\n{roster}\n\n\
|
||||
YOUR JOB (coordinator):\n\
|
||||
1. Break the topic into concrete sub-tasks and assign each to the best-fit spoke.\n\
|
||||
2. Delegate turn-by-turn: each spoke's response feeds your next dispatch.\n\
|
||||
3. Synthesize their outputs into a single {outcome} that satisfies the description.\n\
|
||||
4. Cite each spoke's contribution where it lands in the final artifact.",
|
||||
title = topic.title,
|
||||
outcome = topic.outcome_kind,
|
||||
description = topic.description,
|
||||
roster = roster_lines,
|
||||
let task = build_coordinator_task(
|
||||
&topic.title,
|
||||
&topic.outcome_kind,
|
||||
&topic.description,
|
||||
&topo,
|
||||
&roster_lines,
|
||||
);
|
||||
|
||||
let run_id = uuid::Uuid::now_v7();
|
||||
|
||||
Reference in New Issue
Block a user