osobh/clawmates
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

slice 9 cleanup: drop legacy research/loops backend + tables

Browse Source 
Retires the legacy research/loops backend after the missions arc
(slices 1-9) fully replaced it. Frontend cutover was 4663348; this
commit finishes the job on the backend + database.

Migration:
  - 0053_drop_legacy_research_loops.sql — drops the 8 legacy tables
    (research_topics, research_topic_agents, research_outcomes,
    research_publish_approvals, loops, loop_agents, loop_orgs,
    loop_teams) and the 3 topology_runs FK columns
    (research_topic_id, loop_id, iteration). parent_run_id stays;
    recursive_exec still uses it.

Files deleted (11):
  - crates/cm-api/src/routes/{research,loops,research_setup,
    research_pipeline,wizard_repo,probe}.rs
  - crates/cm-api/src/research_container.rs
  - crates/cm-db/src/repo/{research_topics,research_outcomes,
    research_publish_approvals,loops}.rs
  - crates/cm-runtime/src/loops.rs
  - crates/cm-api/tests/research_publish_role.rs

Files edited:
  - crates/cm-api/src/lib.rs — dropped 20 legacy route registrations
    (all /api/research/* + /api/loops/* + /webhooks/loops + probe)
    and module decls
  - crates/cm-api/src/topology_worker.rs — deleted legacy dispatch
    (freeze_research_outcome, advance_loop_after_completion,
    continue_initial_burst, maybe_transition_research_topic,
    parse_reorder_rationale, per-topic/loop gateway resolver).
    reap_stuck_runs now keys on mission_id (not topic_id).
    Executor path unconditionally uses ZeroClawDriveExecutor::from_env
    — mission_orchestrator provisions each claw as an agent inside
    the shared runtime via RuntimeProvisioner, so per-team gateway
    resolution is no longer applicable.
  - crates/cm-api/src/routes/topology.rs — deleted container-log SSE
    endpoint (research/loop-specific), dropped loop_id filter and
    iteration field from ListRunsQuery/RunSummary
  - crates/cm-api/src/routes/world.rs — removed
    active_research_topics/active_loops/preseed_repo_paths;
    World SSE no longer emits repo:{topic}/loop:{id} landmark orbs
    (follow-up task #21 tracks adding mission:{id} equivalents)
  - crates/cm-api/src/runtime_provision.rs — removed now-unused
    mint_workspace_service_token
  - crates/cm-db/src/repo/topology_runs.rs — removed 9 legacy
    helpers (research_topic_id lookup, loop_id_for_run,
    iteration_for_run, active_runs_for_research_topic, etc.)
  - crates/cm-db/src/repo/teams.rs — removed 4 dead helpers
    (team_for_loop, team_for_research_topic + setters)
  - crates/cm-api/tests/topology_jobs.rs — removed loop/topic
    tests, dropped enqueue_run_with_topic helper
  - crates/bins/clawmates-server/src/main.rs — removed
    spawn_loop_scheduler call
  - crates/cm-api/src/routes/mod.rs, crates/cm-db/src/repo/mod.rs,
    crates/cm-runtime/src/lib.rs — module decls stripped

sqlx cache: regenerated against post-migration schema
  (71 files changed, ~+70 / -8896 net)

Test/build: SQLX_OFFLINE=true cargo check --workspace clean;
cargo test --workspace --no-run clean.

Follow-up (task #21): World view lost the in-flight-work landmarks
when repo:{topic} / loop:{id} orbs disappeared. Add mission:{id}
orbs as the missions-era replacement.
This commit is contained in:
Omar Sobh
2026-07-19 18:37:24 -07:00
parent 56201a6985
commit fdb8cfeecc
71 changed files with 70 additions and 8896 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/cm-api/src/topology_worker.rs
+19 -476
View File
@@ -66,21 +66,20 @@ pub fn spawn(pool: PgPool, runtime: cm_runtime::Runtime, poll: Duration) {
});
}
/// Kill research team containers whose `running` topology_run has been
/// alive past [`REAP_STUCK_AFTER_SECS`] without journaling a single
/// step record. Marks the run `failed` with a diagnostic error so the
/// user sees WHY instead of an infinitely-spinning pipeline.
/// Mark `running` mission-bound topology_runs that have been alive past
/// [`REAP_STUCK_AFTER_SECS`] without journaling a single step record as
/// `failed`, with a diagnostic error so the user sees WHY instead of an
/// infinitely-spinning pipeline.
///
/// Only reaps runs bound to a research topic — non-research runs (raw
/// API-driven topology runs) don't own a container so there's nothing
/// to kill; they're left to the existing stale-checkpoint requeuer.
/// Only reaps runs bound to a mission — non-mission runs (raw API-driven
/// topology runs) are left to the existing stale-checkpoint requeuer.
async fn reap_stuck_runs(pool: &PgPool) -> Result<(), sqlx::Error> {
use sqlx::Row;
let rows: Vec<sqlx::postgres::PgRow> = sqlx::query(
"SELECT id, research_topic_id
"SELECT id, mission_id
FROM topology_runs
WHERE status = 'running'
AND research_topic_id IS NOT NULL
AND mission_id IS NOT NULL
AND created_at < now() - make_interval(secs => $1::float)
AND coalesce(jsonb_array_length(coalesce(checkpoint->'records', '[]'::jsonb)), 0) = 0",
)
@@ -88,36 +87,19 @@ async fn reap_stuck_runs(pool: &PgPool) -> Result<(), sqlx::Error> {
.fetch_all(pool)
.await?;
if rows.is_empty() {
return Ok(());
}
// Best-effort docker cleanup; even if the container is already gone
// (crashed, manually killed), we still want to mark the run failed.
let docker = crate::research_container::connect().ok();
for row in rows {
let id: Uuid = row.get("id");
let topic_id: Uuid = row.get("research_topic_id");
let container = crate::research_container::container_name_for(topic_id);
let mission_id: Uuid = row.get("mission_id");
eprintln!(
"topology_worker::reaper: reaping stuck run run_id={} topic_id={} container={} (no step records after {}s)",
id, topic_id, container, REAP_STUCK_AFTER_SECS,
"topology_worker::reaper: reaping stuck run run_id={} mission_id={} (no step records after {}s)",
id, mission_id, REAP_STUCK_AFTER_SECS,
);
if let Some(d) = &docker {
let _ = d
.stop_container(
&container,
None::<bollard::query_parameters::StopContainerOptions>,
)
.await;
}
let _ = cm_db::repo::topology_runs::fail(
pool,
id,
&format!(
"reaped: no step records after {}s (container {} stopped)",
REAP_STUCK_AFTER_SECS, container
"reaped: no step records after {}s",
REAP_STUCK_AFTER_SECS
),
)
.await;
@@ -150,7 +132,7 @@ async fn run_job(
let _ = cm_db::repo::topology_runs::fail(pool, id, &e).await;
}
}
maybe_transition_research_topic(pool, id).await;
maybe_teardown_ephemeral_team(pool, id).await;
return;
}
@@ -169,82 +151,11 @@ async fn run_job(
.and_then(|c| serde_json::from_value(c).ok())
.unwrap_or_default();
// If this run belongs to a research topic OR a loop with a per-team
// ZeroClaw container spawned, point the executor at THAT container's
// gateway URL so the run's turns hit its isolated daemon instead of
// the workspace-wide one. Falls back to the env-derived executor when
// there's no per-topic/loop container (chat sessions, or research/loop
// runs where spawn failed and we recorded no URL).
// 0046 slice 3b — highest-priority resolver: when the run's loop
// has a team_id set (wizard picked "fresh coding team"), spawn/
// reattach the team-scoped container and route this iteration
// through it. The team container inherits the paired research
// topic's repo path (needed for coding agents to write patches)
// + the team's configured risk_profile.
//
// Falls through to the legacy per-topic / per-loop URL resolvers
// when there's no team binding — safe backward-compat for every
// existing loop with team_id = NULL.
// Team path is authoritative when the run's loop has team_id set:
// if we can't spawn the team container, FAIL the run instead of
// silently degrading to the shared runtime. The shared runtime
// doesn't bind /workspace/repo, so coding agents would spend their
// turns narrating without touching files — a much worse failure
// mode than a red run with a clear error.
let per_topic_url =
match try_team_gateway_url(pool, id, WorkspaceId::from(job.workspace_id)).await {
Ok(url) => url,
Err(e) => {
eprintln!("topology_worker: team gateway resolution failed for run {id}: {e}");
let _ = cm_db::repo::topology_runs::fail(
pool,
id,
&format!("team container unavailable — {e}"),
)
.await;
return;
}
};
let per_topic_url = if per_topic_url.is_some() {
per_topic_url
} else {
match cm_db::repo::topology_runs::research_topic_id(pool, id).await {
Ok(Some(topic_id)) => {
match cm_db::repo::research_topics::get(pool, topic_id, job.workspace_id).await {
Ok(Some(t)) => t.zeroclaw_gateway_url,
_ => None,
}
}
_ => None,
}
};
// Loop lookup runs only when the research lookup didn't hit — a run
// is bound to at most one of {topic, loop}. This preserves the
// existing research fast path unchanged.
let per_topic_url = if per_topic_url.is_some() {
per_topic_url
} else {
match cm_db::repo::topology_runs::loop_id_for_run(pool, id).await {
Ok(Some(loop_id)) => cm_db::repo::loops::zeroclaw_gateway_url(pool, loop_id)
.await
.unwrap_or(None),
_ => None,
}
};
if let Some(url) = &per_topic_url {
// Best-effort readiness gate — a freshly-spawned team container may
// still be starting when the worker claims the run. Cap the wait so
// a broken image can't hang the worker.
if let Err(e) =
crate::research_container::wait_ready(url, std::time::Duration::from_secs(30)).await
{
eprintln!("topology_worker: research team {url} readiness: {e} — proceeding anyway");
}
}
let leaf_result = match &per_topic_url {
Some(url) => ZeroClawDriveExecutor::from_env_for_gateway(url.clone()),
None => ZeroClawDriveExecutor::from_env(),
};
// Missions-era runs drive through the shared, env-derived ZeroClaw
// gateway — per-claw provisioning happens ahead of time via
// `RuntimeProvisioner` (see `mission_orchestrator::on_launch`), so
// there's no per-run container/gateway resolution left to do here.
let leaf_result = ZeroClawDriveExecutor::from_env();
let leaf = match leaf_result {
Ok(e) => e,
Err(e) => {
@@ -281,9 +192,6 @@ async fn run_job(
if let Err(e) = cm_db::repo::topology_runs::complete(pool, id, &value).await {
eprintln!("topology_worker: complete({id}) failed: {e}");
}
freeze_research_outcome(pool, id, &record.final_output).await;
advance_loop_after_completion(pool, id, &record.final_output).await;
continue_initial_burst(pool, id).await;
}
Err(e) => {
// Don't clobber a cancellation (or any already-terminal state) with `failed`.
@@ -296,231 +204,6 @@ async fn run_job(
}
}
}
maybe_transition_research_topic(pool, id).await;
}
/// If this run belongs to a research topic, snapshot the orchestrator's
/// final synthesis as a versioned `research_outcomes` row. The frontend
/// canvas reads `latest_outcome` for anything past `standby` so reviewers
/// see the produced draft rather than the original prompt. Best-effort:
/// a failure here logs but doesn't fail the run.
async fn freeze_research_outcome(pool: &PgPool, run_id: Uuid, final_output: &str) {
let topic_id = match cm_db::repo::topology_runs::research_topic_id(pool, run_id).await {
Ok(Some(id)) => id,
Ok(None) => return,
Err(e) => {
eprintln!("topology_worker: research_topic_id({run_id}) failed: {e}");
return;
}
};
if final_output.trim().is_empty() {
return;
}
if let Err(e) =
cm_db::repo::research_outcomes::insert(pool, topic_id, final_output, Some(run_id)).await
{
eprintln!("topology_worker: research_outcomes::insert({run_id}) failed: {e}");
return;
}
// Fan-out: any exec-kind loop bound to this topic with the
// on_artifact_update trigger enabled wakes up now. Coalesced —
// if a loop already has a queued/running run we skip (D3 fallback:
// the coordinator sees the fresh artifact on its next iteration
// anyway). Best-effort per loop; one loop's Docker/DB hiccup
// doesn't affect the others.
let awakened = cm_db::repo::loops::loops_awaiting_topic(pool, topic_id)
.await
.unwrap_or_default();
for (loop_id, workspace_id, task_template, graph) in awakened {
if cm_db::repo::loops::has_active_run(pool, loop_id)
.await
.unwrap_or(false)
{
continue; // Coalesce.
}
// Fan-outs always target kind='exec' (filter enforced in
// loops_awaiting_topic). compose_and_enqueue_iteration takes
// the exec path and prepends the freshly-inserted artifact.
if let Err(e) = crate::routes::loops::compose_and_enqueue_iteration(
pool,
loop_id,
workspace_id,
&graph,
Some(run_id),
Some(&task_template),
)
.await
{
eprintln!("topology_worker: on_artifact_update enqueue({loop_id}) failed: {e:?}");
}
}
}
/// If the just-completed run was a loop iteration with
/// initial_burst_remaining > 0, enqueue the next iteration and
/// decrement the counter (CAS-safe via take_initial_burst_slot).
/// No-op for non-loop runs and for loops whose burst is exhausted.
async fn continue_initial_burst(pool: &PgPool, run_id: Uuid) {
let loop_id = match cm_db::repo::topology_runs::loop_id_for_run(pool, run_id).await {
Ok(Some(id)) => id,
_ => return,
};
// If another worker races us, only ONE gets the slot; the other
// sees 0 (no-op).
let prev = cm_db::repo::loops::take_initial_burst_slot(pool, loop_id)
.await
.unwrap_or(0);
if prev == 0 {
return;
}
// Coalesce with a concurrently-in-flight iteration (a webhook
// arriving during a burst, say).
if cm_db::repo::loops::has_active_run(pool, loop_id)
.await
.unwrap_or(false)
{
return;
}
// Fetch the loop so we have the workspace + graph. The kind-aware
// dispatcher pulls task_template + kind from the same helper it
// uses at first fire, so bursts across an exec + research pair
// behave identically.
let Ok(Some(l)) = cm_db::repo::loops::get_any_workspace(pool, loop_id).await else {
return;
};
if let Err(e) = crate::routes::loops::compose_and_enqueue_iteration(
pool,
loop_id,
l.workspace_id,
&l.graph,
Some(run_id),
None,
)
.await
{
eprintln!("topology_worker: continue_initial_burst enqueue failed: {e:?}");
}
}
/// Post-terminal hook for loop-bound runs. Parses `COMPLETED: INT-<NN>`
/// markers out of the run's final output and advances the loop's
/// `consumed_int_ids` + `current_int_index`. Only fires for runs that
/// belong to a loop AND that loop is bound to a source research topic
/// (the integrations flow). Standalone loops or unbound runs no-op.
///
/// The marker parser is deliberately forgiving — accepts INT-XX and
/// INT-XXX, optionally with surrounding backticks or dashes, so
/// coordinator prompts that emit slightly different formats still
/// advance the pointer.
async fn advance_loop_after_completion(pool: &PgPool, run_id: Uuid, final_output: &str) {
let loop_id = match cm_db::repo::topology_runs::loop_id_for_run(pool, run_id).await {
Ok(Some(id)) => id,
_ => return,
};
let ctx = match cm_db::repo::loops::source_research_context(pool, loop_id).await {
Ok(Some(c)) => c,
_ => return, // Not a research-bound loop; nothing to advance.
};
let mut completed = parse_completed_int_ids(final_output);
// Drop items already recorded so re-runs don't double-count.
let (_topic, already, _idx) = ctx;
completed.retain(|id| !already.contains(id));
if !completed.is_empty() {
if let Err(e) =
cm_db::repo::loops::advance_after_completion(pool, loop_id, &completed).await
{
eprintln!("topology_worker: loops::advance_after_completion({loop_id}) failed: {e}");
}
}
// Reorder rationale — coordinator emits "REORDER: <text>" when it
// works on an INT-XX out of order (usually because a prereq was
// unmet). Append each occurrence to the loop's reorder_events
// array so a mini-timeline UI can surface the history. Iteration
// number comes from topology_runs; -1 if the lookup fails (best-
// effort — we still record the event with a sentinel).
let iteration = cm_db::repo::topology_runs::iteration_for_run(pool, run_id)
.await
.unwrap_or(Some(-1))
.unwrap_or(-1);
for text in parse_reorder_rationale(final_output) {
if let Err(e) =
cm_db::repo::loops::append_reorder_event(pool, loop_id, run_id, iteration, &text).await
{
eprintln!("topology_worker: loops::append_reorder_event({loop_id}) failed: {e}");
}
}
}
/// Extract "REORDER: <text>" rationales — one per line the coordinator
/// emits when it works out of order. Same permissive line matcher as
/// the completed-marker parser (list dashes, backticks, emphasis).
/// Returns the text after the colon, trimmed. Skips empty rationales.
fn parse_reorder_rationale(text: &str) -> Vec<String> {
let mut out = Vec::new();
for line in text.lines() {
let normalized = line.trim_start_matches(|c: char| {
c.is_whitespace() || c == '-' || c == '*' || c == '#' || c == '>'
});
let upper = normalized.to_ascii_uppercase();
if !upper.starts_with("REORDER:") {
continue;
}
// Preserve original case of the rationale text — only the
// marker matched case-insensitively.
let colon = normalized.find(':').map(|i| i + 1).unwrap_or(0);
let rationale = normalized[colon..].trim();
if !rationale.is_empty() {
out.push(rationale.to_string());
}
}
out
}
/// Extract stable INT-XX ids from a completion line. Matches
/// `COMPLETED: INT-01`, `COMPLETED: INT-01, INT-02`, or `- COMPLETED: `INT-01``.
/// De-duplicates within a single output.
fn parse_completed_int_ids(text: &str) -> Vec<String> {
let mut out = Vec::new();
let mut seen = std::collections::HashSet::new();
for line in text.lines() {
// Case-insensitive, tolerates surrounding whitespace, list dashes,
// markdown emphasis, and backticks.
let normalized = line.trim_start_matches(|c: char| {
c.is_whitespace() || c == '-' || c == '*' || c == '#' || c == '>'
});
let upper = normalized.to_ascii_uppercase();
if !upper.starts_with("COMPLETED:") {
continue;
}
for token in upper
.trim_start_matches("COMPLETED:")
.split(|c: char| c == ',' || c == ';' || c.is_whitespace())
{
let stripped = token.trim_matches(|c: char| c == '`' || c == '*' || c == '.');
if stripped.starts_with("INT-")
&& stripped.len() >= 5
&& seen.insert(stripped.to_string())
{
out.push(stripped.to_string());
}
}
}
out
}
/// Post-terminal hook: if this run belongs to a research topic and it was
/// the last sibling in flight, transition the topic `processing → reviewing`.
/// Best-effort — a DB hiccup here logs but doesn't fail the run.
async fn maybe_transition_research_topic(pool: &PgPool, id: Uuid) {
match cm_db::repo::topology_runs::notify_run_completed(pool, id).await {
Ok(true) => {
// Left intentionally quiet on success; the UI polls the topic
// status. Future: emit a run_event so live viewers see it flip.
}
Ok(false) => {}
Err(e) => eprintln!("topology_worker: notify_run_completed({id}) failed: {e}"),
}
maybe_teardown_ephemeral_team(pool, id).await;
}
@@ -630,143 +313,3 @@ async fn drive<E: TurnExecutor>(
.await
}
/// 0046 slice 3b: resolve the run's team-scoped gateway URL.
///
/// Returns `Some(url)` when the run belongs to a loop whose team_id
/// is set and either the team already has a persisted gateway URL
/// or we can spawn one now (the paired research topic's repo path
/// must resolve so the team container has something to bind at
/// `/workspace/repo`).
///
/// Any missing prereq returns `None` so the caller falls through to
/// the legacy per-topic / per-loop resolvers. Every failure logs to
/// stderr and downgrades to `None` — a broken team resolution must
/// never brick a run that could otherwise complete on the shared
/// research container.
/// Resolve the team-scoped ZeroClaw gateway URL for a run.
///
/// Returns:
/// - `Ok(Some(url))` — this run's loop has a `team_id` and the team
/// container is spawned (or reattached) and ready to drive.
/// - `Ok(None)` — the run has no team binding at all; the caller should
/// fall through to the legacy per-topic / per-loop / shared-runtime
/// resolvers.
/// - `Err(msg)` — the run's loop DOES have a `team_id` but the team
/// container couldn't be spawned. The caller MUST fail the run;
/// silently degrading to the shared runtime hides real infra breakage
/// and leaves the agents narrating instead of touching the repo.
async fn try_team_gateway_url(
pool: &PgPool,
run_id: Uuid,
workspace_id: WorkspaceId,
) -> Result<Option<String>, String> {
let Some(loop_id) = cm_db::repo::topology_runs::loop_id_for_run(pool, run_id)
.await
.ok()
.flatten()
else {
return Ok(None);
};
let Some(team_id) = cm_db::repo::teams::team_for_loop(pool, loop_id)
.await
.ok()
.flatten()
else {
return Ok(None);
};
// Reattach fast path — team already has a persisted URL.
if let Ok(Some((_container, Some(url)))) =
cm_db::repo::teams::team_container_coords(pool, team_id, workspace_id).await
{
return Ok(Some(url));
}
// Cold path — need to spawn. Repo path comes from the paired
// research topic (loops.source_research_topic_id + research_topics.
// repo_workspace_path). Without a repo we can't spawn a coding
// team container (nothing meaningful to bind at /workspace/repo).
let source_topic_id = match cm_db::repo::loops::source_research_context(pool, loop_id).await {
Ok(Some((tid, _consumed, _idx))) => tid,
Ok(None) => {
return Err(format!(
"team {team_id} has no paired source_research_topic_id; \
coding loops need a research topic to bind /workspace/repo"
));
}
Err(e) => return Err(format!("source_research_context({loop_id}): {e}")),
};
let source_topic = match cm_db::repo::research_topics::get(
pool,
source_topic_id,
workspace_id.as_uuid(),
)
.await
{
Ok(Some(t)) => t,
Ok(None) => {
return Err(format!(
"source research topic {source_topic_id} not found in workspace"
));
}
Err(e) => return Err(format!("research_topics::get({source_topic_id}): {e}")),
};
let Some(repo_path) = source_topic.repo_workspace_path.clone() else {
return Err(format!(
"source research topic {source_topic_id} has no repo_workspace_path; \
team can't bind /workspace/repo"
));
};
// Team's risk_profile → stamped into every [agents.*] binding on
// the freshly-written config.toml.
let risk_profile = cm_db::repo::teams::get_team_runtime_config(pool, team_id, workspace_id)
.await
.ok()
.flatten()
.and_then(|c| c.risk_profile);
let docker = crate::research_container::connect()
.map_err(|e| format!("docker connect failed for team {team_id}: {e}"))?;
let state_root = crate::research_container::team_state_root(team_id);
// Mint a workspace-owner service session so the team runtime's
// clawmates_door MCP calls pass cm-auth (the static bearer baked
// into the template config isn't a valid auth_sessions row and
// gets 401'd, leaving every agent with 0 tools). MCP is best-effort:
// if the mint fails we still spawn the container with the stale
// bearer — some tools will 401 but the run isn't wholly broken.
let mcp_bearer = crate::runtime_provision::mint_workspace_service_token(pool, workspace_id)
.await
.map_err(|e| {
eprintln!("try_team_gateway_url: mint MCP bearer failed for team {team_id}: {e}");
e
})
.ok();
let spawned = crate::research_container::spawn_team(
&docker,
team_id,
std::path::Path::new(&repo_path),
&state_root,
risk_profile.as_deref(),
mcp_bearer.as_deref(),
)
.await
.map_err(|e| format!("spawn_team({team_id}): {e}"))?;
// Persist coords so future iterations skip the spawn dance.
if let Err(e) = cm_db::repo::teams::set_team_container_coords(
pool,
team_id,
workspace_id,
Some(&spawned.name),
Some(&spawned.gateway_url),
)
.await
{
eprintln!("try_team_gateway_url: persist coords failed for team {team_id}: {e}");
}
Ok(Some(spawned.gateway_url))
}