osobh/clawmates
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

slice 9 cleanup: drop legacy research/loops backend + tables

Browse Source 
Retires the legacy research/loops backend after the missions arc
(slices 1-9) fully replaced it. Frontend cutover was 4663348; this
commit finishes the job on the backend + database.

Migration:
  - 0053_drop_legacy_research_loops.sql — drops the 8 legacy tables
    (research_topics, research_topic_agents, research_outcomes,
    research_publish_approvals, loops, loop_agents, loop_orgs,
    loop_teams) and the 3 topology_runs FK columns
    (research_topic_id, loop_id, iteration). parent_run_id stays;
    recursive_exec still uses it.

Files deleted (11):
  - crates/cm-api/src/routes/{research,loops,research_setup,
    research_pipeline,wizard_repo,probe}.rs
  - crates/cm-api/src/research_container.rs
  - crates/cm-db/src/repo/{research_topics,research_outcomes,
    research_publish_approvals,loops}.rs
  - crates/cm-runtime/src/loops.rs
  - crates/cm-api/tests/research_publish_role.rs

Files edited:
  - crates/cm-api/src/lib.rs — dropped 20 legacy route registrations
    (all /api/research/* + /api/loops/* + /webhooks/loops + probe)
    and module decls
  - crates/cm-api/src/topology_worker.rs — deleted legacy dispatch
    (freeze_research_outcome, advance_loop_after_completion,
    continue_initial_burst, maybe_transition_research_topic,
    parse_reorder_rationale, per-topic/loop gateway resolver).
    reap_stuck_runs now keys on mission_id (not topic_id).
    Executor path unconditionally uses ZeroClawDriveExecutor::from_env
    — mission_orchestrator provisions each claw as an agent inside
    the shared runtime via RuntimeProvisioner, so per-team gateway
    resolution is no longer applicable.
  - crates/cm-api/src/routes/topology.rs — deleted container-log SSE
    endpoint (research/loop-specific), dropped loop_id filter and
    iteration field from ListRunsQuery/RunSummary
  - crates/cm-api/src/routes/world.rs — removed
    active_research_topics/active_loops/preseed_repo_paths;
    World SSE no longer emits repo:{topic}/loop:{id} landmark orbs
    (follow-up task #21 tracks adding mission:{id} equivalents)
  - crates/cm-api/src/runtime_provision.rs — removed now-unused
    mint_workspace_service_token
  - crates/cm-db/src/repo/topology_runs.rs — removed 9 legacy
    helpers (research_topic_id lookup, loop_id_for_run,
    iteration_for_run, active_runs_for_research_topic, etc.)
  - crates/cm-db/src/repo/teams.rs — removed 4 dead helpers
    (team_for_loop, team_for_research_topic + setters)
  - crates/cm-api/tests/topology_jobs.rs — removed loop/topic
    tests, dropped enqueue_run_with_topic helper
  - crates/bins/clawmates-server/src/main.rs — removed
    spawn_loop_scheduler call
  - crates/cm-api/src/routes/mod.rs, crates/cm-db/src/repo/mod.rs,
    crates/cm-runtime/src/lib.rs — module decls stripped

sqlx cache: regenerated against post-migration schema
  (71 files changed, ~+70 / -8896 net)

Test/build: SQLX_OFFLINE=true cargo check --workspace clean;
cargo test --workspace --no-run clean.

Follow-up (task #21): World view lost the in-flight-work landmarks
when repo:{topic} / loop:{id} orbs disappeared. Add mission:{id}
orbs as the missions-era replacement.
This commit is contained in:
Omar Sobh
2026-07-19 18:37:24 -07:00
parent 56201a6985
commit fdb8cfeecc
71 changed files with 70 additions and 8896 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/cm-api/src/routes/world.rs
-165
View File
@@ -61,106 +61,6 @@ async fn active_runs(pool: &PgPool, ws: WorkspaceId) -> Vec<(String, String)> {
.collect()
}
/// Active research topics + their assigned agents. Each returned row is a
/// `(topic_id, title, agent_id, repo_workspace_path)` — one row per
/// (topic, agent) pair. Emitted from the SSE loop as `repo:<topic_id>`
/// project orbs so the World shows a clickable, labeled landmark for
/// every in-flight R&D initiative — no need for a file touch to land
/// first. `repo_workspace_path` (when non-null) is the on-disk clone
/// location; the SSE loop uses it to pre-seed the repo tree.
async fn active_research_topics(
pool: &PgPool,
ws: WorkspaceId,
) -> Vec<(String, String, String, Option<String>)> {
let rows = sqlx::query(
"SELECT t.id::text AS topic_id,
t.title AS title,
t.repo_workspace_path AS repo_path,
ra.agent_id::text AS agent_id
FROM research_topics t
JOIN research_topic_agents ra ON ra.topic_id = t.id
WHERE t.workspace_id = $1
AND t.status IN ('processing', 'reviewing', 'publishing')",
)
.bind(ws.as_uuid())
.fetch_all(pool)
.await
.unwrap_or_default();
rows.into_iter()
.map(|r| {
(
r.get::<String, _>("topic_id"),
r.get::<String, _>("title"),
r.get::<String, _>("agent_id"),
r.try_get::<Option<String>, _>("repo_path").unwrap_or(None),
)
})
.collect()
}
/// Cap on pre-seeded file entries per repo. Large repos surface only the
/// top N so the SSE payload stays bounded — a subsequent tool call
/// exercising a specific path will fill in additional nodes on demand.
const REPO_PRESEED_CAP: usize = 200;
/// Read the top-level file list of a topic's cloned repo via `git ls-files`
/// so the SSE loop can pre-seed dir:/file: nodes in the client engine.
/// Bounded by `REPO_PRESEED_CAP`. Returns an empty vec on any failure
/// (missing clone, git not on PATH, empty repo) — a missing pre-seed
/// degrades gracefully to the pre-V3 behavior (tree builds as agents
/// touch files).
async fn preseed_repo_paths(clone_path: &str) -> Vec<String> {
let path = std::path::Path::new(clone_path);
if !path.join(".git").exists() {
return Vec::new();
}
let out = tokio::process::Command::new("git")
.arg("-C")
.arg(path)
.arg("ls-files")
.output()
.await;
let Ok(out) = out else { return Vec::new() };
if !out.status.success() {
return Vec::new();
}
String::from_utf8_lossy(&out.stdout)
.lines()
.filter(|l| !l.trim().is_empty())
.take(REPO_PRESEED_CAP)
.map(|s| s.to_string())
.collect()
}
/// Enabled scheduled loops + their assigned agents. Same shape as
/// `active_research_topics` — `(loop_id, title, agent_id)` per (loop, agent).
/// Emitted as `loop:<loop_id>` landmark orbs so recurring/scheduled work is
/// visible in the World at all times, not just while a run is mid-flight.
/// Contrast with research topics (transient statuses processing/reviewing/
/// publishing) — loops are persistent landmarks the user can click.
async fn active_loops(pool: &PgPool, ws: WorkspaceId) -> Vec<(String, String, String)> {
let rows = sqlx::query(
"SELECT l.id::text AS loop_id, l.title AS title, la.agent_id::text AS agent_id
FROM loops l
JOIN loop_agents la ON la.loop_id = l.id
WHERE l.workspace_id = $1
AND l.enabled = TRUE",
)
.bind(ws.as_uuid())
.fetch_all(pool)
.await
.unwrap_or_default();
rows.into_iter()
.map(|r| {
(
r.get::<String, _>("loop_id"),
r.get::<String, _>("title"),
r.get::<String, _>("agent_id"),
)
})
.collect()
}
/// A short human label for a tool's input (for the tool-call target).
fn summarize_input(input: &Value) -> String {
for k in ["target", "path", "url", "query", "name", "file", "command"] {
@@ -461,71 +361,6 @@ pub async fn world_live(State(state): State<AppState>, Authed(user): Authed) ->
}
}
// Active research topics → landmark project orbs. One `repo:<id>`
// per topic, labeled with the topic title so users can click it
// and drop into the repo-focus (Gource) view before any files are
// touched. Assigned agents gently converge on their topic's orb
// so the affinity is visible even in idle windows.
let research = active_research_topics(&pool, ws).await;
let mut seen_topics = std::collections::HashSet::new();
for (topic_id, title, agent_id, repo_path) in &research {
let node_id = format!("repo:{topic_id}");
if seen_topics.insert(topic_id.clone()) {
yield sse(
"node.activity",
json!({ "nodeId": node_id, "label": title, "kind": "service", "heat": 0.0 }),
);
// Pre-seed the repo tree (V3). One-shot on first sight
// of the topic per SSE client. Each file emits with
// heat=0 so the tree is quiet-solid at rest — activity
// still hot-swaps as agents touch files. Bounded to
// REPO_PRESEED_CAP so payload stays reasonable.
if let Some(clone_path) = repo_path {
for p in preseed_repo_paths(clone_path).await {
let leaf = std::path::Path::new(&p)
.file_name()
.and_then(|s| s.to_str())
.unwrap_or(&p)
.to_string();
yield sse(
"node.activity",
json!({
"nodeId": format!("file:{p}"),
"label": leaf,
"kind": "service",
"heat": 0.0,
}),
);
}
}
}
yield sse(
"world.touch",
json!({ "agentId": agent_id, "nodeId": node_id, "kind": "service", "weight": 0.15 }),
);
}
// Scheduled loops → landmark orbs, symmetric to research topics.
// Persistent landmarks: emitted whenever a loop is enabled, so a
// loop between fires still reads as an in-flight project. When
// a loop actually runs, the topology_worker journals events
// which the run-cursor block below picks up and heats the orb.
let loops = active_loops(&pool, ws).await;
let mut seen_loops = std::collections::HashSet::new();
for (loop_id, title, agent_id) in &loops {
let node_id = format!("loop:{loop_id}");
if seen_loops.insert(loop_id.clone()) {
yield sse(
"node.activity",
json!({ "nodeId": node_id, "label": title, "kind": "service", "heat": 0.0 }),
);
}
yield sse(
"world.touch",
json!({ "agentId": agent_id, "nodeId": node_id, "kind": "service", "weight": 0.15 }),
);
}
// Real convergence: each running agent beams toward its active-run node.
for (run_id, agent_id) in &runs {
let node_id = format!("run:{}", &run_id[..run_id.len().min(8)]);