loops: parse COMPLETED: INT-XX markers to advance loop pointer (option b)
ci / gates (push) Successful in 11s
ci / frontend (push) Successful in 26s
ci / rust (push) Failing after 56s
ci / e2e (push) Has been skipped
ci / publish (push) Has been skipped

Closes the loop bridge — the missing piece from the previous commit.
Without this, `current_int_index` stayed at 0 forever and every
iteration re-worked INT-01. Now the topology_worker's completion hook
parses the run's final output for `COMPLETED: INT-<NN>` markers and
atomically advances the loop's consumed_int_ids + current_int_index.

Backend:
- topology_worker::advance_loop_after_completion — new post-terminal
  hook that fires alongside freeze_research_outcome. Reads loop_id_for_run
  (skips non-loop runs) + source_research_context (skips standalone
  loops without a bound source topic).
- parse_completed_int_ids — forgiving parser: matches `COMPLETED: INT-01`,
  `- COMPLETED: `INT-01``, `COMPLETED: INT-01, INT-02`, case-insensitive,
  tolerates list dashes / backticks / markdown emphasis. De-dupes within
  a single output.
- cm_db::repo::loops::advance_after_completion — atomic UPDATE that:
  · appends only NEW ids to consumed_int_ids (idempotent on re-runs)
  · bumps current_int_index by the count of new ids landed
  Set semantics via `SELECT DISTINCT unnest(...)` so ordering-based
  bugs can't accumulate duplicates.

Behavior end-to-end:
1. Loop wizard imports an integrations artifact (previous commit).
2. run_now / webhook_receive → compose_iteration_task prepends artifact
   + focus instruction ("address INT-<current+1>, log COMPLETED at end").
3. Coordinator run does the work, emits `COMPLETED: INT-<NN>`.
4. topology_worker completion hook parses the marker, advances the
   loop, and the NEXT iteration sees an updated `consumed:` list +
   incremented `current_int_index` in its focus instruction.

Follow-ups still queued:
- Loop card refresh button — pull latest artifact after reject-with-
  revision on the source topic (right now the prepend uses the LATEST
  outcome automatically, so refresh is UX only, not correctness).
- Reorder rationale extraction — coordinator emits "REORDER: INT-05
  before INT-04 because prereq X is unmet"; today that's just prose
  in the output, not indexed.
This commit is contained in:
Omar Sobh
2026-07-09 18:48:34 -07:00
parent ce73abe5ab
commit 4a140cb7db
2 changed files with 100 additions and 0 deletions
+64
View File
@@ -170,6 +170,7 @@ async fn run_job(
eprintln!("topology_worker: complete({id}) failed: {e}");
}
freeze_research_outcome(pool, id, &record.final_output).await;
advance_loop_after_completion(pool, id, &record.final_output).await;
}
Err(e) => {
// Don't clobber a cancellation (or any already-terminal state) with `failed`.
@@ -209,6 +210,69 @@ async fn freeze_research_outcome(pool: &PgPool, run_id: Uuid, final_output: &str
}
}
/// Post-terminal hook for loop-bound runs. Parses `COMPLETED: INT-<NN>`
/// markers out of the run's final output and advances the loop's
/// `consumed_int_ids` + `current_int_index`. Only fires for runs that
/// belong to a loop AND that loop is bound to a source research topic
/// (the integrations flow). Standalone loops or unbound runs no-op.
///
/// The marker parser is deliberately forgiving — accepts INT-XX and
/// INT-XXX, optionally with surrounding backticks or dashes, so
/// coordinator prompts that emit slightly different formats still
/// advance the pointer.
async fn advance_loop_after_completion(pool: &PgPool, run_id: Uuid, final_output: &str) {
let loop_id = match cm_db::repo::topology_runs::loop_id_for_run(pool, run_id).await {
Ok(Some(id)) => id,
_ => return,
};
let ctx = match cm_db::repo::loops::source_research_context(pool, loop_id).await {
Ok(Some(c)) => c,
_ => return, // Not a research-bound loop; nothing to advance.
};
let mut completed = parse_completed_int_ids(final_output);
// Drop items already recorded so re-runs don't double-count.
let (_topic, already, _idx) = ctx;
completed.retain(|id| !already.contains(id));
if completed.is_empty() {
return;
}
if let Err(e) = cm_db::repo::loops::advance_after_completion(pool, loop_id, &completed).await {
eprintln!("topology_worker: loops::advance_after_completion({loop_id}) failed: {e}");
}
}
/// Extract stable INT-XX ids from a completion line. Matches
/// `COMPLETED: INT-01`, `COMPLETED: INT-01, INT-02`, or `- COMPLETED: `INT-01``.
/// De-duplicates within a single output.
fn parse_completed_int_ids(text: &str) -> Vec<String> {
let mut out = Vec::new();
let mut seen = std::collections::HashSet::new();
for line in text.lines() {
// Case-insensitive, tolerates surrounding whitespace, list dashes,
// markdown emphasis, and backticks.
let normalized = line.trim_start_matches(|c: char| {
c.is_whitespace() || c == '-' || c == '*' || c == '#' || c == '>'
});
let upper = normalized.to_ascii_uppercase();
if !upper.starts_with("COMPLETED:") {
continue;
}
for token in upper
.trim_start_matches("COMPLETED:")
.split(|c: char| c == ',' || c == ';' || c.is_whitespace())
{
let stripped = token.trim_matches(|c: char| c == '`' || c == '*' || c == '.');
if stripped.starts_with("INT-")
&& stripped.len() >= 5
&& seen.insert(stripped.to_string())
{
out.push(stripped.to_string());
}
}
}
out
}
/// Post-terminal hook: if this run belongs to a research topic and it was
/// the last sibling in flight, transition the topic `processing → reviewing`.
/// Best-effort — a DB hiccup here logs but doesn't fail the run.