loops: parse COMPLETED: INT-XX markers to advance loop pointer (option b)
ci / gates (push) Successful in 11s
ci / frontend (push) Successful in 26s
ci / rust (push) Failing after 56s
ci / e2e (push) Has been skipped
ci / publish (push) Has been skipped

Closes the loop bridge — the missing piece from the previous commit.
Without this, `current_int_index` stayed at 0 forever and every
iteration re-worked INT-01. Now the topology_worker's completion hook
parses the run's final output for `COMPLETED: INT-<NN>` markers and
atomically advances the loop's consumed_int_ids + current_int_index.

Backend:
- topology_worker::advance_loop_after_completion — new post-terminal
  hook that fires alongside freeze_research_outcome. Reads loop_id_for_run
  (skips non-loop runs) + source_research_context (skips standalone
  loops without a bound source topic).
- parse_completed_int_ids — forgiving parser: matches `COMPLETED: INT-01`,
  `- COMPLETED: `INT-01``, `COMPLETED: INT-01, INT-02`, case-insensitive,
  tolerates list dashes / backticks / markdown emphasis. De-dupes within
  a single output.
- cm_db::repo::loops::advance_after_completion — atomic UPDATE that:
  · appends only NEW ids to consumed_int_ids (idempotent on re-runs)
  · bumps current_int_index by the count of new ids landed
  Set semantics via `SELECT DISTINCT unnest(...)` so ordering-based
  bugs can't accumulate duplicates.

Behavior end-to-end:
1. Loop wizard imports an integrations artifact (previous commit).
2. run_now / webhook_receive → compose_iteration_task prepends artifact
   + focus instruction ("address INT-<current+1>, log COMPLETED at end").
3. Coordinator run does the work, emits `COMPLETED: INT-<NN>`.
4. topology_worker completion hook parses the marker, advances the
   loop, and the NEXT iteration sees an updated `consumed:` list +
   incremented `current_int_index` in its focus instruction.

Follow-ups still queued:
- Loop card refresh button — pull latest artifact after reject-with-
  revision on the source topic (right now the prepend uses the LATEST
  outcome automatically, so refresh is UX only, not correctness).
- Reorder rationale extraction — coordinator emits "REORDER: INT-05
  before INT-04 because prereq X is unmet"; today that's just prose
  in the output, not indexed.
This commit is contained in:
Omar Sobh
2026-07-09 18:48:34 -07:00
parent ce73abe5ab
commit 4a140cb7db
2 changed files with 100 additions and 0 deletions
+64
View File
@@ -170,6 +170,7 @@ async fn run_job(
eprintln!("topology_worker: complete({id}) failed: {e}"); eprintln!("topology_worker: complete({id}) failed: {e}");
} }
freeze_research_outcome(pool, id, &record.final_output).await; freeze_research_outcome(pool, id, &record.final_output).await;
advance_loop_after_completion(pool, id, &record.final_output).await;
} }
Err(e) => { Err(e) => {
// Don't clobber a cancellation (or any already-terminal state) with `failed`. // Don't clobber a cancellation (or any already-terminal state) with `failed`.
@@ -209,6 +210,69 @@ async fn freeze_research_outcome(pool: &PgPool, run_id: Uuid, final_output: &str
} }
} }
/// Post-terminal hook for loop-bound runs. Parses `COMPLETED: INT-<NN>`
/// markers out of the run's final output and advances the loop's
/// `consumed_int_ids` + `current_int_index`. Only fires for runs that
/// belong to a loop AND that loop is bound to a source research topic
/// (the integrations flow). Standalone loops or unbound runs no-op.
///
/// The marker parser is deliberately forgiving — accepts INT-XX and
/// INT-XXX, optionally with surrounding backticks or dashes, so
/// coordinator prompts that emit slightly different formats still
/// advance the pointer.
async fn advance_loop_after_completion(pool: &PgPool, run_id: Uuid, final_output: &str) {
let loop_id = match cm_db::repo::topology_runs::loop_id_for_run(pool, run_id).await {
Ok(Some(id)) => id,
_ => return,
};
let ctx = match cm_db::repo::loops::source_research_context(pool, loop_id).await {
Ok(Some(c)) => c,
_ => return, // Not a research-bound loop; nothing to advance.
};
let mut completed = parse_completed_int_ids(final_output);
// Drop items already recorded so re-runs don't double-count.
let (_topic, already, _idx) = ctx;
completed.retain(|id| !already.contains(id));
if completed.is_empty() {
return;
}
if let Err(e) = cm_db::repo::loops::advance_after_completion(pool, loop_id, &completed).await {
eprintln!("topology_worker: loops::advance_after_completion({loop_id}) failed: {e}");
}
}
/// Extract stable INT-XX ids from a completion line. Matches
/// `COMPLETED: INT-01`, `COMPLETED: INT-01, INT-02`, or `- COMPLETED: `INT-01``.
/// De-duplicates within a single output.
fn parse_completed_int_ids(text: &str) -> Vec<String> {
let mut out = Vec::new();
let mut seen = std::collections::HashSet::new();
for line in text.lines() {
// Case-insensitive, tolerates surrounding whitespace, list dashes,
// markdown emphasis, and backticks.
let normalized = line.trim_start_matches(|c: char| {
c.is_whitespace() || c == '-' || c == '*' || c == '#' || c == '>'
});
let upper = normalized.to_ascii_uppercase();
if !upper.starts_with("COMPLETED:") {
continue;
}
for token in upper
.trim_start_matches("COMPLETED:")
.split(|c: char| c == ',' || c == ';' || c.is_whitespace())
{
let stripped = token.trim_matches(|c: char| c == '`' || c == '*' || c == '.');
if stripped.starts_with("INT-")
&& stripped.len() >= 5
&& seen.insert(stripped.to_string())
{
out.push(stripped.to_string());
}
}
}
out
}
/// Post-terminal hook: if this run belongs to a research topic and it was /// Post-terminal hook: if this run belongs to a research topic and it was
/// the last sibling in flight, transition the topic `processing → reviewing`. /// the last sibling in flight, transition the topic `processing → reviewing`.
/// Best-effort — a DB hiccup here logs but doesn't fail the run. /// Best-effort — a DB hiccup here logs but doesn't fail the run.
+36
View File
@@ -247,6 +247,42 @@ pub async fn set_zeroclaw_container(
Ok(()) Ok(())
} }
/// Atomically append `completed` INT-XX ids to the loop's
/// `consumed_int_ids` array and bump `current_int_index` by the count
/// of NEW ids landed. Existing ids are not re-appended (idempotent on
/// re-runs). Called from the topology_worker completion hook after
/// parsing "COMPLETED: INT-XX" markers out of the run's final output.
pub async fn advance_after_completion(
pool: &PgPool,
loop_id: Uuid,
completed: &[String],
) -> Result<(), DbError> {
if completed.is_empty() {
return Ok(());
}
// Use array set semantics: append only ids not already present.
// The subquery computes the new list; length delta feeds the index bump.
sqlx::query(
"UPDATE loops
SET consumed_int_ids = (
SELECT ARRAY(
SELECT DISTINCT unnest(consumed_int_ids || $2::TEXT[])
)
),
current_int_index = current_int_index + (
SELECT count(*) FROM unnest($2::TEXT[]) AS n(v)
WHERE NOT (consumed_int_ids @> ARRAY[v])
),
updated_at = now()
WHERE id = $1",
)
.bind(loop_id)
.bind(completed)
.execute(pool)
.await?;
Ok(())
}
/// Bind (or unbind) a loop's source research topic. When set, the loop's /// Bind (or unbind) a loop's source research topic. When set, the loop's
/// enqueue path prepends the topic's latest artifact + a "focus on the /// enqueue path prepends the topic's latest artifact + a "focus on the
/// next unconsumed INT" instruction to the coordinator task (option b, /// next unconsumed INT" instruction to the coordinator task (option b,