Omar Sobh
4ada5557f2
Foundation for folding research into loops as a first-class kind.
This commit ships the plumbing; the research-kind dispatch itself
lands next. Behavior for existing exec-kind loops is unchanged unless
they opt into the new trigger fields.
Migration 0044:
- kind TEXT NOT NULL DEFAULT 'exec' CHECK ('exec' | 'research'). New
research-kind will run the research pipeline each iteration (next
commit); 'exec' preserves today's behavior.
- initial_burst_remaining INT NOT NULL DEFAULT 0 — countdown for the
triggers.initial_burst quota. Decremented CAS-safely on each
completion until it hits 0.
- Two partial indexes: (kind, source_research_topic_id) for kind-
aware lookups, and (source_research_topic_id) filtered on
on_artifact_update=true + enabled=true for the fan-out hook.
Trigger schema extended with two optional fields:
- initial_burst: N — fire N iterations back-to-back at create time.
create_loop enqueues the first iteration inline (subject to
empty-roster gate), sets remaining=N-1, and the completion hook
continues the chain until exhausted.
- on_artifact_update: true — when a bound research_outcomes row is
inserted for the source topic, wake up one iteration of this loop.
Coalesced against has_active_run so a burst of rapid revisions
doesn't queue duplicates.
Backend:
- cm_db::repo::loops helpers (all dynamic sqlx, no offline cache
regen needed):
- set_initial_burst_remaining
- take_initial_burst_slot (CAS UPDATE returning prev value; 0 on
exhausted or race loss)
- loops_awaiting_topic (fan-out query: kind=exec + enabled +
on_artifact_update=true bound to the given topic)
- has_active_run (queued|running iteration existence check)
- get_any_workspace (bypasses the workspace scope guard; used by
the completion hook where the run row is authoritative)
- routes/loops::compose_iteration_task made pub so the completion
hook can build the same enriched task string as run_now.
- topology_worker::freeze_research_outcome now fans out to awakened
loops after the outcome insert, using compose_iteration_task and
coalescing on has_active_run.
- topology_worker::continue_initial_burst runs on every completion:
· take_initial_burst_slot (CAS) — no-op if already exhausted
· has_active_run coalesce guard
· re-fetches the loop via get_any_workspace + compose_iteration_task
· enqueues via loops::enqueue_iteration with parent_run_id set
Follow-ups already queued:
- kind='research' dispatch in run_job — build the research
coordinator task from the topic config, run the research pipeline
each iteration. Requires factoring start_topic's task-build.
- ResearchWizard "When should this run?" step (Just once / Nightly /
Manual) creating the topic + paired research-kind loop.
- LoopsWizard trigger UI matching the design proposal (burst count,
cron, on-artifact checkbox).
40 lines
2.0 KiB
SQL
40 lines
2.0 KiB
SQL
-- Folds research into loops as a "kind" of loop. Two kinds exist:
|
|
--
|
|
-- 'exec' — today's behavior. Each iteration runs the coordinator
|
|
-- task_template (optionally prepended with a research
|
|
-- artifact + focus instructions for INT-consumption
|
|
-- loops).
|
|
-- 'research' — the fire runs the research pipeline against a bound
|
|
-- topic config, appending a versioned research_outcomes
|
|
-- row each time. Used for nightly paper-survey loops
|
|
-- (D1 answer: every runnable thing is a loop; even a
|
|
-- one-shot topic gets its own kind='research' loop with
|
|
-- initial_burst=1).
|
|
--
|
|
-- Default 'exec' + CHECK constraint keeps every existing row valid.
|
|
-- The paired research + coding pattern (nightly research produces
|
|
-- outcome, coding loop wakes on on_artifact_update to consume next INT)
|
|
-- composes two loops of different kinds sharing a source topic id.
|
|
ALTER TABLE loops
|
|
ADD COLUMN kind TEXT NOT NULL DEFAULT 'exec'
|
|
CHECK (kind IN ('exec', 'research')),
|
|
-- Countdown for the triggers.initial_burst quota. On create_loop
|
|
-- we enqueue the first iteration inline and set this to burst-1;
|
|
-- on each completion the topology_worker checks and, when > 0,
|
|
-- enqueues the next iteration + decrements. Once it hits 0 the
|
|
-- burst is exhausted (further iterations happen only through
|
|
-- cron / webhook / on_completion / on_artifact_update).
|
|
ADD COLUMN initial_burst_remaining INT NOT NULL DEFAULT 0;
|
|
|
|
CREATE INDEX loops_kind_source_idx
|
|
ON loops (kind, source_research_topic_id)
|
|
WHERE source_research_topic_id IS NOT NULL;
|
|
|
|
-- Fast lookup for the on_artifact_update hook: given a topic_id, find
|
|
-- all exec-kind loops bound to it with the trigger enabled.
|
|
CREATE INDEX loops_on_artifact_update_idx
|
|
ON loops (source_research_topic_id)
|
|
WHERE source_research_topic_id IS NOT NULL
|
|
AND (triggers ->> 'on_artifact_update')::boolean = true
|
|
AND enabled = true;
|