2c593c32ae4b4362ff91759932d65d231242d12c
24
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Omar Sobh
|
d8c8793c4a |
ci fixes: cargo fmt, eslint entities, max-lines split
CI on
|
||
|
Omar Sobh
|
fdb8cfeecc |
slice 9 cleanup: drop legacy research/loops backend + tables
Retires the legacy research/loops backend after the missions arc
(slices 1-9) fully replaced it. Frontend cutover was 4663348; this
commit finishes the job on the backend + database.
Migration:
- 0053_drop_legacy_research_loops.sql — drops the 8 legacy tables
(research_topics, research_topic_agents, research_outcomes,
research_publish_approvals, loops, loop_agents, loop_orgs,
loop_teams) and the 3 topology_runs FK columns
(research_topic_id, loop_id, iteration). parent_run_id stays;
recursive_exec still uses it.
Files deleted (11):
- crates/cm-api/src/routes/{research,loops,research_setup,
research_pipeline,wizard_repo,probe}.rs
- crates/cm-api/src/research_container.rs
- crates/cm-db/src/repo/{research_topics,research_outcomes,
research_publish_approvals,loops}.rs
- crates/cm-runtime/src/loops.rs
- crates/cm-api/tests/research_publish_role.rs
Files edited:
- crates/cm-api/src/lib.rs — dropped 20 legacy route registrations
(all /api/research/* + /api/loops/* + /webhooks/loops + probe)
and module decls
- crates/cm-api/src/topology_worker.rs — deleted legacy dispatch
(freeze_research_outcome, advance_loop_after_completion,
continue_initial_burst, maybe_transition_research_topic,
parse_reorder_rationale, per-topic/loop gateway resolver).
reap_stuck_runs now keys on mission_id (not topic_id).
Executor path unconditionally uses ZeroClawDriveExecutor::from_env
— mission_orchestrator provisions each claw as an agent inside
the shared runtime via RuntimeProvisioner, so per-team gateway
resolution is no longer applicable.
- crates/cm-api/src/routes/topology.rs — deleted container-log SSE
endpoint (research/loop-specific), dropped loop_id filter and
iteration field from ListRunsQuery/RunSummary
- crates/cm-api/src/routes/world.rs — removed
active_research_topics/active_loops/preseed_repo_paths;
World SSE no longer emits repo:{topic}/loop:{id} landmark orbs
(follow-up task #21 tracks adding mission:{id} equivalents)
- crates/cm-api/src/runtime_provision.rs — removed now-unused
mint_workspace_service_token
- crates/cm-db/src/repo/topology_runs.rs — removed 9 legacy
helpers (research_topic_id lookup, loop_id_for_run,
iteration_for_run, active_runs_for_research_topic, etc.)
- crates/cm-db/src/repo/teams.rs — removed 4 dead helpers
(team_for_loop, team_for_research_topic + setters)
- crates/cm-api/tests/topology_jobs.rs — removed loop/topic
tests, dropped enqueue_run_with_topic helper
- crates/bins/clawmates-server/src/main.rs — removed
spawn_loop_scheduler call
- crates/cm-api/src/routes/mod.rs, crates/cm-db/src/repo/mod.rs,
crates/cm-runtime/src/lib.rs — module decls stripped
sqlx cache: regenerated against post-migration schema
(71 files changed, ~+70 / -8896 net)
Test/build: SQLX_OFFLINE=true cargo check --workspace clean;
cargo test --workspace --no-run clean.
Follow-up (task #21): World view lost the in-flight-work landmarks
when repo:{topic} / loop:{id} orbs disappeared. Add mission:{id}
orbs as the missions-era replacement.
|
||
|
Omar Sobh
|
68d84bf1ad | fmt: apply cargo fmt to topology_worker.rs | ||
|
Omar SobhandClaude Opus 4.7
|
2a99bba6c1 |
topology worker: fail runs when team container spawn fails, don't silently degrade
Coding loops with team_id set MUST run inside their team container so
agents have /workspace/repo bind-mounted. When try_team_gateway_url
silently returned None on error (mkdir perm denied, spawn_team failed,
etc.), the executor fell through to the shared runtime — which has no
repo mount — and the coding agents narrated for 6 minutes without
touching a file. Runs completed "green" with zero commits, hiding real
infra breakage.
Change try_team_gateway_url from Option<String> to Result<Option<String>, String>:
- Ok(Some) — team spawned/reattached; drive it.
- Ok(None) — no team binding on this run; existing per-topic/per-loop
fallback resolvers still apply.
- Err — team was expected but spawn failed; caller fails the run
with a descriptive error instead of silently degrading.
Each failure path in try_team_gateway_url now formats a specific
diagnostic string (docker connect, spawn_team, missing repo_workspace_path,
etc.) that surfaces into topology_runs.error.
The MCP-bearer mint failure is deliberately kept as a warning + best-
effort: some tools will 401 but the container still boots. Only failures
that would prevent code from being touched escalate to Err.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
|
||
|
Omar Sobh
|
65b19b0fc5 | fmt: trim trailing blank line in topology_worker.rs | ||
|
Omar SobhandClaude Opus 4.7
|
657b666219 |
mcp door: extend service-session bearer to per-topic + per-loop spawns
Per-team runtimes had their MCP bearer swap wired in the prior slice, but per-topic (research_container::spawn) and per-loop (spawn_loop) containers still baked the stale template bearer and 401'd every tools/list. Same fix, extended: mint a workspace-owner service session via runtime_provision::mint_workspace_service_token and inject via prewrite_daemon_config_with_risk. Move mint_workspace_service_token from topology_worker into runtime_provision so all three spawn call sites share the helper. Callers updated: routes/research.rs (start_topic), routes/research_setup.rs (prepare_topic_runtime), routes/loops.rs (ensure_loop_container). Co-Authored-By: Claude Opus 4.7 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.7
|
49f94a5360 |
mcp door: mint workspace-owner service session for team runtime bearer
The runtime template's static clawmates_door bearer is rejected by cm_auth::authenticate() (needs an auth_sessions row). Every per-team agent was getting `unauthorized: missing or invalid bearer token` and `0 tool(s) registered from 0 server(s)`. Add AuthService::mint_service_session + users::owner_of_workspace and mint a 30d service session in try_team_gateway_url; inject it into the freshly-spawned team container's config.toml [[mcp.servers]] clawmates Authorization header via prewrite_daemon_config_with_risk (bearer arg). Follow-up: apply the same pattern to research::spawn (per-topic) and per-loop spawn paths. Co-Authored-By: Claude Opus 4.7 <[email protected]> |
||
|
Omar Sobh
|
0d0bb5ffaa |
teams: runtime spawn hookup for per-team containers (slice 3b)
Wires the per-loop-team arc end-to-end. When a loop with team_id fires an iteration, the worker now spawns/reattaches a dedicated team container, mounts the paired research topic's repo at /workspace/repo (rw), and stamps the team's risk_profile into every [agents.*] binding on the freshly-written config.toml. Legacy loops with team_id = NULL keep taking the per-topic / per-loop path unchanged. research_container.rs - team_container_name_for(team_id) = 'team-<uuid>-container' - team_state_root(team_id) — /var/lib/clawmates-team-state/<uuid>/state (overridable via CLAWMATES_TEAM_STATE_ROOT) - prewrite_daemon_config_with_risk: line-based sed that swaps only the risk_profile line inside each [agents.<name>] block. Avoids the regex-eats-array-literal trap that bricked the shared runtime config on the earlier out-of-band edit. - spawn_team: full-shape idempotent spawner. Same mount + env + label pattern as spawn/spawn_loop; additionally supports Claude settings bind-mount + external-bridge attach. topology_worker.rs - try_team_gateway_url resolver runs BEFORE the existing per-topic and per-loop lookups. Cold path: reads team runtime config + paired research topic repo path, spawns the container, persists coords back to teams.zeroclaw_container/zeroclaw_gateway_url. Any failure logs + returns None so the caller falls through to the legacy shared-container path — team spawn can never brick a run that could otherwise complete. Not shipped in this slice: - Wizard 'existing team' picker (currently just fresh vs reuse) - Teams tier UI to list/edit/delete teams - Auto-teardown for stale team containers (piggyback on existing reaper is a follow-up) |
||
|
Omar Sobh
|
067bebd071 |
wedged-run fix: pre-approve claude perms + verbose step log + reaper
Root cause: claude CLI in the per-topic research container runs as
uid=0(root). ZeroClaw's claude_cli provider passes
--dangerously-skip-permissions which Claude CLI rejects under root
for security — so the CLI hangs waiting for interactive permission
approval that never arrives, hitting the 600s provider timeout with
zero step records journaled.
Three-part fix:
1. Claude settings bind-mount (research_container.rs):
Optional CLAWMATES_CLAUDE_SETTINGS_PATH env — when set, mount the
host file at /root/.claude/settings.json (read-only) in every
spawned team container. deploy/claude-settings.json ships the
canonical config (permissions.defaultMode = bypassPermissions +
hasCompletedOnboarding). CLI accepts requests immediately with no
--dangerously-skip-permissions flag needed.
2. Verbose per-step log line (topology_worker.rs):
Every checkpoint now writes to stderr:
topology_worker::step run_id=X step=N node=Y role=Z phase=W
output_bytes=B tokens=T gated=G
Visible in docker logs clawmates_server_1 — gives us live
'topology is flowing' signal without opening the canvas, and
makes it obvious when a topology_kind is skipping stages it
shouldn't.
3. Stuck-container reaper (topology_worker.rs):
New 60s-tick loop reap_stuck_runs: for any research topology_run
older than 15 min with zero checkpoint.records, docker-stop its
container and mark the run failed with a diagnostic error. Only
reaps research-bound runs (non-research runs don't own a
container). The existing 180s stale-checkpoint requeuer stays
in place for other failure modes.
Deploy: gw-04 needs
ln -sf /path/to/repo/deploy/claude-settings.json /opt/clawmates/claude-settings.json
CLAWMATES_CLAUDE_SETTINGS_PATH=/opt/clawmates/claude-settings.json
in the server env, plus the timeout lowered from 600 -> 120 in
compose. Both handled in the deploy step outside this commit.
|
||
|
Omar Sobh
|
91a51dce11 |
loops: kind='research' dispatch — research runs as loops
Delivers the research/loop fold: kind='research' loops run the research pipeline each iteration, appending a new research_outcomes version. The paired on_artifact_update fan-out then wakes up any kind='exec' loops bound to the same topic to consume new INTs. Every runnable thing is now a loop (D1). Backend — DB helpers: - cm_db::repo::loops::kind_and_binding — reads (kind, source_topic, task_template) so callers can dispatch without hydrating the whole Loop struct. - cm_db::repo::loops::enqueue_iteration_with_topic — new variant that sets research_topic_id on topology_runs alongside loop_id, so the completion hook's freeze_research_outcome writes a new outcome version for research-kind iterations. - cm_db::repo::research_topics::get_any_workspace — cross-workspace fetch used by the research task builder (the loop row is authoritative for the workspace binding via kind_and_binding). Backend — dispatch: - routes::loops::compose_research_iteration_task — builds the coordinator prompt for a research iteration: topic title + description + outcome_kind + prior version pointer + refresh instructions (survey new sources, preserve stable INT ids, mark superseded items as deprecated rather than delete). The completion hook writes the resulting synthesis as research_outcomes v(prior+1). - routes::loops::compose_and_enqueue_iteration — one-shot dispatch: reads the kind, picks compose_iteration_task (exec) or compose_research_iteration_task (research), enqueues with or without research_topic_id set. All four enqueue callsites now route through compose_and_enqueue: - create_loop (initial_burst) - run_now - webhook_receive - topology_worker::continue_initial_burst - topology_worker::freeze_research_outcome (on_artifact_update fan-out) Research-kind loops naturally form the "nightly refresh" side of a paired research + coding loop: research writes a fresh outcome version → fan-out wakes exec loops with on_artifact_update → coding loops consume the next INT (which the research loop may have just added). D2 answer (inherit repo binding): repo lives on the topic; both loops sharing the source topic id read from the same context, no duplication. D3 answer (coordinator resolves): the research iteration prompt tells the team to preserve stable INT ids and mark deprecations rather than delete, so coding loops' consumed lists stay valid across versions. Follow-up (next commit): ResearchWizard schedule step — "Just once / Nightly / Manual" that creates the paired research-kind loop with initial_burst=1 (just once) or cron 0 3 * * * (nightly) + optional paired coding loop with on_artifact_update. |
||
|
Omar Sobh
|
4ada5557f2 |
loops: kind column + initial_burst + on_artifact_update trigger fan-out
Foundation for folding research into loops as a first-class kind.
This commit ships the plumbing; the research-kind dispatch itself
lands next. Behavior for existing exec-kind loops is unchanged unless
they opt into the new trigger fields.
Migration 0044:
- kind TEXT NOT NULL DEFAULT 'exec' CHECK ('exec' | 'research'). New
research-kind will run the research pipeline each iteration (next
commit); 'exec' preserves today's behavior.
- initial_burst_remaining INT NOT NULL DEFAULT 0 — countdown for the
triggers.initial_burst quota. Decremented CAS-safely on each
completion until it hits 0.
- Two partial indexes: (kind, source_research_topic_id) for kind-
aware lookups, and (source_research_topic_id) filtered on
on_artifact_update=true + enabled=true for the fan-out hook.
Trigger schema extended with two optional fields:
- initial_burst: N — fire N iterations back-to-back at create time.
create_loop enqueues the first iteration inline (subject to
empty-roster gate), sets remaining=N-1, and the completion hook
continues the chain until exhausted.
- on_artifact_update: true — when a bound research_outcomes row is
inserted for the source topic, wake up one iteration of this loop.
Coalesced against has_active_run so a burst of rapid revisions
doesn't queue duplicates.
Backend:
- cm_db::repo::loops helpers (all dynamic sqlx, no offline cache
regen needed):
- set_initial_burst_remaining
- take_initial_burst_slot (CAS UPDATE returning prev value; 0 on
exhausted or race loss)
- loops_awaiting_topic (fan-out query: kind=exec + enabled +
on_artifact_update=true bound to the given topic)
- has_active_run (queued|running iteration existence check)
- get_any_workspace (bypasses the workspace scope guard; used by
the completion hook where the run row is authoritative)
- routes/loops::compose_iteration_task made pub so the completion
hook can build the same enriched task string as run_now.
- topology_worker::freeze_research_outcome now fans out to awakened
loops after the outcome insert, using compose_iteration_task and
coalescing on has_active_run.
- topology_worker::continue_initial_burst runs on every completion:
· take_initial_burst_slot (CAS) — no-op if already exhausted
· has_active_run coalesce guard
· re-fetches the loop via get_any_workspace + compose_iteration_task
· enqueues via loops::enqueue_iteration with parent_run_id set
Follow-ups already queued:
- kind='research' dispatch in run_job — build the research
coordinator task from the topic config, run the research pipeline
each iteration. Requires factoring start_topic's task-build.
- ResearchWizard "When should this run?" step (Just once / Nightly /
Manual) creating the topic + paired research-kind loop.
- LoopsWizard trigger UI matching the design proposal (burst count,
cron, on-artifact checkbox).
|
||
|
Omar Sobh
|
0c17de52dd |
loops: reorder rationale extraction — REORDER: markers logged per iteration
Coordinator can now log WHY it worked on an INT-XX out of order
("REORDER: INT-05 before INT-04 because prereq X is unmet") and the
completion hook captures each rationale as an append-only event on
the loop. Sets up a reviewable timeline of when the plan was
adjusted, independent of the underlying `consumed_int_ids` advance.
Migration 0043:
- loops.reorder_events JSONB NOT NULL DEFAULT '[]'::jsonb — append-
only array of {run_id, iteration, text, ts}. Kept on the loop row
(rather than a dedicated table) so the mini-timeline is one read
away from the loop card.
Backend:
- topology_worker::parse_reorder_rationale — line matcher symmetric
with parse_completed_int_ids. Tolerates list dashes / prefixes /
markdown emphasis; case-insensitive marker match, preserves case of
the rationale text.
- cm_db::repo::loops::append_reorder_event — one INSERT-like append
per rationale, uses jsonb_build_object with postgres now() so ts is
wall-clock canonical (no client-clock skew).
- topology_runs::iteration_for_run — new helper so events carry the
iteration index.
- routes::loops::compose_iteration_task — coordinator prompt now
explicitly asks for `REORDER: <one-sentence>` at the top of the
first substantive turn when working out of order, AND spells out
that both markers must appear literally with colons (no bold, no
code fence) so the line parser doesn't miss them.
Non-loop and standalone-loop runs are unaffected — the hook only
fires when the run belongs to a source-bound loop.
Follow-up: expose reorder_events on the loops list endpoint + render
a small collapsed timeline on the LoopsList card.
|
||
|
Omar Sobh
|
4a140cb7db |
loops: parse COMPLETED: INT-XX markers to advance loop pointer (option b)
Closes the loop bridge — the missing piece from the previous commit.
Without this, `current_int_index` stayed at 0 forever and every
iteration re-worked INT-01. Now the topology_worker's completion hook
parses the run's final output for `COMPLETED: INT-<NN>` markers and
atomically advances the loop's consumed_int_ids + current_int_index.
Backend:
- topology_worker::advance_loop_after_completion — new post-terminal
hook that fires alongside freeze_research_outcome. Reads loop_id_for_run
(skips non-loop runs) + source_research_context (skips standalone
loops without a bound source topic).
- parse_completed_int_ids — forgiving parser: matches `COMPLETED: INT-01`,
`- COMPLETED: `INT-01``, `COMPLETED: INT-01, INT-02`, case-insensitive,
tolerates list dashes / backticks / markdown emphasis. De-dupes within
a single output.
- cm_db::repo::loops::advance_after_completion — atomic UPDATE that:
· appends only NEW ids to consumed_int_ids (idempotent on re-runs)
· bumps current_int_index by the count of new ids landed
Set semantics via `SELECT DISTINCT unnest(...)` so ordering-based
bugs can't accumulate duplicates.
Behavior end-to-end:
1. Loop wizard imports an integrations artifact (previous commit).
2. run_now / webhook_receive → compose_iteration_task prepends artifact
+ focus instruction ("address INT-<current+1>, log COMPLETED at end").
3. Coordinator run does the work, emits `COMPLETED: INT-<NN>`.
4. topology_worker completion hook parses the marker, advances the
loop, and the NEXT iteration sees an updated `consumed:` list +
incremented `current_int_index` in its focus instruction.
Follow-ups still queued:
- Loop card refresh button — pull latest artifact after reject-with-
revision on the source topic (right now the prepend uses the LATEST
outcome automatically, so refresh is UX only, not correctness).
- Reorder rationale extraction — coordinator emits "REORDER: INT-05
before INT-04 because prereq X is unmet"; today that's just prose
in the output, not indexed.
|
||
|
Omar Sobh
|
44d6e95022 |
fmt: apply rustfmt across the P2 arc
CI's rustfmt check flagged the multi-line sqlx::query() calls I introduced in P2 (loop_id_for_run, zeroclaw_gateway_url, etc.). No behavior change — pure formatting. |
||
|
Omar Sobh
|
03f1830d1f |
loops: Path B container isolation (P2)
Symmetric with the research pipeline: every enabled loop can now have its own per-loop team container so scheduled runs don't share state with other loops or with research. Same daemon image, same clawmates network, deterministic name loop-<id>-team. Backend surface: - Migration 0040 adds nullable `zeroclaw_container` + `zeroclaw_gateway_url` columns to loops (parallel to research_topics). - research_container.rs grows loop_container_name_for(), spawn_loop() (state-only mount, no repo), and teardown_loop(). Kept in the same module to share the docker connect() + inherited_env() plumbing; each pattern gets its own labels (clawmates.role=loop-team) so ps filters can tell them apart. - cm_db::repo::loops gains set_zeroclaw_container() + zeroclaw_gateway_url() (dynamic sqlx queries — no offline cache regen needed). - cm_db::repo::topology_runs gets loop_id_for_run(): mirror of research_topic_id, used by the worker. Wiring: - routes/loops::run_now + webhook_receive call ensure_loop_container() before enqueuing an iteration. Idempotent: an already-running container is just reattached. Failures are logged and do NOT block the enqueue — topology_worker falls back to the workspace gateway when the URL isn't set on the loop. - routes/loops::disable_loop + delete_loop both fire teardown_loop() so paused / deleted loops don't hold a docker slot. - topology_worker's per-run URL resolution: existing research fast path unchanged; when it doesn't hit, the worker now looks up loop_id and reads the loop's gateway URL. Deploy step (required on gw-04 for state to persist across container restarts): add a `/var/lib/clawmates-loops:/var/lib/clawmates-loops` bind mount + `CLAWMATES_LOOPS_STATE_ROOT=/var/lib/clawmates-loops` env var to clawmates_server_1 in the compose. Without it, loops still run — the state dir lives inside the API container's filesystem so persistence is limited to that container's lifetime. Follow-up: - Scheduler-tick fires (cron-driven, not run_now) — they call enqueue_iteration in cm-scheduler and don't yet go through ensure_loop_container. Add a symmetric spawn there so cron fires also land on the isolated daemon. - Compose file reconciliation — deploy/compose/docker-compose.yml in the repo has drifted from prod; when we sync it, add the loops mount at the same time. |
||
|
Omar Sobh
|
88c78bd16e |
research: topology_worker points executor at per-topic gateway (commit 2/3)
Commit 2 of the path-B plan. The container that commit 1 spawns
now actually receives the run's turns — up until now it was
started but unused. This is the payoff commit: research runs are
truly isolated per topic.
Backend
- topology_exec.rs: from_env() refactored to a thin wrapper over a
new from_env_for_gateway(url) helper. Same shape (env-derived
aliases + default + token) but the caller supplies the URL. The
auth token/pairing code still comes from ZEROCLAW_TOKEN /
ZEROCLAW_PAIRING_CODE on the server; research_container's
inherited_env propagates those into the team container so the
same credentials work at both endpoints.
- research_container.rs: new wait_ready(url, deadline) that polls
<url>/health with a 1.5s per-request timeout every 500ms until
it 200s or the deadline passes. reqwest-based so it doesn't need
bollard. Called by the worker after claim, before pair, to bridge
the "container is starting, gateway not yet listening" gap.
- topology_worker.rs run_job:
1. Look up research_topic_id for the claimed run.
2. If Some, load the topic and read zeroclaw_gateway_url.
3. If a URL is present:
- best-effort wait_ready(url, 30s); a timeout logs but
doesn't abort — the pair call below will just fail
faster than pinging forever
- build the leaf via from_env_for_gateway(url)
Else fall back to from_env() (workspace-wide gateway).
4. The rest of run_job is unchanged — the leaf drops into
either SubTopologyExecutor (org/company) or direct drive
(team tier) as before.
What now works end-to-end
Starting a research topic with a bound repo:
1. clone-shallow into per-topic workspace
2. docker create + start the clawmates-runtime container, name
= research-<topic>-team, joined to clawmates_core so the
server reaches it by name
3. persist container name + gateway URL on the topic row
4. enqueue the topology run tagged with research_topic_id
5. worker claims → looks up the topic → waits for the team
gateway's /health → constructs a from_env_for_gateway
executor pointed at http://research-<topic>-team:42617
6. every turn's `/ws/chat?agent=…` hits the isolated container;
agents inside see the repo at /workspace/repo (rw); each
topic's memory/state lives under its own /zeroclaw-data mount
Deploy prereqs (unchanged from commit 1)
- clawmates_server compose service needs a bind-mount of
CLAWMATES_RESEARCH_WORKSPACE_ROOT so the paths spawn() writes to
are visible on the host and the spawned team container mounts
the same underlying data.
- socket-proxy ACL needs POST + DELETE on /containers (prod ✓).
|
||
|
Omar Sobh
|
a2d3d85ebe |
research pipeline v2: topology-aware start + persisted draft
Three connected changes that turn "Start research" from a status flip into a real pipeline that produces a reviewable artifact: - Migration 0036: adds research_topics.topology_kind (default 'hub_spoke') and a new research_outcomes table (id, topic_id, version DESC, body_md, produced_by_run_id, created_at) so each run's final synthesis is versioned and persistent. - Wizard now has a topology picker in the Outcome step — hub_spoke / pipeline / hierarchical / star_moe — with copy that steers users to the right shape (Pipeline for research → distill → analyze → implement rosters, hub_spoke for the coordinator- and-specialists default). - start_topic reads the chosen topology_kind, parses it into a cm_topology::TopologyKind, and dispatches a per-shape coordinator prompt via build_coordinator_task. Pipeline explicitly tells stage 1 not to write the final artifact and propagates a "final stage MUST emit a complete markdown document with measurable acceptance criteria" instruction downstream. The graph builder is called with the topology the user actually picked instead of hard-coded HubSpoke. - topology_worker::freeze_research_outcome fires after every successful complete(). It looks up research_topic_id on the run; if set and final_output is non-empty, it inserts a new research_outcomes row (version auto-derived server-side via coalesce(max(version), 0) + 1). Best-effort — a DB hiccup logs but doesn't fail the run. - TopicDetail now includes topology_kind and latest_outcome. ResearchCanvas swaps in the outcome's body_md (rendered as pre-wrap markdown, versioned header, produced-at timestamp) whenever an outcome exists; the original prompt collapses into an "Original prompt" <details> below so it's still one click away. Pre-run topics still show the description as before. Follow-ups still open: reject-with-revision loop feeding the coordinator, publishing → published transition + real artifact export (md / pdf), and an approvals inbox surface for reviewers. |
||
|
Omar Sobh
|
806ba869e5 |
teams: ephemeral lifecycle for Scheduled + Triggered planner modes
Migration 0033: adds teams.lifecycle ('permanent' | 'ephemeral') and a
topology_runs.team_id back-ref with a partial index for the sibling-in-
flight check.
cm-db repo:
- teams::insert_team_with_lifecycle (insert_team keeps the permanent default)
- topology_runs::enqueue_run_for_team (populates team_id)
- topology_runs::check_ephemeral_teardown — atomic SELECT that only
returns Some when the team is ephemeral AND no siblings are still
queued/running; carries the workspace + bound claw ids for cleanup.
cm-api:
- topology_worker post-terminal hook maybe_teardown_ephemeral_team
runs deprovision_claw on each bound claw (best-effort; failures log
but don't block Postgres deletion), then hard_purge each agent row,
then delete_team.
- routes::teams::build_team_with_lifecycle (build_team keeps default);
run_team enqueues with team_id.
- planner ScaffoldRequest gains mode; lifecycle_for(mode) sets the team
to ephemeral for scheduled + triggered, permanent otherwise.
Frontend MasterPlannerModal passes mode in the scaffold payload so the
backend can derive lifecycle without duplicating the mode taxonomy.
Tests: 3 new (returns claws when no siblings, holds when siblings queued,
ignores permanent teams). 10/10 topology_jobs green; workspace clippy
--tests clean.
|
||
|
Omar Sobh
|
8a4e222aec |
research: auto-transition processing → reviewing on last run
Hooks the topology_worker's post-terminal path into a new notify_run_completed repo helper that atomically transitions the topic processing → reviewing when the completed run has research_topic_id set AND no siblings for that topic are still queued or running. Guarded on status='processing' so a retry, a re-fire, or a topic already past processing are all no-ops. Best-effort at the worker; DB hiccups are logged and never fail the run. The manual /submit-review endpoint stays as an escape hatch for topics that end up parked in processing with nothing to complete (updated the doc comment). |
||
|
Omar SobhandClaude Opus 4.8
|
3554a3aaf2 |
CI: remove k8s stages, fix the Docker-level pipeline green
Survey + fixes so the pipeline passes at the Docker level (no k8s).
- Remove k8s: drop the `sandbox-k8s` job (kind/Calico/--features k8s-tests) and the
"Helm chart lints" gate step. release.yml was already k8s-clean.
- Rust job:
- `cargo fmt --all` — fix pre-existing formatting drift (fmt --check was failing).
- clippy -D warnings: fix 3 lib warnings (cm-brain sort_by_key→Reverse, cm-api
fleet.rs doc list indentation, node_rules map_or→is_none_or).
- Regenerate the .sqlx offline cache (was missing the cm-runtime run_loop test
query → offline compile failed). DB-backed tests use testcontainers at runtime.
- Set SQLX_OFFLINE=true on the rust + e2e jobs so query! macros compile against
the committed cache deterministically (no DB needed at compile time).
- Frontend job:
- Fix the 1 ESLint error (useAgentTelemetry: no setState-synchronously-in-effect;
tag the slice with agentId + derive null on mismatch).
- Fix 2 stale panel-params tests (`terminal` is a valid app id now; assert the
current APP_IDS + use a genuinely-unknown id for the reject case).
Verified locally: fmt clean, clippy --all-targets -D warnings clean (offline),
frontend lint 0 errors, tsc clean, 86/86 frontend tests pass, build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
34f744734b |
Large World graph, agent platform, brain stack & dashboard rebuild
Frontend - Large World: collapse org/company/team tiers into one expandable React Flow hierarchy (WorldFlow) with per-click expand, persisted node positions, a compact tree sidebar, wrench multi-select delete across levels, and a sized right slide-out (phone/tablet/full) showing an agent summary + drill button. - Agent page: GitHub-style animated contribution grid (VitalsCard), collapsible System Prompt + Personality cards, restructured anatomy cards, bigger avatar with name/title header row, Markdown/JSON-aware rendering, brain registry + history, avatar generate/upload. - User-icon menu (Infrastructure/Brains/Tools/Profile/Credits) + ToolPanel; Master Planner deploy wizard (Specialists/Swarm/Scheduled/Triggered); Team Runs view; reap-progress modal; dashboard is the single live interface. Backend - cm-brain crate (.brain as the agent definition) + brain apply/history. - Hard-purge reap (FK-ordered) + sandbox release + SSE batch-delete. - Swarm self-verifying loop, mode-aware planner, web.search tool, webhooks (migration 0013), org/company/team delete endpoints, scheduler sweeps. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
3eca4ed70c |
Recursive deploy ladder: Company + Org tiers, mesh mark, two-tier rail
Completes the scale ladder (single → team → company → org). Every tier is a
topology whose nodes are the tier below; running a parent recursively runs each
child's sub-topology down to the leaf claws.
Backend:
- migration 0011: companies/company_teams, orgs/org_companies, topology_runs.tier
- cm-db repos for companies + orgs (mirror teams)
- TurnRequest.attrs (forwarded from node.attrs) for child-id binding
- SubTopologyExecutor (recursive_exec.rs): a parent "turn" runs the child's
sub-topology; durability via parent updated_at keepalive + cancel propagation
+ depth cap; boxed future breaks the org→company recursion
- topology_worker selects executor by job.tier
- routes: /api/companies, /api/orgs (create/list/get/run) + unified
/api/structure/{level}/{id} for the zoom canvas
Frontend:
- MeshMark: node-mesh brand glyph (replaces the claw PNG), tier variants
- TopologyGraphView: optional onNodeClick/nodeMeta + dark-token theming
- StructureCanvas + Breadcrumb: one recursive zoom view for every tier
(drill down on node click, breadcrumb up); TeamRunPanel extracted + shared
- two-tier Discord-style rail: StructureRail (mesh mark + org/company/team
glyphs + tools popover + deploy + user) | RosterColumn (selected group's
children, or your claws); SecondaryNav for cross-cutting tools
- ComposeWizard (company/org) wired into DeployWizard; /companies + /orgs pages
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
||
|
Omar SobhandClaude Opus 4.8
|
6e87433c66 |
Close gaps: GLM-judge (anthropic registry) + run cancel + deep-link
#2 GLM judge (closes #114): registry NamedProvider gains a `format` field; build_provider_registry builds an AnthropicProvider for format="anthropic". GLM's coding/OpenAI endpoint is ToS-throttled for raw SDK, but its Anthropic endpoint (api.z.ai/api/anthropic) accepts raw API calls (verified x-api-key -> glm-4.7), so CLAWMATES_JUDGE_MODEL=glm:glm-4.7 routes the door governor / topology judge through GLM with no runtime-routing. (Kimi-as-judge still needs a Platform key — coding key is agent-only.) #3 run-control: POST /api/topology-runs/{id}/cancel (workspace-scoped, queued/running only); the worker honors it at the step boundary (checks current_status in the checkpoint callback) and won't clobber a cancel with `failed`. Frontend Run tab gains a Cancel button and clickable recent runs that deep-link into a live/replayed stream (SSE replays from checkpoint). cm-* tests (incl. new cancel test) + clippy + frontend lint/typecheck green. Co-Authored-By: Claude Opus 4.8 <[email protected]> |
||
|
Omar SobhandClaude Opus 4.8
|
272669e1f5 |
Durable topology jobs (3+4/4): background worker + async run API
POST /api/topologies/run now ENQUEUES a durable job and returns 202
{run_id, status:queued} instead of executing inside the HTTP request — the
prerequisite for long-horizon runs (no client/proxy/LB timeout, survives
restarts).
topology_worker: a spawned loop that requeues stale running jobs, claims the
next queued one (CAS via FOR UPDATE SKIP LOCKED), drives it through
execute_resumable, and checkpoints RunProgress after every step; on crash the
stale sweep requeues it and the next claim resumes from the last checkpoint.
Wired into server startup beside the scheduler + resume sweeper.
GET /api/topology-runs/{id} now reports lifecycle status/kind/error/checkpoint
+ the result blob (kept the `comparison` field name for back-compat with the
compare UI; null until completed). list_runs includes status + kind.
Tests: durable lifecycle (enqueue→claim→checkpoint→complete) + stale-requeue
resume, both green; p0 endpoints (compare path) unchanged. 13 + 2 tests pass,
clippy clean.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|