Commit Graph
19 Commits
Author SHA1 Message Date
Omar Sobh f1f3de4db0 missions: cargo fmt for run-output endpoint
ci / gates (push) Successful in 6s
ci / frontend (push) Successful in 27s
ci / rust (push) Successful in 4m30s
ci / e2e (push) Skipped
ci / publish (push) Successful in 4m3s
2026-07-21 19:14:35 -07:00
Omar Sobh e2956cdfed missions: surface run output on terminal phase runs
ci / gates (push) Successful in 7s
ci / rust (push) Failing after 10s
ci / frontend (push) Successful in 25s
ci / e2e (push) Skipped
ci / publish (push) Skipped
Adds GET /api/topology-runs/{id}/output — trimmed view of the
runs checkpoint (totals + per-turn output previews, capped at
12 turns × 6kB each). The full checkpoint blob can be hundreds
of KB so it was never viable to send through mission polling.

Phase card run rows now expose a "show output" toggle for any
terminal run (completed/failed/cancelled), rendering turns,
tokens, records count, and per-turn agent text. Running rows
still get the live activity stream from the prior slice.

Diagnostic value: on a mission that "completed" without visible
work, this immediately shows whether the agents produced real
output (workspace missing / instructions vague / etc.) or
whether nothing ran at all.
2026-07-21 15:26:26 -07:00
Omar Sobh fdb8cfeecc slice 9 cleanup: drop legacy research/loops backend + tables
Retires the legacy research/loops backend after the missions arc
(slices 1-9) fully replaced it. Frontend cutover was 4663348; this
commit finishes the job on the backend + database.

Migration:
  - 0053_drop_legacy_research_loops.sql — drops the 8 legacy tables
    (research_topics, research_topic_agents, research_outcomes,
    research_publish_approvals, loops, loop_agents, loop_orgs,
    loop_teams) and the 3 topology_runs FK columns
    (research_topic_id, loop_id, iteration). parent_run_id stays;
    recursive_exec still uses it.

Files deleted (11):
  - crates/cm-api/src/routes/{research,loops,research_setup,
    research_pipeline,wizard_repo,probe}.rs
  - crates/cm-api/src/research_container.rs
  - crates/cm-db/src/repo/{research_topics,research_outcomes,
    research_publish_approvals,loops}.rs
  - crates/cm-runtime/src/loops.rs
  - crates/cm-api/tests/research_publish_role.rs

Files edited:
  - crates/cm-api/src/lib.rs — dropped 20 legacy route registrations
    (all /api/research/* + /api/loops/* + /webhooks/loops + probe)
    and module decls
  - crates/cm-api/src/topology_worker.rs — deleted legacy dispatch
    (freeze_research_outcome, advance_loop_after_completion,
    continue_initial_burst, maybe_transition_research_topic,
    parse_reorder_rationale, per-topic/loop gateway resolver).
    reap_stuck_runs now keys on mission_id (not topic_id).
    Executor path unconditionally uses ZeroClawDriveExecutor::from_env
    — mission_orchestrator provisions each claw as an agent inside
    the shared runtime via RuntimeProvisioner, so per-team gateway
    resolution is no longer applicable.
  - crates/cm-api/src/routes/topology.rs — deleted container-log SSE
    endpoint (research/loop-specific), dropped loop_id filter and
    iteration field from ListRunsQuery/RunSummary
  - crates/cm-api/src/routes/world.rs — removed
    active_research_topics/active_loops/preseed_repo_paths;
    World SSE no longer emits repo:{topic}/loop:{id} landmark orbs
    (follow-up task #21 tracks adding mission:{id} equivalents)
  - crates/cm-api/src/runtime_provision.rs — removed now-unused
    mint_workspace_service_token
  - crates/cm-db/src/repo/topology_runs.rs — removed 9 legacy
    helpers (research_topic_id lookup, loop_id_for_run,
    iteration_for_run, active_runs_for_research_topic, etc.)
  - crates/cm-db/src/repo/teams.rs — removed 4 dead helpers
    (team_for_loop, team_for_research_topic + setters)
  - crates/cm-api/tests/topology_jobs.rs — removed loop/topic
    tests, dropped enqueue_run_with_topic helper
  - crates/bins/clawmates-server/src/main.rs — removed
    spawn_loop_scheduler call
  - crates/cm-api/src/routes/mod.rs, crates/cm-db/src/repo/mod.rs,
    crates/cm-runtime/src/lib.rs — module decls stripped

sqlx cache: regenerated against post-migration schema
  (71 files changed, ~+70 / -8896 net)

Test/build: SQLX_OFFLINE=true cargo check --workspace clean;
cargo test --workspace --no-run clean.

Follow-up (task #21): World view lost the in-flight-work landmarks
when repo:{topic} / loop:{id} orbs disappeared. Add mission:{id}
orbs as the missions-era replacement.
2026-07-19 18:37:24 -07:00
Omar SobhandClaude Opus 4.7 5bbab870fc topology/log-sse: resolve team container name for coding-loop runs
ci / gates (push) Successful in 4s
ci / frontend (push) Successful in 26s
ci / rust (push) Successful in 3m59s
ci / e2e (push) Skipped
ci / publish (push) Successful in 2m16s
The log SSE endpoint was always resolving the container name from a
research topic id (research-<topic_id>-team). For paired coding loops
that own a team_id, the actual container the topology worker spawns is
team-<team_id>-container (spawn_team) — not the research topic's one.
That mismatch produced a stream of 404s from Docker:

  Docker responded with status code 404: No such container:
  research-<topic_id>-team

Mirror topology_worker::try_team_gateway_url's precedence:
  1. run's loop has team_id → team-<team_id>-container
  2. run has research_topic_id → research-<topic_id>-team
  3. loop.source_research_topic_id (legacy paired flow) → same as 2
  4. else → clear error, no more 404 spam

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-07-18 19:51:02 -07:00
Omar Sobh 2ba40b03b7 loops: per-iteration live logs (Steps + Container tabs), collapse graph JSON
ci / gates (push) Successful in 6s
ci / frontend (push) Successful in 38s
ci / rust (push) Successful in 3m15s
ci / e2e (push) Skipped
ci / publish (push) Successful in 2m35s
The Loops canvas' IterationsTimeline was a flat status-pill list —
no way to see WHAT an iteration was actually doing. Meanwhile the
Research canvas had full LiveRunLogs with Steps + Container tabs
against the same underlying topology_run SSE endpoints. This
factors LiveRunLogs so both canvases share it.

Frontend
- LiveRunLogs gains a directRunId?: string prop. In this mode it
  skips the topic-scoped active-runs + pipeline-state polls and
  pins activeRun to the given id. topicId stays optional (topic
  mode unchanged from ResearchCanvas' perspective).
- LoopsCanvas IterationsTimeline: each row is now a click-to-
  expand card. On expand, renders <LiveRunLogs directRunId={...}/>
  right below the header — Steps + Container tabs, full SSE tail,
  same 320px terminal.
- Auto-opens the newest running/queued iteration so a click on
  'Run now' immediately exposes the live pane.
- New CollapsibleSection helper wraps the graph JSON block so it
  starts closed. Reference material stays one click away without
  cluttering the canvas.

Backend
- run_container_log_sse now resolves the tail target via
  loop.source_research_topic_id when the run itself has no
  research_topic_id. Paired coding loops (kind=exec with a
  source_research_topic_id) reuse the paired topic's team
  container, so we tail its docker logs. Pure loop runs still
  error with a clearer message.
2026-07-16 17:30:07 -07:00
Omar Sobh ae113dd319 live-run-logs: add Container tab that tails filtered daemon logs
ci / rust (push) Successful in 2m49s
ci / e2e (push) Skipped
ci / gates (push) Successful in 5s
ci / frontend (push) Successful in 36s
ci / publish (push) Successful in 3m58s
Steps summaries only fire AFTER each topology step completes — so a
stalled first turn was completely dark. Add a second tab that
streams the team runtime container's daemon log live via a new SSE
endpoint.

Backend
- GET /api/topology-runs/:id/container-log — workspace-scoped SSE
  around bollard's docker.logs(follow=true, tail=200). Buffers on
  newline so partial mux chunks don't truncate a log line.
- compact_container_log: parse a zeroclaw daemon line
  ('[actor] ... zc_action=X zc_outcome=Y ... msg') into
  '[actor] action (outcome) · msg'. Framing-only continuations are
  dropped; non-zc lines (bash echoes, backtraces) pass through as-is
  so nothing interesting is lost. ANSI escapes stripped.
- Everything funnels through one async_stream! so early exits
  (workspace check / docker connect / no bound topic) yield an
  'error' event and return without breaking Sse::new's single stream
  type.

Frontend
- LiveRunLogs gets a sub-tabs strip: Steps · Container.
- New useContainerLog(runId, active) hook — gated by tab so we don't
  hold two open SSE streams when the operator isn't looking.
- Same terminal widget renders each container line with a level
  color (info/done grey, line default, error red). Sub-tab pill
  shows count + status live.
2026-07-15 18:42:03 -07:00
Omar Sobh 6d1dda6197 loops: iteration timeline — GET /api/topology-runs?loop_id=X
Extend the topology-runs list route with an optional loop_id filter that
returns iterations for a single loop, newest-iteration-first. Adds the
iteration and finished_at columns to the summary (skip-null on the JSON
so compares stay compact). Backed by list_by_loop in the repo, which uses
the existing topology_runs_loop_idx partial index.

LoopsCanvas fetches the runs in parallel with the loop detail and renders
an iteration timeline card (iteration #, status pill, start time, duration,
run id prefix) between the graph section and the actions row.
2026-07-06 12:28:34 -07:00
Omar SobhandClaude Opus 4.8 3554a3aaf2 CI: remove k8s stages, fix the Docker-level pipeline green
ci / frontend (push) Successful in 23s
ci / rust (push) Failing after 27s
ci / e2e (push) Has been skipped
ci / gates (push) Successful in 5s
Survey + fixes so the pipeline passes at the Docker level (no k8s).

- Remove k8s: drop the `sandbox-k8s` job (kind/Calico/--features k8s-tests) and the
  "Helm chart lints" gate step. release.yml was already k8s-clean.
- Rust job:
  - `cargo fmt --all` — fix pre-existing formatting drift (fmt --check was failing).
  - clippy -D warnings: fix 3 lib warnings (cm-brain sort_by_key→Reverse, cm-api
    fleet.rs doc list indentation, node_rules map_or→is_none_or).
  - Regenerate the .sqlx offline cache (was missing the cm-runtime run_loop test
    query → offline compile failed). DB-backed tests use testcontainers at runtime.
  - Set SQLX_OFFLINE=true on the rust + e2e jobs so query! macros compile against
    the committed cache deterministically (no DB needed at compile time).
- Frontend job:
  - Fix the 1 ESLint error (useAgentTelemetry: no setState-synchronously-in-effect;
    tag the slice with agentId + derive null on mismatch).
  - Fix 2 stale panel-params tests (`terminal` is a valid app id now; assert the
    current APP_IDS + use a genuinely-unknown id for the reject case).

Verified locally: fmt clean, clippy --all-targets -D warnings clean (offline),
frontend lint 0 errors, tsc clean, 86/86 frontend tests pass, build OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-26 18:15:31 -07:00
Omar SobhandClaude Opus 4.8 34f744734b Large World graph, agent platform, brain stack & dashboard rebuild
Frontend
- Large World: collapse org/company/team tiers into one expandable React Flow
  hierarchy (WorldFlow) with per-click expand, persisted node positions, a
  compact tree sidebar, wrench multi-select delete across levels, and a sized
  right slide-out (phone/tablet/full) showing an agent summary + drill button.
- Agent page: GitHub-style animated contribution grid (VitalsCard), collapsible
  System Prompt + Personality cards, restructured anatomy cards, bigger avatar
  with name/title header row, Markdown/JSON-aware rendering, brain registry +
  history, avatar generate/upload.
- User-icon menu (Infrastructure/Brains/Tools/Profile/Credits) + ToolPanel;
  Master Planner deploy wizard (Specialists/Swarm/Scheduled/Triggered);
  Team Runs view; reap-progress modal; dashboard is the single live interface.

Backend
- cm-brain crate (.brain as the agent definition) + brain apply/history.
- Hard-purge reap (FK-ordered) + sandbox release + SSE batch-delete.
- Swarm self-verifying loop, mode-aware planner, web.search tool, webhooks
  (migration 0013), org/company/team delete endpoints, scheduler sweeps.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-22 23:21:54 -07:00
Omar SobhandClaude Opus 4.8 3eca4ed70c Recursive deploy ladder: Company + Org tiers, mesh mark, two-tier rail
Completes the scale ladder (single → team → company → org). Every tier is a
topology whose nodes are the tier below; running a parent recursively runs each
child's sub-topology down to the leaf claws.

Backend:
- migration 0011: companies/company_teams, orgs/org_companies, topology_runs.tier
- cm-db repos for companies + orgs (mirror teams)
- TurnRequest.attrs (forwarded from node.attrs) for child-id binding
- SubTopologyExecutor (recursive_exec.rs): a parent "turn" runs the child's
  sub-topology; durability via parent updated_at keepalive + cancel propagation
  + depth cap; boxed future breaks the org→company recursion
- topology_worker selects executor by job.tier
- routes: /api/companies, /api/orgs (create/list/get/run) + unified
  /api/structure/{level}/{id} for the zoom canvas

Frontend:
- MeshMark: node-mesh brand glyph (replaces the claw PNG), tier variants
- TopologyGraphView: optional onNodeClick/nodeMeta + dark-token theming
- StructureCanvas + Breadcrumb: one recursive zoom view for every tier
  (drill down on node click, breadcrumb up); TeamRunPanel extracted + shared
- two-tier Discord-style rail: StructureRail (mesh mark + org/company/team
  glyphs + tools popover + deploy + user) | RosterColumn (selected group's
  children, or your claws); SecondaryNav for cross-cutting tools
- ComposeWizard (company/org) wired into DeployWizard; /companies + /orgs pages

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-18 14:25:06 -07:00
Omar SobhandClaude Opus 4.8 6e87433c66 Close gaps: GLM-judge (anthropic registry) + run cancel + deep-link
ci / frontend (push) Has been cancelled
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / e2e (push) Has been cancelled
#2 GLM judge (closes #114): registry NamedProvider gains a `format` field;
build_provider_registry builds an AnthropicProvider for format="anthropic".
GLM's coding/OpenAI endpoint is ToS-throttled for raw SDK, but its Anthropic
endpoint (api.z.ai/api/anthropic) accepts raw API calls (verified x-api-key
-> glm-4.7), so CLAWMATES_JUDGE_MODEL=glm:glm-4.7 routes the door governor /
topology judge through GLM with no runtime-routing. (Kimi-as-judge still needs
a Platform key — coding key is agent-only.)

#3 run-control: POST /api/topology-runs/{id}/cancel (workspace-scoped,
queued/running only); the worker honors it at the step boundary (checks
current_status in the checkpoint callback) and won't clobber a cancel with
`failed`. Frontend Run tab gains a Cancel button and clickable recent runs
that deep-link into a live/replayed stream (SSE replays from checkpoint).

cm-* tests (incl. new cancel test) + clippy + frontend lint/typecheck green.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-18 03:07:41 -07:00
Omar SobhandClaude Opus 4.8 f845dfb15f Topology runs: SSE live-progress (replace polling)
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
Backend: GET /api/topology-runs/{id}/events streams Server-Sent Events by
tailing the durable per-step checkpoint the worker already writes — a `step`
event per newly-completed step (replayed on connect so reload/reconnect
re-attaches), then a terminal `done` event with the final output/error. Each
step carries its index as the SSE id, so the browser's Last-Event-ID resumes
without duplicates on reconnect. No new table, no worker change — reuses the
checkpoint; avoids run_events' agent_runs FK.

Frontend: the Run tab now opens an EventSource (through the same-origin proxy,
which adds the bearer) instead of polling — appending steps as they stream and
finalizing on `done`. One streaming connection, lower latency, auto-resume.

cm-api builds + clippy clean; frontend lint + typecheck + next build clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-17 22:32:39 -07:00
Omar SobhandClaude Opus 4.8 272669e1f5 Durable topology jobs (3+4/4): background worker + async run API
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
POST /api/topologies/run now ENQUEUES a durable job and returns 202
{run_id, status:queued} instead of executing inside the HTTP request — the
prerequisite for long-horizon runs (no client/proxy/LB timeout, survives
restarts).

topology_worker: a spawned loop that requeues stale running jobs, claims the
next queued one (CAS via FOR UPDATE SKIP LOCKED), drives it through
execute_resumable, and checkpoints RunProgress after every step; on crash the
stale sweep requeues it and the next claim resumes from the last checkpoint.
Wired into server startup beside the scheduler + resume sweeper.

GET /api/topology-runs/{id} now reports lifecycle status/kind/error/checkpoint
+ the result blob (kept the `comparison` field name for back-compat with the
compare UI; null until completed). list_runs includes status + kind.

Tests: durable lifecycle (enqueue→claim→checkpoint→complete) + stale-requeue
resume, both green; p0 endpoints (compare path) unchanged. 13 + 2 tests pass,
clippy clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-17 19:12:54 -07:00
Omar SobhandClaude Opus 4.8 2ec8832ef6 llm: named-provider registry — judges & topology nodes on GLM/Kimi
Adds a multi-provider registry so judges and topology execution can run on
providers beyond the default. cm-config gains [[llm.providers]] (name, base_url,
api_key_env) — all OpenAI-compatible (GLM, Kimi/Moonshot). The server builds an
Arc<dyn LlmProvider> per entry (OpenAiCompatProvider) keyed by name; a missing
key is skipped with a warning, not a boot failure. cm-runtime RuntimeConfig
carries a ProviderRegistry; Runtime::resolve_provider("<name>:<model>") selects a
registry provider (else the default). The door governor (Runtime::judge) and the
topology compare endpoint both resolve through it, so CLAWMATES_JUDGE_MODEL and
CLAWMATES_TOPOLOGY_EXEC_MODEL accept "glm:glm-4.6" / "kimi:kimi-k2".

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-17 04:17:14 -07:00
Omar SobhandClaude Opus 4.8 325038e806 judges use claude-opus-4-8; everything else stays on sonnet-4-6
Both LLM-as-judge sites — the door governor (Runtime::judge) and the topology
comparison scorer (JudgeScorer) — now use the judge model (default
claude-opus-4-8, override CLAWMATES_JUDGE_MODEL). Execution/reasoning turns keep
using the configured default model (claude-sonnet-4-6).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-17 03:36:43 -07:00
Omar SobhandClaude Opus 4.8 6d77a0acc1 Topology P-zc 1A: ZeroClawDriveExecutor — real role-agents over /ws/chat
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
cm-orchestrator owns the topology graph; each turn now drives a real ZeroClaw
role-agent in a container via the proven gateway drive recipe, instead of a
tool-free cm-llm call.

- crates/cm-api/src/topology_exec.rs: ZeroClawDriveExecutor impl TurnExecutor —
  pair (POST /pair + X-Pairing-Code, token cached) -> ws /ws/chat?agent=<alias>
  -> send {type:message,content} -> drain chunk/done/approval_request/error.
  approval_request is recorded as a BLOCKED GatedAction, never auto-approved (§15).
  Role->alias via ZEROCLAW_AGENT_MAP, fallback ZEROCLAW_DEFAULT_AGENT (scout).
- POST /api/topologies/run {task,graph} -> execute() -> RunRecord, persisted
  best-effort to the existing topology_runs table (no migration). compare stays
  tool-free. from_env() is read in-handler so cm-api still boots unset.
- deploy/clawmates-runtime: example config now declares a tool-free multi-agent
  role-cast; README documents the ZEROCLAW_* knobs + run endpoint.

tokio-tungstenite 0.26 (already in lock) + dev axum `ws` for the hermetic test.
3 lib tests green, clippy clean, SQLX_OFFLINE build clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-16 12:52:54 -07:00
Omar SobhandClaude Opus 4.8 7baf2082d0 feat(topology): persist comparison runs + history
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
Save each comparison and let users reload past ones.
- migration 0008: topology_runs (workspace-scoped; full comparison as JSONB).
- cm-db repo::topology_runs (insert / list_recent / get) + regenerated .sqlx.
- cm-api: compare persists best-effort (never loses the LLM result on a DB
  hiccup); GET /api/topology-runs (recent) + GET /api/topology-runs/{id}.
  Integration test asserts persist → list → get.
- frontend: "Recent comparisons" list on the Compare tab; click to reload a
  saved run. e2e p8 green (39 suite); offline build + clippy clean.

Server self-migrates at boot (cm_db::MIGRATOR), so 0008 applies on deploy.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-16 05:11:52 -07:00
Omar SobhandClaude Opus 4.8 a35c82d860 feat(api): POST /api/topologies/compare (provider-backed)
ci / gates (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / e2e (push) Has been cancelled
Run a task across topologies via the API and return a leaderboard + quality/cost
Pareto front. The handler builds a ProviderExecutor + JudgeScorer over the
runtime's configured provider/model, then calls cm_orchestrator::compare.

- cm-runtime: expose provider()/model()/max_tokens() accessors on Runtime.
- cm-api: depend on cm-orchestrator (provider feature); add the compare route.
- Integration test runs a 2-topology comparison through the real server
  (scripted provider) → 200 with results + leaderboard. 4 topology tests green;
  offline build + clippy clean.

The topology endpoints (catalog/classify/build/compare) ship to gw-04 with the
upcoming ReactFlow UI in one server+frontend redeploy.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-15 22:28:39 -07:00
Omar SobhandClaude Opus 4.8 fa7e5dec5f feat(api): topology endpoints (catalog / classify / build)
ci / gates (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / e2e (push) Has been cancelled
Stateless, auth-gated routes backing the topology builder UI:
- GET  /api/topologies          — catalog of the 12 kinds + descriptions + role mix
- POST /api/topologies/classify — infer a kind + metrics from a posted graph
- POST /api/topologies/build    — build a canonical graph from {kind, roles}

Add Serialize to cm-topology Classification/GraphMetrics; add ApiError::BadRequest
(400) for invalid build input; add cm-topology dep. 3 integration tests
(catalog/build/auth) green against Postgres; offline build + clippy clean.

No DB or provider yet — running/comparing topologies is a later provider-backed
endpoint. Server redeploy will batch with the ReactFlow UI that consumes these.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-15 22:04:26 -07:00