feat(api): POST /api/topologies/compare (provider-backed)
ci / gates (push) Has been cancelled
ci / sandbox-k8s (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / e2e (push) Has been cancelled

Run a task across topologies via the API and return a leaderboard + quality/cost
Pareto front. The handler builds a ProviderExecutor + JudgeScorer over the
runtime's configured provider/model, then calls cm_orchestrator::compare.

- cm-runtime: expose provider()/model()/max_tokens() accessors on Runtime.
- cm-api: depend on cm-orchestrator (provider feature); add the compare route.
- Integration test runs a 2-topology comparison through the real server
  (scripted provider) → 200 with results + leaderboard. 4 topology tests green;
  offline build + clippy clean.

The topology endpoints (catalog/classify/build/compare) ship to gw-04 with the
upcoming ReactFlow UI in one server+frontend redeploy.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Omar Sobh
2026-06-15 22:28:39 -07:00
co-authored by Claude Opus 4.8
parent fa7e5dec5f
commit a35c82d860
6 changed files with 82 additions and 1 deletions
+29 -1
View File
@@ -3,11 +3,13 @@
//! back the topology builder UI. Running/comparing topologies is a later,
//! provider-backed endpoint.
use axum::extract::State;
use axum::Json;
use cm_orchestrator::{compare, Comparison, JudgeScorer, ProviderExecutor};
use cm_topology::{build, classify, heuristics, Classification, TopologyGraph, TopologyKind};
use serde::{Deserialize, Serialize};
use crate::{ApiError, Authed};
use crate::{ApiError, AppState, Authed};
/// A role and its suggested share of the team.
#[derive(Serialize)]
@@ -70,3 +72,29 @@ pub async fn build_graph(
let graph = build(req.kind, &roles).map_err(|_| ApiError::BadRequest)?;
Ok(Json(graph))
}
/// Request body for running a multi-topology comparison.
#[derive(Deserialize)]
pub struct CompareRequest {
pub task: String,
pub graphs: Vec<TopologyGraph>,
}
/// `POST /api/topologies/compare` — run a task across the given topologies and
/// return a leaderboard + quality/cost Pareto front. Uses the configured
/// provider for both execution (tool-free turns) and the LLM judge.
pub async fn compare_topologies(
State(state): State<AppState>,
_auth: Authed,
Json(req): Json<CompareRequest>,
) -> Result<Json<Comparison>, ApiError> {
let provider = state.runtime.provider();
let model = state.runtime.model().to_string();
let executor =
ProviderExecutor::new(provider.clone(), model.clone(), state.runtime.max_tokens());
let scorer = JudgeScorer::new(provider, model, 16);
let cmp = compare(&req.graphs, &req.task, &executor, &scorer)
.await
.map_err(|_| ApiError::Internal)?;
Ok(Json(cmp))
}