feat(api): POST /api/topologies/compare (provider-backed)
Run a task across topologies via the API and return a leaderboard + quality/cost Pareto front. The handler builds a ProviderExecutor + JudgeScorer over the runtime's configured provider/model, then calls cm_orchestrator::compare. - cm-runtime: expose provider()/model()/max_tokens() accessors on Runtime. - cm-api: depend on cm-orchestrator (provider feature); add the compare route. - Integration test runs a 2-topology comparison through the real server (scripted provider) → 200 with results + leaderboard. 4 topology tests green; offline build + clippy clean. The topology endpoints (catalog/classify/build/compare) ship to gw-04 with the upcoming ReactFlow UI in one server+frontend redeploy. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
fa7e5dec5f
commit
a35c82d860
@@ -3,11 +3,13 @@
|
||||
//! back the topology builder UI. Running/comparing topologies is a later,
|
||||
//! provider-backed endpoint.
|
||||
|
||||
use axum::extract::State;
|
||||
use axum::Json;
|
||||
use cm_orchestrator::{compare, Comparison, JudgeScorer, ProviderExecutor};
|
||||
use cm_topology::{build, classify, heuristics, Classification, TopologyGraph, TopologyKind};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::{ApiError, Authed};
|
||||
use crate::{ApiError, AppState, Authed};
|
||||
|
||||
/// A role and its suggested share of the team.
|
||||
#[derive(Serialize)]
|
||||
@@ -70,3 +72,29 @@ pub async fn build_graph(
|
||||
let graph = build(req.kind, &roles).map_err(|_| ApiError::BadRequest)?;
|
||||
Ok(Json(graph))
|
||||
}
|
||||
|
||||
/// Request body for running a multi-topology comparison.
|
||||
#[derive(Deserialize)]
|
||||
pub struct CompareRequest {
|
||||
pub task: String,
|
||||
pub graphs: Vec<TopologyGraph>,
|
||||
}
|
||||
|
||||
/// `POST /api/topologies/compare` — run a task across the given topologies and
|
||||
/// return a leaderboard + quality/cost Pareto front. Uses the configured
|
||||
/// provider for both execution (tool-free turns) and the LLM judge.
|
||||
pub async fn compare_topologies(
|
||||
State(state): State<AppState>,
|
||||
_auth: Authed,
|
||||
Json(req): Json<CompareRequest>,
|
||||
) -> Result<Json<Comparison>, ApiError> {
|
||||
let provider = state.runtime.provider();
|
||||
let model = state.runtime.model().to_string();
|
||||
let executor =
|
||||
ProviderExecutor::new(provider.clone(), model.clone(), state.runtime.max_tokens());
|
||||
let scorer = JudgeScorer::new(provider, model, 16);
|
||||
let cmp = compare(&req.graphs, &req.task, &executor, &scorer)
|
||||
.await
|
||||
.map_err(|_| ApiError::Internal)?;
|
||||
Ok(Json(cmp))
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user