paper §7.2: heterogeneous cross-model topology result (per-role model)
Adds the Phase C result: five backends (Claude/Kimi/GLM/Gemini/Groq) as interchangeable role-agents, per-node model selection via node.attrs["agent"], and a 3-config drafter->refiner sweep judged for quality. Heterogeneous GLM->Gemini won over the all-Claude baseline — an existence proof of "which model wins which role in which topology." Notes quota-bounded sweep scope as future work. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
99e2fd3ec3
commit
7ae8f13db2
@@ -172,6 +172,40 @@ on whether the user optimizes for quality (debate) or value (pipeline) — a cho
|
|||||||
the platform surfaces rather than hard‑codes. The §5 safety result is unchanged
|
the platform surfaces rather than hard‑codes. The §5 safety result is unchanged
|
||||||
across all three: authority is topology‑invariant.
|
across all three: authority is topology‑invariant.
|
||||||
|
|
||||||
|
### 7.2 Heterogeneous cross‑model topologies (per‑role model selection)
|
||||||
|
|
||||||
|
Because a node binds to an agent only at run time, the *model* behind each role is
|
||||||
|
itself a free variable — orthogonal to topology and to the §5 safety surface. The
|
||||||
|
platform exposes five subscription/API backends as interchangeable role‑agents on a
|
||||||
|
single runtime — **Claude** (Sonnet‑4.6), **Kimi** (K2‑coding), **GLM** (4.7/5.2,
|
||||||
|
Zhipu), **Gemini** (2.5‑flash), and **Groq** (Llama‑3.3‑70b) — each driven by alias
|
||||||
|
over the runtime gateway. A topology can therefore assign a *different vendor's
|
||||||
|
model to each role*; assignment is declared per node in the graph
|
||||||
|
(`node.attrs["agent"]`), so one run request specifies a full heterogeneous cast
|
||||||
|
with no server reconfiguration.
|
||||||
|
|
||||||
|
We ran a fixed 2‑stage pipeline (drafter → refiner) on *"Draft, then refine, a
|
||||||
|
one‑sentence mission statement for a nonprofit teaching coding to refugees"* under
|
||||||
|
three model assignments, scored by an independent Claude judge for clarity, impact,
|
||||||
|
and concision:
|
||||||
|
|
||||||
|
| Assignment (drafter → refiner) | Judge rank |
|
||||||
|
|--------------------------------|:----------:|
|
||||||
|
| GLM‑4.7 → Gemini‑2.5‑flash (heterogeneous) | **1st** |
|
||||||
|
| Claude → Claude (homogeneous) | 2nd |
|
||||||
|
| Kimi → Claude (heterogeneous) | 3rd |
|
||||||
|
|
||||||
|
The heterogeneous **GLM→Gemini** cast won — the GLM draft plus Gemini's terse
|
||||||
|
refinement produced the most concise, on‑brief statement, beating the homogeneous
|
||||||
|
all‑Claude baseline. This is an existence proof of the platform's distinctive
|
||||||
|
question — *which model wins which role in which topology* — and confirms model
|
||||||
|
choice is a per‑role lever the platform can sweep without touching topology or
|
||||||
|
safety. (Costs are billed to each vendor's subscription, not per‑token; the run
|
||||||
|
journal records per‑step tokens where the backend surfaces them — Claude/GLM/Gemini
|
||||||
|
do, the Kimi CLI's stream path does not.) A larger model×role×topology sweep is
|
||||||
|
future work (§8), bounded by the subscription plans' 5‑hour/weekly quotas and
|
||||||
|
concurrency caps, which favor sequential pipelines over wide swarm/mesh fan‑out.
|
||||||
|
|
||||||
## 8. Limitations and future work
|
## 8. Limitations and future work
|
||||||
|
|
||||||
- Results are offline; the production path (real models + tool‑using turns with
|
- Results are offline; the production path (real models + tool‑using turns with
|
||||||
|
|||||||
Reference in New Issue
Block a user