clawmates/papers/dynamic-agentic-topologies.md

# Dynamic Agentic Topologies: Safe, Switchable Organizational Patterns for Multi-Agent Systems

**Status:** working draft. Framework, architecture, and the safety result are
implemented (`crates/cm-topology`, `crates/cm-orchestrator`); the empirical
tables below are **preliminary, offline (deterministic scripted provider)** and
are placeholders for runs against production models.

**Author:** Omar Sobh · **System:** ClawMates

---

## Abstract

The performance of a multi-agent system depends not only on its agents but on
their **organizational topology** — how work is decomposed, delegated, and
recombined. Prior work shows the best topology is task-dependent and can be
searched or evolved (ADAS, Darwin‑Gödel, SwarmAgentic). Those systems, however,
largely *simulate* execution and offer no safety contract. We present **Dynamic
Agentic Topologies**, a platform that (1) represents organizational topologies as
typed graphs, (2) executes a task under any topology on a **real, per‑tenant,
human‑in‑the‑loop‑safe runtime**, and (3) compares topologies empirically on a
quality/cost Pareto front. Our central design result: because a topology is only
an *orchestration over safe agent turns*, **switching topology cannot escalate an
agent's authority** — every sandbox‑leaving action remains gated by the same
approval/secret‑broker/audit layer regardless of structure. This makes topology a
*free variable* teams can tune for results without trading away safety.

## 1. Introduction

Builders of agentic systems face two coupled questions: *which agents?* and *how
are they organized?* The second — the topology — is under‑explored in production
because (a) there is no common vocabulary or runtime for it, and (b) letting
structure vary risks letting authority vary with it. We address both. A team
defines a system once and runs it under hierarchies, pipelines, swarms, meshes,
debates, and more; the platform measures which pattern yields better results for
the task, while a fixed safety contract holds across all of them.

Contributions:
1. **A topology model + classifier** (12 execution‑meaningful kinds) as a typed
   graph with a structural classifier (§3, §4).
2. **A topology runtime** that executes a task under any topology by sequencing
   safe agent turns, with a provable safety invariant (§5).
3. **A comparison harness** producing leaderboards and quality/cost Pareto fronts
   (§6), and a *workflow‑of‑topologies* composition (§6.3).
4. A reproducible benchmark and preliminary results (§7).

## 2. Related work

- **ADAS — Automated Design of Agentic Systems** (Hu, Lu, Clune): meta‑agent
  search discovers agent architectures in code.
- **Darwin‑Gödel Machine** (Zhang et al.): open‑ended, self‑improving coding
  agents with an archive of stepping stones.
- **SwarmAgentic** (Zhang et al.): language‑space particle‑swarm optimization over
  whole multi‑agent systems.
- **Autonomous Organizational Evolution** (Sobh): composes the three into
  `O(t+1)=DGM(SwarmAgentic(ADAS(O(t))))` for self‑modifying hierarchies.

These establish that topology matters and is searchable. Our work supplies the
missing **safe execution substrate** and an **empirical comparison** layer, and
contributes the safety result that makes topology switching deployable.

## 3. Topology model

A topology is a directed graph `TopologyGraph { kind, nodes, edges }`:
- **Node** = a role slot `{ id, role, level?, attrs }`, bound to a concrete agent
  (a "claw") at run time.
- **Edge** = a typed relationship `{ from, to, kind }`,
  `kind ∈ {delegates_to, reports_to, pipes_to, peers_with, routes_to, bids_to,
  reads_writes}`.

The v1 taxonomy (`TopologyKind`) covers twelve execution‑meaningful patterns:
hierarchical, flat, pipeline, swarm, mesh, hub‑spoke, ring, star/MoE, market,
blackboard, debate, holacratic. Each carries advisory **heuristics** (a default
role distribution + optimization weights). Topologies are imported from a loose
JSON/YAML spec via a normalizing adapter.

## 4. Topology classification

Given a graph, we compute structural metrics — density, degree spread, hub
dominance, average local clustering, connected components, diameter, and a
directed hierarchy score — and score each candidate kind. Clear shapes resolve
with high confidence (tree→hierarchical, line→pipeline, cycle→ring, star→hub‑
spoke, complete→mesh, empty→flat); "soft" kinds (market/blackboard/debate/
holacratic) are weakly inferred and taken from the declared kind. Classification
lets us import third‑party org structures and characterize graphs objectively for
the benchmark.

## 5. Safe topology execution (the core result)

A topology run is executed by **sequencing agent turns** according to the
pattern. The runtime is parameterized by a single capability — a `TurnExecutor`
that runs one agent turn — and **has no other capability**: it cannot send email,
move money, or touch files; it can only ask an agent to act. The real executor
wraps ClawMates' per‑tenant runtime, where every sandbox‑leaving action is
intercepted and held for human approval, executed via a single‑use secret broker,
and written to an append‑only audit journal (the §15 contract).

> **Invariant (authority is topology‑invariant).** For any topologies `T₁, T₂`
> over the same agents, the set of side effects an agent can cause without human
> approval is identical under `T₁` and `T₂`. *Sketch:* the only effectful path is
> a turn's gated action, which is mediated by the safety layer independently of
> the orchestration order; the orchestrator performs no effects itself. Hence a
> change of topology re‑routes *information and decisions* but never *authority*.

This is why topology can be a free variable: teams may search, switch, or evolve
structure to improve results with **no** change to the safety surface. The
runtime journals per‑step records and aggregate metrics (tokens, turns, gated
actions, approvals granted/blocked) for analysis.

Execution patterns (v1): five distinct executors cover all twelve kinds —
hierarchical (delegate down / synthesize up; also hub‑spoke, star/MoE, market),
pipeline (staged threading; also ring), swarm (parallel attempts + aggregate;
also flat, holacratic), mesh (two peer‑exchange rounds + aggregate; also
blackboard), and debate (propose → critique → revise → judge).

## 6. Comparison and composition

### 6.1 Harness
`compare(graphs, task, executor, scorer)` runs the **same task** across a set of
topologies on the **same executor**, scores each output, and returns per‑topology
results plus a **leaderboard** (by quality) and a **quality/cost Pareto front**
(maximize quality, minimize tokens), with best‑quality and best‑value picks.

### 6.2 Scoring
Quality is produced by a pluggable `Scorer`. A deterministic length proxy is used
offline; an **LLM judge** (`JudgeScorer`) rates outputs 0–100 against the task for
real runs.

### 6.3 Workflow of topologies
`run_workflow(stages, task, executor)` chains whole topology runs, threading each
stage's output into the next (e.g. *swarm* brainstorm → *hierarchical* execute →
*debate* review). Each stage is itself a safe run, so the §5 invariant holds at
every step.

## 7. Preliminary results (offline, illustrative)

Single task ("draft a go‑to‑market launch plan"), deterministic scripted
provider, length‑proxy scorer. Numbers are placeholders for production‑model runs.

| Topology     | Quality | Tokens | Turns | Pareto |
|--------------|--------:|-------:|------:|:------:|
| Hierarchical |    1.00 |    504 |     4 |        |
| Pipeline     |    0.92 |    252 |     3 |   ★    |
| Swarm        |    1.00 |    342 |     4 |   ★    |
| Mesh         |    1.00 |   1344 |     7 |        |
| Debate       |    1.00 |    680 |     4 |        |

Even in this toy setting the structure of the trade‑off is visible: the efficient
frontier is {Pipeline (best value), Swarm (best quality at low cost)}; mesh's two
peer rounds make it the most expensive; hierarchical and debate are dominated.

### 7.1 Live results (real model + real judge)

We then ran the same harness end‑to‑end against a production model
(**claude‑sonnet‑4‑6**) with an LLM‑judge scorer (0–100 → [0,1]), deployed live,
on the task *"Summarize the top 3 risks of a Q3 developer‑tool launch and
recommend a go/no‑go"* with roles {researcher, writer}:

| Topology  | Quality | Tokens | Turns | Pareto |
|-----------|--------:|-------:|------:|:------:|
| Debate    |   0.840 |   5064 |     4 |   ★    |
| Pipeline  |   0.790 |   1769 |     2 |   ★    |
| Swarm     |   0.730 |   3129 |     3 |        |

With a real model and judge the trade‑off sharpens and the ranking *changes* from
the offline proxy: **debate** now wins on quality (its adversarial rounds cost
~3× the tokens), **pipeline** is the best value (highest quality‑per‑token), and
**swarm is strictly dominated** — it spends more than pipeline for lower quality.
The efficient frontier is {debate, pipeline}. This confirms the central empirical
claim: topology is a real, measurable quality/cost lever, and the winner depends
on whether the user optimizes for quality (debate) or value (pipeline) — a choice
the platform surfaces rather than hard‑codes. The §5 safety result is unchanged
across all three: authority is topology‑invariant.

### 7.2 Heterogeneous cross‑model topologies (per‑role model selection)

Because a node binds to an agent only at run time, the *model* behind each role is
itself a free variable — orthogonal to topology and to the §5 safety surface. The
platform exposes five subscription/API backends as interchangeable role‑agents on a
single runtime — **Claude** (Sonnet‑4.6), **Kimi** (K2‑coding), **GLM** (4.7/5.2,
Zhipu), **Gemini** (2.5‑flash), and **Groq** (Llama‑3.3‑70b) — each driven by alias
over the runtime gateway. A topology can therefore assign a *different vendor's
model to each role*; assignment is declared per node in the graph
(`node.attrs["agent"]`), so one run request specifies a full heterogeneous cast
with no server reconfiguration.

We ran a fixed 2‑stage pipeline (drafter → refiner) on *"Draft, then refine, a
one‑sentence mission statement for a nonprofit teaching coding to refugees"* under
three model assignments, scored by an independent Claude judge for clarity, impact,
and concision:

| Assignment (drafter → refiner) | Judge rank |
|--------------------------------|:----------:|
| GLM‑4.7 → Gemini‑2.5‑flash (heterogeneous) | **1st** |
| Claude → Claude (homogeneous) | 2nd |
| Kimi → Claude (heterogeneous) | 3rd |

The heterogeneous **GLM→Gemini** cast won — the GLM draft plus Gemini's terse
refinement produced the most concise, on‑brief statement, beating the homogeneous
all‑Claude baseline. This is an existence proof of the platform's distinctive
question — *which model wins which role in which topology* — and confirms model
choice is a per‑role lever the platform can sweep without touching topology or
safety. (Costs are billed to each vendor's subscription, not per‑token; the run
journal records per‑step tokens where the backend surfaces them — Claude/GLM/Gemini
do, the Kimi CLI's stream path does not.) A larger model×role×topology sweep is
future work (§8), bounded by the subscription plans' 5‑hour/weekly quotas and
concurrency caps, which favor sequential pipelines over wide swarm/mesh fan‑out.

### 7.3 Per‑role model leaderboard (single‑role, fixed task)

To isolate the model axis from topology, we ran a single‑role drafter on a fixed
brief — *"a single‑sentence nonprofit mission statement, under 25 words"* — once per
backend (one turn each; per‑node model via `attrs["agent"]`), then ranked the
outputs with an independent Claude judge on clarity, impact, and concision:

| Rank | Model (backend) | Note |
|:----:|-----------------|------|
| 1 | **GLM‑4.7** (Zhipu, Sonnet‑class) | tight, vivid, zero wasted words |
| 2 | Gemini‑2.5‑flash (Google) | clear; "foster" slightly dilutes |
| 3 | Claude Sonnet‑4.6 (Anthropic) | strong impact, one clause too many |
| 4 | Kimi K2 (Moonshot) | clear, ending a touch redundant |
| 5 | GLM‑5.2 (Zhipu, Opus‑class) | jargon‑heavy, longest — *over‑wrote* the 25‑word brief |

A notable inversion: the **flagship GLM‑5.2 ranked last** on this *concision‑weighted*
task — its longer, richer output is an asset for complex reasoning but a liability
when the rubric rewards brevity. This is the core lesson the platform is built to
surface: there is no globally "best" model — the winner is role‑, task‑, and
rubric‑dependent, so model choice belongs to the same tunable layer as topology,
measured rather than assumed.

### 7.4 Topology × model grid (the interaction effect)

The platform's defining question is not "best topology" *or* "best model" in
isolation but their **interaction**. We crossed two topologies (**pipeline**, 3
stages; **debate**, propose→critique→revise→judge) with four homogeneous/mixed
model casts (**Claude**, **GLM‑4.7**, **Gemini‑2.5‑flash**, and a heterogeneous
**GLM→Kimi→Claude**) on a fixed decision task — *"Rust or Go for a 3‑person startup
backend?"* — and ranked all eight final outputs 1–8 with an independent judge on
reasoning quality, clarity, and concision:

| Topology ＼ cast | Claude | GLM‑4.7 | Gemini | Heterogeneous |
|------------------|:------:|:-------:|:------:|:-------------:|
| **Pipeline**     |   4    |  **2**  |   6    |       3       |
| **Debate**       | **8**  |  **1**  |   7    |       5       |

Three results stand out. First, **all eight cells converged on the same decision**
("Go") — so on a task with a clear prior, topology and model move *justification
quality*, not the answer. Second, and central: **the topology × model interaction is
real and non‑monotone.** Debate *amplified* the single strongest model (GLM‑4.7:
pipeline 2nd → debate 1st, its extra adversarial round adding a hiring‑cost nuance)
but *degraded* every other cast (Claude: 4th → **dead last**, its judge wasting words
on an "upstream agents agree" meta‑citation; Gemini: 6th → 7th; heterogeneous: 3rd →
5th). **Debate is not a universal upgrade — it pays off only with a model strong
enough to exploit the extra rounds, and is actively harmful otherwise.** Third,
**model strength is task‑dependent, not absolute**: Gemini‑2.5‑flash placed *2nd* on
the §7.3 concision brief but *6th–7th* here — the judge faulted it for a "generic
list" and "the same idea repeated," i.e. concise‑but‑shallow, an asset for brevity
and a liability for nuanced reasoning. The same model is strong or weak depending on
the role's rubric. Together these confirm the result the platform is built to produce
and that neither a topology‑only nor a model‑only study can see: the optimal
*configuration* is a joint choice over {topology × per‑role model × task}, discovered
empirically, with the §5 safety invariant holding across every cell. The sweep ran
sequentially under the subscription plans' quota/concurrency caps, the practical
envelope for this class of experiment.

## 8. Limitations and future work

- Results are offline; the production path (real models + tool‑using turns with
  live §15 approvals, journaled to `run_events`) is the next integration.
- Several kinds share an executor; richer per‑kind semantics and the remaining
  governance/novel forms are future work.
- **Evolution:** with the comparison harness as a fitness function, an
  ADAS/Darwin‑Gödel/quality‑diversity loop can propose and switch topologies —
  the bridge to *Autonomous Organizational Evolution* on a safe substrate.
- Product surfaces (a visual topology builder and Pareto explorer) will let
  non‑experts run these comparisons.

## 9. Reproduction

```bash
cargo test -p cm-topology
cargo test -p cm-orchestrator --features provider
cargo run -p cm-orchestrator --example topology_bench --features provider
# real models: set ANTHROPIC_API_KEY before the example.
```

Code: `crates/cm-topology` (model/classifier/heuristics), `crates/cm-orchestrator`
(runtime/harness/workflow/judge), `docs/topology-platform.md` (architecture).