From 2621e97ad29f5233d5c0b7084f20276626a1f0b9 Mon Sep 17 00:00:00 2001 From: Omar Sobh Date: Mon, 15 Jun 2026 21:10:19 -0700 Subject: [PATCH] docs(paper): draft "Dynamic Agentic Topologies" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Working draft consolidating the platform: thesis, related work (ADAS/Darwin- Gödel/SwarmAgentic + Autonomous Organizational Evolution), the topology model + classifier, the safe-execution result (authority is topology-invariant — the §15 contract holds across any topology because the orchestrator only sequences safe turns), the comparison harness + Pareto + workflow-of-topologies, preliminary offline results, limitations/future (real-model runs, evolution), and a reproduction section. Empirical tables are offline/illustrative placeholders. Co-Authored-By: Claude Opus 4.8 --- papers/dynamic-agentic-topologies.md | 176 +++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 papers/dynamic-agentic-topologies.md diff --git a/papers/dynamic-agentic-topologies.md b/papers/dynamic-agentic-topologies.md new file mode 100644 index 0000000..ede0287 --- /dev/null +++ b/papers/dynamic-agentic-topologies.md @@ -0,0 +1,176 @@ +# Dynamic Agentic Topologies: Safe, Switchable Organizational Patterns for Multi-Agent Systems + +**Status:** working draft. Framework, architecture, and the safety result are +implemented (`crates/cm-topology`, `crates/cm-orchestrator`); the empirical +tables below are **preliminary, offline (deterministic scripted provider)** and +are placeholders for runs against production models. + +**Author:** Omar Sobh · **System:** ClawMates + +--- + +## Abstract + +The performance of a multi-agent system depends not only on its agents but on +their **organizational topology** — how work is decomposed, delegated, and +recombined. Prior work shows the best topology is task-dependent and can be +searched or evolved (ADAS, Darwin‑Gödel, SwarmAgentic). Those systems, however, +largely *simulate* execution and offer no safety contract. We present **Dynamic +Agentic Topologies**, a platform that (1) represents organizational topologies as +typed graphs, (2) executes a task under any topology on a **real, per‑tenant, +human‑in‑the‑loop‑safe runtime**, and (3) compares topologies empirically on a +quality/cost Pareto front. Our central design result: because a topology is only +an *orchestration over safe agent turns*, **switching topology cannot escalate an +agent's authority** — every sandbox‑leaving action remains gated by the same +approval/secret‑broker/audit layer regardless of structure. This makes topology a +*free variable* teams can tune for results without trading away safety. + +## 1. Introduction + +Builders of agentic systems face two coupled questions: *which agents?* and *how +are they organized?* The second — the topology — is under‑explored in production +because (a) there is no common vocabulary or runtime for it, and (b) letting +structure vary risks letting authority vary with it. We address both. A team +defines a system once and runs it under hierarchies, pipelines, swarms, meshes, +debates, and more; the platform measures which pattern yields better results for +the task, while a fixed safety contract holds across all of them. + +Contributions: +1. **A topology model + classifier** (12 execution‑meaningful kinds) as a typed + graph with a structural classifier (§3, §4). +2. **A topology runtime** that executes a task under any topology by sequencing + safe agent turns, with a provable safety invariant (§5). +3. **A comparison harness** producing leaderboards and quality/cost Pareto fronts + (§6), and a *workflow‑of‑topologies* composition (§6.3). +4. A reproducible benchmark and preliminary results (§7). + +## 2. Related work + +- **ADAS — Automated Design of Agentic Systems** (Hu, Lu, Clune): meta‑agent + search discovers agent architectures in code. +- **Darwin‑Gödel Machine** (Zhang et al.): open‑ended, self‑improving coding + agents with an archive of stepping stones. +- **SwarmAgentic** (Zhang et al.): language‑space particle‑swarm optimization over + whole multi‑agent systems. +- **Autonomous Organizational Evolution** (Sobh): composes the three into + `O(t+1)=DGM(SwarmAgentic(ADAS(O(t))))` for self‑modifying hierarchies. + +These establish that topology matters and is searchable. Our work supplies the +missing **safe execution substrate** and an **empirical comparison** layer, and +contributes the safety result that makes topology switching deployable. + +## 3. Topology model + +A topology is a directed graph `TopologyGraph { kind, nodes, edges }`: +- **Node** = a role slot `{ id, role, level?, attrs }`, bound to a concrete agent + (a "claw") at run time. +- **Edge** = a typed relationship `{ from, to, kind }`, + `kind ∈ {delegates_to, reports_to, pipes_to, peers_with, routes_to, bids_to, + reads_writes}`. + +The v1 taxonomy (`TopologyKind`) covers twelve execution‑meaningful patterns: +hierarchical, flat, pipeline, swarm, mesh, hub‑spoke, ring, star/MoE, market, +blackboard, debate, holacratic. Each carries advisory **heuristics** (a default +role distribution + optimization weights). Topologies are imported from a loose +JSON/YAML spec via a normalizing adapter. + +## 4. Topology classification + +Given a graph, we compute structural metrics — density, degree spread, hub +dominance, average local clustering, connected components, diameter, and a +directed hierarchy score — and score each candidate kind. Clear shapes resolve +with high confidence (tree→hierarchical, line→pipeline, cycle→ring, star→hub‑ +spoke, complete→mesh, empty→flat); "soft" kinds (market/blackboard/debate/ +holacratic) are weakly inferred and taken from the declared kind. Classification +lets us import third‑party org structures and characterize graphs objectively for +the benchmark. + +## 5. Safe topology execution (the core result) + +A topology run is executed by **sequencing agent turns** according to the +pattern. The runtime is parameterized by a single capability — a `TurnExecutor` +that runs one agent turn — and **has no other capability**: it cannot send email, +move money, or touch files; it can only ask an agent to act. The real executor +wraps ClawMates' per‑tenant runtime, where every sandbox‑leaving action is +intercepted and held for human approval, executed via a single‑use secret broker, +and written to an append‑only audit journal (the §15 contract). + +> **Invariant (authority is topology‑invariant).** For any topologies `T₁, T₂` +> over the same agents, the set of side effects an agent can cause without human +> approval is identical under `T₁` and `T₂`. *Sketch:* the only effectful path is +> a turn's gated action, which is mediated by the safety layer independently of +> the orchestration order; the orchestrator performs no effects itself. Hence a +> change of topology re‑routes *information and decisions* but never *authority*. + +This is why topology can be a free variable: teams may search, switch, or evolve +structure to improve results with **no** change to the safety surface. The +runtime journals per‑step records and aggregate metrics (tokens, turns, gated +actions, approvals granted/blocked) for analysis. + +Execution patterns (v1): five distinct executors cover all twelve kinds — +hierarchical (delegate down / synthesize up; also hub‑spoke, star/MoE, market), +pipeline (staged threading; also ring), swarm (parallel attempts + aggregate; +also flat, holacratic), mesh (two peer‑exchange rounds + aggregate; also +blackboard), and debate (propose → critique → revise → judge). + +## 6. Comparison and composition + +### 6.1 Harness +`compare(graphs, task, executor, scorer)` runs the **same task** across a set of +topologies on the **same executor**, scores each output, and returns per‑topology +results plus a **leaderboard** (by quality) and a **quality/cost Pareto front** +(maximize quality, minimize tokens), with best‑quality and best‑value picks. + +### 6.2 Scoring +Quality is produced by a pluggable `Scorer`. A deterministic length proxy is used +offline; an **LLM judge** (`JudgeScorer`) rates outputs 0–100 against the task for +real runs. + +### 6.3 Workflow of topologies +`run_workflow(stages, task, executor)` chains whole topology runs, threading each +stage's output into the next (e.g. *swarm* brainstorm → *hierarchical* execute → +*debate* review). Each stage is itself a safe run, so the §5 invariant holds at +every step. + +## 7. Preliminary results (offline, illustrative) + +Single task ("draft a go‑to‑market launch plan"), deterministic scripted +provider, length‑proxy scorer. Numbers are placeholders for production‑model runs. + +| Topology | Quality | Tokens | Turns | Pareto | +|--------------|--------:|-------:|------:|:------:| +| Hierarchical | 1.00 | 504 | 4 | | +| Pipeline | 0.92 | 252 | 3 | ★ | +| Swarm | 1.00 | 342 | 4 | ★ | +| Mesh | 1.00 | 1344 | 7 | | +| Debate | 1.00 | 680 | 4 | | + +Even in this toy setting the structure of the trade‑off is visible: the efficient +frontier is {Pipeline (best value), Swarm (best quality at low cost)}; mesh's two +peer rounds make it the most expensive; hierarchical and debate are dominated. +The contribution to validate next is whether, **with a real judge and real +models**, different *task types* select different frontiers. + +## 8. Limitations and future work + +- Results are offline; the production path (real models + tool‑using turns with + live §15 approvals, journaled to `run_events`) is the next integration. +- Several kinds share an executor; richer per‑kind semantics and the remaining + governance/novel forms are future work. +- **Evolution:** with the comparison harness as a fitness function, an + ADAS/Darwin‑Gödel/quality‑diversity loop can propose and switch topologies — + the bridge to *Autonomous Organizational Evolution* on a safe substrate. +- Product surfaces (a visual topology builder and Pareto explorer) will let + non‑experts run these comparisons. + +## 9. Reproduction + +```bash +cargo test -p cm-topology +cargo test -p cm-orchestrator --features provider +cargo run -p cm-orchestrator --example topology_bench --features provider +# real models: set ANTHROPIC_API_KEY before the example. +``` + +Code: `crates/cm-topology` (model/classifier/heuristics), `crates/cm-orchestrator` +(runtime/harness/workflow/judge), `docs/topology-platform.md` (architecture).