docs(paper): draft "Dynamic Agentic Topologies"
Working draft consolidating the platform: thesis, related work (ADAS/Darwin- Gödel/SwarmAgentic + Autonomous Organizational Evolution), the topology model + classifier, the safe-execution result (authority is topology-invariant — the §15 contract holds across any topology because the orchestrator only sequences safe turns), the comparison harness + Pareto + workflow-of-topologies, preliminary offline results, limitations/future (real-model runs, evolution), and a reproduction section. Empirical tables are offline/illustrative placeholders. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Omar Sobh
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
92151f2a90
commit
2621e97ad2
@@ -0,0 +1,176 @@
|
|||||||
|
# Dynamic Agentic Topologies: Safe, Switchable Organizational Patterns for Multi-Agent Systems
|
||||||
|
|
||||||
|
**Status:** working draft. Framework, architecture, and the safety result are
|
||||||
|
implemented (`crates/cm-topology`, `crates/cm-orchestrator`); the empirical
|
||||||
|
tables below are **preliminary, offline (deterministic scripted provider)** and
|
||||||
|
are placeholders for runs against production models.
|
||||||
|
|
||||||
|
**Author:** Omar Sobh · **System:** ClawMates
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Abstract
|
||||||
|
|
||||||
|
The performance of a multi-agent system depends not only on its agents but on
|
||||||
|
their **organizational topology** — how work is decomposed, delegated, and
|
||||||
|
recombined. Prior work shows the best topology is task-dependent and can be
|
||||||
|
searched or evolved (ADAS, Darwin‑Gödel, SwarmAgentic). Those systems, however,
|
||||||
|
largely *simulate* execution and offer no safety contract. We present **Dynamic
|
||||||
|
Agentic Topologies**, a platform that (1) represents organizational topologies as
|
||||||
|
typed graphs, (2) executes a task under any topology on a **real, per‑tenant,
|
||||||
|
human‑in‑the‑loop‑safe runtime**, and (3) compares topologies empirically on a
|
||||||
|
quality/cost Pareto front. Our central design result: because a topology is only
|
||||||
|
an *orchestration over safe agent turns*, **switching topology cannot escalate an
|
||||||
|
agent's authority** — every sandbox‑leaving action remains gated by the same
|
||||||
|
approval/secret‑broker/audit layer regardless of structure. This makes topology a
|
||||||
|
*free variable* teams can tune for results without trading away safety.
|
||||||
|
|
||||||
|
## 1. Introduction
|
||||||
|
|
||||||
|
Builders of agentic systems face two coupled questions: *which agents?* and *how
|
||||||
|
are they organized?* The second — the topology — is under‑explored in production
|
||||||
|
because (a) there is no common vocabulary or runtime for it, and (b) letting
|
||||||
|
structure vary risks letting authority vary with it. We address both. A team
|
||||||
|
defines a system once and runs it under hierarchies, pipelines, swarms, meshes,
|
||||||
|
debates, and more; the platform measures which pattern yields better results for
|
||||||
|
the task, while a fixed safety contract holds across all of them.
|
||||||
|
|
||||||
|
Contributions:
|
||||||
|
1. **A topology model + classifier** (12 execution‑meaningful kinds) as a typed
|
||||||
|
graph with a structural classifier (§3, §4).
|
||||||
|
2. **A topology runtime** that executes a task under any topology by sequencing
|
||||||
|
safe agent turns, with a provable safety invariant (§5).
|
||||||
|
3. **A comparison harness** producing leaderboards and quality/cost Pareto fronts
|
||||||
|
(§6), and a *workflow‑of‑topologies* composition (§6.3).
|
||||||
|
4. A reproducible benchmark and preliminary results (§7).
|
||||||
|
|
||||||
|
## 2. Related work
|
||||||
|
|
||||||
|
- **ADAS — Automated Design of Agentic Systems** (Hu, Lu, Clune): meta‑agent
|
||||||
|
search discovers agent architectures in code.
|
||||||
|
- **Darwin‑Gödel Machine** (Zhang et al.): open‑ended, self‑improving coding
|
||||||
|
agents with an archive of stepping stones.
|
||||||
|
- **SwarmAgentic** (Zhang et al.): language‑space particle‑swarm optimization over
|
||||||
|
whole multi‑agent systems.
|
||||||
|
- **Autonomous Organizational Evolution** (Sobh): composes the three into
|
||||||
|
`O(t+1)=DGM(SwarmAgentic(ADAS(O(t))))` for self‑modifying hierarchies.
|
||||||
|
|
||||||
|
These establish that topology matters and is searchable. Our work supplies the
|
||||||
|
missing **safe execution substrate** and an **empirical comparison** layer, and
|
||||||
|
contributes the safety result that makes topology switching deployable.
|
||||||
|
|
||||||
|
## 3. Topology model
|
||||||
|
|
||||||
|
A topology is a directed graph `TopologyGraph { kind, nodes, edges }`:
|
||||||
|
- **Node** = a role slot `{ id, role, level?, attrs }`, bound to a concrete agent
|
||||||
|
(a "claw") at run time.
|
||||||
|
- **Edge** = a typed relationship `{ from, to, kind }`,
|
||||||
|
`kind ∈ {delegates_to, reports_to, pipes_to, peers_with, routes_to, bids_to,
|
||||||
|
reads_writes}`.
|
||||||
|
|
||||||
|
The v1 taxonomy (`TopologyKind`) covers twelve execution‑meaningful patterns:
|
||||||
|
hierarchical, flat, pipeline, swarm, mesh, hub‑spoke, ring, star/MoE, market,
|
||||||
|
blackboard, debate, holacratic. Each carries advisory **heuristics** (a default
|
||||||
|
role distribution + optimization weights). Topologies are imported from a loose
|
||||||
|
JSON/YAML spec via a normalizing adapter.
|
||||||
|
|
||||||
|
## 4. Topology classification
|
||||||
|
|
||||||
|
Given a graph, we compute structural metrics — density, degree spread, hub
|
||||||
|
dominance, average local clustering, connected components, diameter, and a
|
||||||
|
directed hierarchy score — and score each candidate kind. Clear shapes resolve
|
||||||
|
with high confidence (tree→hierarchical, line→pipeline, cycle→ring, star→hub‑
|
||||||
|
spoke, complete→mesh, empty→flat); "soft" kinds (market/blackboard/debate/
|
||||||
|
holacratic) are weakly inferred and taken from the declared kind. Classification
|
||||||
|
lets us import third‑party org structures and characterize graphs objectively for
|
||||||
|
the benchmark.
|
||||||
|
|
||||||
|
## 5. Safe topology execution (the core result)
|
||||||
|
|
||||||
|
A topology run is executed by **sequencing agent turns** according to the
|
||||||
|
pattern. The runtime is parameterized by a single capability — a `TurnExecutor`
|
||||||
|
that runs one agent turn — and **has no other capability**: it cannot send email,
|
||||||
|
move money, or touch files; it can only ask an agent to act. The real executor
|
||||||
|
wraps ClawMates' per‑tenant runtime, where every sandbox‑leaving action is
|
||||||
|
intercepted and held for human approval, executed via a single‑use secret broker,
|
||||||
|
and written to an append‑only audit journal (the §15 contract).
|
||||||
|
|
||||||
|
> **Invariant (authority is topology‑invariant).** For any topologies `T₁, T₂`
|
||||||
|
> over the same agents, the set of side effects an agent can cause without human
|
||||||
|
> approval is identical under `T₁` and `T₂`. *Sketch:* the only effectful path is
|
||||||
|
> a turn's gated action, which is mediated by the safety layer independently of
|
||||||
|
> the orchestration order; the orchestrator performs no effects itself. Hence a
|
||||||
|
> change of topology re‑routes *information and decisions* but never *authority*.
|
||||||
|
|
||||||
|
This is why topology can be a free variable: teams may search, switch, or evolve
|
||||||
|
structure to improve results with **no** change to the safety surface. The
|
||||||
|
runtime journals per‑step records and aggregate metrics (tokens, turns, gated
|
||||||
|
actions, approvals granted/blocked) for analysis.
|
||||||
|
|
||||||
|
Execution patterns (v1): five distinct executors cover all twelve kinds —
|
||||||
|
hierarchical (delegate down / synthesize up; also hub‑spoke, star/MoE, market),
|
||||||
|
pipeline (staged threading; also ring), swarm (parallel attempts + aggregate;
|
||||||
|
also flat, holacratic), mesh (two peer‑exchange rounds + aggregate; also
|
||||||
|
blackboard), and debate (propose → critique → revise → judge).
|
||||||
|
|
||||||
|
## 6. Comparison and composition
|
||||||
|
|
||||||
|
### 6.1 Harness
|
||||||
|
`compare(graphs, task, executor, scorer)` runs the **same task** across a set of
|
||||||
|
topologies on the **same executor**, scores each output, and returns per‑topology
|
||||||
|
results plus a **leaderboard** (by quality) and a **quality/cost Pareto front**
|
||||||
|
(maximize quality, minimize tokens), with best‑quality and best‑value picks.
|
||||||
|
|
||||||
|
### 6.2 Scoring
|
||||||
|
Quality is produced by a pluggable `Scorer`. A deterministic length proxy is used
|
||||||
|
offline; an **LLM judge** (`JudgeScorer`) rates outputs 0–100 against the task for
|
||||||
|
real runs.
|
||||||
|
|
||||||
|
### 6.3 Workflow of topologies
|
||||||
|
`run_workflow(stages, task, executor)` chains whole topology runs, threading each
|
||||||
|
stage's output into the next (e.g. *swarm* brainstorm → *hierarchical* execute →
|
||||||
|
*debate* review). Each stage is itself a safe run, so the §5 invariant holds at
|
||||||
|
every step.
|
||||||
|
|
||||||
|
## 7. Preliminary results (offline, illustrative)
|
||||||
|
|
||||||
|
Single task ("draft a go‑to‑market launch plan"), deterministic scripted
|
||||||
|
provider, length‑proxy scorer. Numbers are placeholders for production‑model runs.
|
||||||
|
|
||||||
|
| Topology | Quality | Tokens | Turns | Pareto |
|
||||||
|
|--------------|--------:|-------:|------:|:------:|
|
||||||
|
| Hierarchical | 1.00 | 504 | 4 | |
|
||||||
|
| Pipeline | 0.92 | 252 | 3 | ★ |
|
||||||
|
| Swarm | 1.00 | 342 | 4 | ★ |
|
||||||
|
| Mesh | 1.00 | 1344 | 7 | |
|
||||||
|
| Debate | 1.00 | 680 | 4 | |
|
||||||
|
|
||||||
|
Even in this toy setting the structure of the trade‑off is visible: the efficient
|
||||||
|
frontier is {Pipeline (best value), Swarm (best quality at low cost)}; mesh's two
|
||||||
|
peer rounds make it the most expensive; hierarchical and debate are dominated.
|
||||||
|
The contribution to validate next is whether, **with a real judge and real
|
||||||
|
models**, different *task types* select different frontiers.
|
||||||
|
|
||||||
|
## 8. Limitations and future work
|
||||||
|
|
||||||
|
- Results are offline; the production path (real models + tool‑using turns with
|
||||||
|
live §15 approvals, journaled to `run_events`) is the next integration.
|
||||||
|
- Several kinds share an executor; richer per‑kind semantics and the remaining
|
||||||
|
governance/novel forms are future work.
|
||||||
|
- **Evolution:** with the comparison harness as a fitness function, an
|
||||||
|
ADAS/Darwin‑Gödel/quality‑diversity loop can propose and switch topologies —
|
||||||
|
the bridge to *Autonomous Organizational Evolution* on a safe substrate.
|
||||||
|
- Product surfaces (a visual topology builder and Pareto explorer) will let
|
||||||
|
non‑experts run these comparisons.
|
||||||
|
|
||||||
|
## 9. Reproduction
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo test -p cm-topology
|
||||||
|
cargo test -p cm-orchestrator --features provider
|
||||||
|
cargo run -p cm-orchestrator --example topology_bench --features provider
|
||||||
|
# real models: set ANTHROPIC_API_KEY before the example.
|
||||||
|
```
|
||||||
|
|
||||||
|
Code: `crates/cm-topology` (model/classifier/heuristics), `crates/cm-orchestrator`
|
||||||
|
(runtime/harness/workflow/judge), `docs/topology-platform.md` (architecture).
|
||||||
Reference in New Issue
Block a user