ARG-Designer: Autoregressive MAS Topology

• ARG-Designer is a topology-design framework that generates directed acyclic collaboration graphs from task queries using conditional autoregressive techniques.
• It employs a hierarchical autoregressive generator with GRUs and metric-learning retrieval to dynamically select agent roles and communication links.
• Empirical results demonstrate state-of-the-art performance with improved token efficiency and seamless integration of new roles without retraining.

Searching arXiv for the primary paper and related MAS-topology design work to ground the article in current literature. ARG-Designer is a topology-design framework for large-language-model-based multi-agent systems (MAS) that formulates collaboration design as conditional autoregressive graph generation rather than template pruning. Given a natural language task query, it constructs a directed acyclic collaboration graph from scratch, jointly determining the number of agents, selecting their roles from an extensible role pool, and establishing directed communication links. In the reported formulation, this shift is intended to remove redundant agents and edges inherited from dense templates, improve token efficiency, and support the addition of new agent roles at inference time without retraining (Li et al., 24 Jul 2025).

1. Problem setting and conceptual position

In LLM-based MAS, the collaboration topology specifies the agents, their roles, and how they communicate. That topology governs information flow, reasoning structure, and task performance. ARG-Designer is positioned against two earlier design patterns. The first is the use of fixed, hand-crafted topologies such as chains, trees, or fully connected graphs, which can perform adequately in select scenarios but are rigid across heterogeneous tasks. The second is the more recent “graph learning” family, including AgentPrune, AgentDropout, and G-Designer, which still operates under a template graph modification paradigm with a predefined set of agents and hard-coded interaction structures.

Two limitations motivate the ARG-Designer formulation. The first is redundant composition: template graphs are initialized with many roles and dense edges, and pruning can leave irrelevant agents or links in place, increasing token consumption and sometimes degrading decision quality. The second is limited extensibility: when new agent functionalities emerge, methods trained on a fixed agent set do not adapt naturally, while constructing and pruning a very large universal template is described as prohibitively expensive. ARG-Designer addresses these issues by generating a bespoke topology directly from the task query rather than editing a pre-existing one. This design makes composition variable-size through an END decision and makes role selection open to an extensible pool, which distinguishes it from pruning-based methods (Li et al., 24 Jul 2025).

2. Formalization as conditional autoregressive graph generation

ARG-Designer models an MAS topology as a directed acyclic graph $G = (V, E)$, where $V = \{v_1, \ldots, v_N\}$ are agents and each node $v_i$ carries a role $r_i \in R$, with $R$ drawn from an extensible role pool. Edges $E \subseteq V \times V$ are directed communication links. Given a natural language task query $Q$, the system generates nodes and edges incrementally.

The graph distribution is factorized as

$$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$

where $G_{<i}$ contains the first $i-1$ nodes and the edges among them. For each node index $V = \{v_1, \ldots, v_N\}$0, generation proceeds in two stages. In node generation, the model selects the role of $V = \{v_1, \ldots, v_N\}$1 or emits END to terminate construction. In edge generation, it decides for each earlier node $V = \{v_1, \ldots, v_N\}$2, $V = \{v_1, \ldots, v_N\}$3, whether to add the incoming edge $V = \{v_1, \ldots, v_N\}$4.

The same process can be written as an autoregressive action sequence

$V = \{v_1, \ldots, v_N\}$5

with $V = \{v_1, \ldots, v_N\}$6. Acyclicity is guaranteed by construction because edges are only created from earlier nodes to the newly added node. Duplicate edges are excluded because each pair $V = \{v_1, \ldots, v_N\}$7 is decided once, and sparsity is not imposed a priori but learned through the edge decisions themselves (Li et al., 24 Jul 2025).

3. Generator architecture and execution protocol

ARG-Designer is implemented as a hierarchical autoregressive generator built on gated recurrent units. It separates topology generation into a node generator and an edge generator. The task query is encoded by a BERT-like sentence encoder followed by FFN and LayerNorm,

$V = \{v_1, \ldots, v_N\}$8

using all-MiniLM-L6-v2 with embedding dimension $V = \{v_1, \ldots, v_N\}$9. Each role $v_i$0 has an embedding $v_i$1, collected into a role embedding matrix $v_i$2, and a learnable embedding $v_i$3 represents termination.

For node generation, ARG-Designer first summarizes previously generated roles with $v_i$4 to obtain a historical embedding $v_i$5. It then fuses history with the task representation through a scalar gate,

$v_i$6

An edge feature $v_i$7, which encodes the previous node’s connectivity pattern, is concatenated with $v_i$8 and passed through $v_i$9 and $r_i \in R$0 to obtain the node-level hidden state $r_i \in R$1.

Role prediction does not use a fixed-dimensional classifier. Instead, the model applies metric-learning retrieval in a shared embedding space:

$r_i \in R$2

Because selection is similarity-based, new roles can be appended to $r_i \in R$3 at inference time without retraining.

For edge generation, a dedicated $r_i \in R$4 predicts incoming edges to the new node $r_i \in R$5. Its initial hidden state is

$r_i \in R$6

and for each prior node $r_i \in R$7,

$r_i \in R$8

$r_i \in R$9

Once a graph has been generated, MAS execution proceeds for $R$0 communication rounds, with $R$1 in the experiments. A topological sort enforces dependency ordering in each round. Each agent $R$2 emits a message

$R$3

from a prompt

$R$4

which fuses role and internal state with the task and predecessor messages. After the final round, an aggregator produces the output either by voting or by delegating to a terminal agent; the reported implementation uses a summarizer agent (Li et al., 24 Jul 2025).

4. Learning procedure, curriculum, and decoding

Training maximizes the conditional likelihood of ground-truth graphs with supervised learning and teacher forcing. For a dataset $R$5, the objective is

$R$6

Using the node-edge factorization, the total loss is

$R$7

with

$R$8

and

$R$9

The reported balance is $E \subseteq V \times V$0. Teacher forcing feeds ground-truth decisions at each step to stabilize training.

The training regime is explicitly curricular and two-stage. An exploration dataset $E \subseteq V \times V$1 pairs task queries $E \subseteq V \times V$2 with complex configurations $E \subseteq V \times V$3 through $E \subseteq V \times V$4, retaining only empirically successful graph-task pairs:

$E \subseteq V \times V$5

This stage teaches diverse collaborative patterns. An efficiency dataset

$E \subseteq V \times V$6

then emphasizes sparse, compact designs. Here, $E \subseteq V \times V$7 contains minimal manually verified efficient configurations, $E \subseteq V \times V$8 contains successful pruned variants of dense graphs from $E \subseteq V \times V$9, and $Q$0 is a random subset of $Q$1 used to prevent catastrophic forgetting. Training cold-starts on $Q$2 and then fine-tunes on $Q$3 with a lower learning rate.

At inference time, the model samples actions autoregressively until END or a maximum node cap $Q$4 is reached. The paper notes that decoding uses sampling rather than greedy search, and beam search is not reported. New roles can be appended to $Q$5 at inference time, which is the core mechanism behind the claimed plug-and-play extensibility. The reported generation complexity is $Q$6, excluding constant GRU and MLP costs; node selection is $Q$7 per step, and edge generation contributes $Q$8 decisions for node $Q$9. The implementation uses GPT-4o ($$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$0 via the OpenAI API), all-MiniLM-L6-v2 for node encoding, $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$1 training queries per dataset, and a publicly available repository (Li et al., 24 Jul 2025).

5. Empirical evaluation, token efficiency, and ablations

Evaluation is conducted on six test splits across three categories: MMLU for general reasoning; GSM8K, MultiArith, SVAMP, and AQuA for mathematical reasoning; and HumanEval for code generation. Metrics are accuracy for the first five datasets and Pass@1 for HumanEval. Baselines include single-agent CoT and Self-Consistency, fixed-topology MAS variants such as Chain, Tree, Complete Graph, and Random Graph, the debate-style LLM-Debate, and learnable-topology MAS systems including AgentPrune, AgentDropout, and G-Designer. All baselines use $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$2 rounds and the same GPT-4o backend with a summarizer agent for aggregation (Li et al., 24 Jul 2025).

Benchmark	Metric	ARG-Designer
MMLU	Accuracy	89.54
GSM8K	Accuracy	94.37
AQuA	Accuracy	86.45
MultiArith	Accuracy	98.93
SVAMP	Accuracy	95.63
HumanEval	Pass@1	90.91

The reported average across the six benchmarks is 92.48, and the system is described as state-of-the-art on all six. Selected pairwise comparisons indicate improvements over G-Designer on AQuA from 81.60 to 86.45, GSM8K from 93.80 to 94.37, SVAMP from 93.10 to 95.63, and HumanEval from 88.33 to 90.91. Against LLM-Debate, the paper reports gains on AQuA from 77.65 to 86.45 and on GSM8K from 91.40 to 94.37.

A central empirical claim is token efficiency, defined as the total prompt token cost incurred across all agents and rounds, including aggregation. On GSM8K, ARG-Designer achieves 94.37% accuracy using only $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$3 prompt tokens while surpassing G-Designer and consuming approximately 50% fewer tokens. Fine-tuning on the efficiency stage reduces token usage from $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$4 to $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$5 on GSM8K, a 34% reduction, while maintaining strong accuracy; on MMLU, accuracy improves from 88.23% to 89.54% while token usage is reduced by approximately 30%.

The ablation study averaged over MMLU, GSM8K, and HumanEval reports 91.88 for the full model, 91.27 without fine-tuning, 89.77 without the task embedding, and 90.63 without the historical embedding. The largest drop occurs when the task embedding is removed, which the authors interpret as confirming the importance of conditioning on $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$6. This suggests that the main empirical advantage is not merely sparse communication, but joint conditioning of composition and structure on the task query itself (Li et al., 24 Jul 2025).

6. Robustness, extensibility, limitations, and applications

ARG-Designer is reported to show the least performance degradation under a prompt injection attack to a single agent, with a degradation of 2.15% relative to the fixed or naive MAS baselines considered. The paper attributes this to fault-tolerant, distributed designs learned through the curriculum. Extensibility is demonstrated by adding a new role, “Lawyer,” after training: the role is embedded and appended to $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$7, and on an MMLU legal question concerning a contract embargo, ARG-Designer places “Lawyer” centrally and coordinates other experts without retraining.

The generated topologies are qualitatively task-specific. For HumanEval, ARG-Designer often constructs compact pipelines such as an “Implementer” receiving structured hints from a “Planner” or “Tester,” with sparse links intended to minimize unnecessary communication. For GSM8K, it tends to form procedural reasoning and verification structures such as “Decomposer” $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$8 “Reasoner” $$P(G \mid Q, R) = \prod_{i=1}^{|V|} P(v_i \mid G_{<i}, Q, R)\cdot \prod_{j=1}^{i-1} P(e_{ji} \mid v_i, G_{<i}, Q),$$9 “Checker.” For legal MMLU examples, it integrates “Lawyer” as a hub that consults “Researcher,” “Summarizer,” and “Policy Analyst” selectively. These examples are presented as evidence that the system does not merely shrink a fixed graph, but varies both role composition and communication pattern according to task demands.

The paper also identifies several limitations. Training data are synthetic graph-task pairs built from $G_{<i}$0 and $G_{<i}$1, and performance is therefore bounded by the quality and diversity of those datasets. The autoregressive procedure introduces an implicit node-ordering bias, and the GRU architecture is acknowledged as less expressive than transformer decoders for long sequences. The current training objective maximizes the likelihood of successful graphs rather than an execution-time utility such as $G_{<i}$2, so reinforcement learning tied directly to end-task quality or cost is proposed as a future direction. Risks are also noted: automatically designed MAS may coordinate in unexpected ways, amplify biases, or remain susceptible to multi-agent prompt attacks, which motivates safe design constraints and robust verification.

Within the application scope described by the paper, ARG-Designer is relevant to complex reasoning pipelines, code generation, data analysis and decision-making, and domain-specialized workflows such as legal, medical, and financial settings. A plausible implication is that its primary systems contribution lies in making MAS topology a generated object rather than a fixed scaffold, so that task adaptation, communication sparsity, and role extensibility are all handled within a single autoregressive design process (Li et al., 24 Jul 2025).