AgentBalance: Cost-Aware Multi-Agent LLM Design

• AgentBalance is a framework for constructing cost-effective multi-agent systems that operate under explicit token-cost and latency budgets using a backbone-then-topology design.
• It first generates heterogeneous agents by matching LLM backbones to role-specific tasks based on cost-performance profiles and query difficulty.
• Then, it synthesizes an adaptive communication topology through gating and latency-aware strategies, achieving up to 22% performance gains under tight budgets.

AgentBalance is a framework for constructing cost-effective LLM-based multi-agent systems (MAS) under explicit token-cost and latency budgets through a backbone-then-topology design. Its central claim is that, when budgets are binding, the dominant design decision is often which backbones are assigned to which agent roles; only after that should inter-agent communication be optimized. The framework therefore first performs backbone-oriented agent generation through LLM pool construction, pool selection, and role-backbone matching, and then performs adaptive MAS topology generation through agent representation learning, gating, and latency-aware topology synthesis. Evaluated on MMLU, MATH, and HumanEval with 14 candidate LLM backbones, AgentBalance reports up to 10% and 22% performance gains under matched token-cost and latency budgets, respectively, along with strong AUC on performance-versus-budget curves, plug-in improvements for existing MAS, and generalization to unseen LLMs (Cai et al., 12 Dec 2025).

1. Definition, scope, and design intuition

AgentBalance is formulated for deployment settings in which MAS must operate under explicit token-cost and latency budgets rather than unconstrained accuracy maximization. The motivating applications listed for this setting include web search, social-network analytics, online customer support, ride-hailing dispatch, and real-time agents. In these settings, performance gains from deeper reasoning, more inter-agent interaction, or larger reasoning-heavy backbones are operationally constrained because those same choices increase both token expenditure and end-to-end latency (Cai et al., 12 Dec 2025).

The framework is positioned against two families of prior approaches. The first family, described as topology-first methods, includes AgentPrune, G-Designer, and AgentDropout; these methods mainly improve efficiency by changing inter-agent communication while usually assuming a single strong homogeneous backbone. The second family includes MasRouter, which incorporates multiple backbones but is still described as effectively topology-first. The critique is twofold: backbone choice moves the cost-performance frontier more than topology changes do, and the best topology depends on the chosen backbones. This suggests that topology optimization performed before backbone selection can be structurally misaligned with the actual computational and reasoning characteristics of the instantiated agents (Cai et al., 12 Dec 2025).

The system models a MAS as a directed acyclic graph

$G=(V,E),$

where each node is an agent

$$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$

Here, $backbone_i$ is the assigned LLM, $role_i$ is the role-specific instruction, $state_i$ is accumulated history or knowledge, and $plugin_i$ denotes optional tools or APIs. Each agent receives a prompt $P_i$ and outputs

$\mathcal{R}_i = v_i(P_i).$

The communication structure is encoded by an adjacency matrix $E \in \{0,1\}^{|V|\times |V|}$, where $E_{ij}=1$ means agent $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$0 sends a message to $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$1 (Cai et al., 12 Dec 2025).

2. Optimization objective and formal problem formulation

The formal problem is to maximize expected task performance under explicit token-cost and latency budgets. Given a base MAS framework $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$2, AgentBalance optimizes

$$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$3

Here, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$4 is a query sampled from the task distribution $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$5, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$6 is the instantiated MAS for that query, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$7 is task performance, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$8 is token-cost, and $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$9 is latency. The budget parameters are $backbone_i$0 and $backbone_i$1 (Cai et al., 12 Dec 2025).

The base framework contains a template graph $backbone_i$2 and a candidate backbone set $backbone_i$3. Each template is

$backbone_i$4

with backbone unspecified. For a query $backbone_i$5, the configurator returns

$backbone_i$6

with

$backbone_i$7

where $backbone_i$8 is the selected subset of templates, $backbone_i$9 is the backbone assignment mapping, and $role_i$0 is the chosen communication edge set. This decomposition makes three decisions explicit: which roles are active, which backbone is assigned to each role, and how those instantiated agents communicate (Cai et al., 12 Dec 2025).

The constrained optimization is implemented through a Lagrangian-style reward

$role_i$1

where $role_i$2 and $role_i$3 weight token-cost and latency penalties. For each query, the discrete decision tuple is

$role_i$4

where $role_i$5 is the selected LLM pool, $role_i$6 are per-role backbone assignments, $role_i$7 are gating decisions, and $role_i$8 is the synthesized topology. The policy factorizes as

$role_i$9

Training minimizes

$state_i$0

where $state_i$1 weights a hop-length regularizer and $state_i$2 is the learned hop limit (Cai et al., 12 Dec 2025).

3. Backbone-oriented agent generation

The first stage of AgentBalance constructs heterogeneous agents before topology is optimized. It begins from a candidate backbone set $state_i$3, and for each model $state_i$4 builds a profile triple

$state_i$5

Here, $state_i$6 is aggregated benchmark performance, $state_i$7 is per-token price times token count with an extra multiplier for reasoning models, and $state_i$8 is a latency proxy derived from activated parameters and optionally calibrated by small local runs. The paper notes that conventional LLMs are assumed to have similar completion lengths, while reasoning models are assumed to produce longer outputs by a task-dependent factor $state_i$9 (Cai et al., 12 Dec 2025).

Models are then filtered by the 3D Pareto frontier over $plugin_i$0. One explicit example is that Qwen2.5-72B is discarded because it is slower and costlier yet underperforms Qwen3-32B. The surviving backbones are clustered with $plugin_i$1-medoids over

$plugin_i$2

Each cluster becomes an LLM pool. The experiments use four pools, and all agents for a given query are drawn from one selected pool, which serves as a query-level budget control mechanism (Cai et al., 12 Dec 2025).

Backbone selection is conditioned on query difficulty. A lightweight estimator $plugin_i$3 predicts query difficulty from text; the appendix specifies that it is built from MPNet plus a small MLP, pretrained on RouterBench, with supervision given by the fraction of LLMs that solve an example. The paper defines an ease score

$plugin_i$4

and predicts a score $plugin_i$5. In the main pipeline, the predicted difficulty is

$plugin_i$6

which is shifted by a user-controlled offset $plugin_i$7:

$plugin_i$8

Pool selection uses a cost-aware logit

$plugin_i$9

where $P_i$0 is the pool embedding and $P_i$1 is a strictly increasing normalized cost curve over pool indices. After masking disallowed pools and applying softmax, a smoothed bucketizer maps $P_i$2 to a categorical pool-selection probability (Cai et al., 12 Dec 2025).

Within the selected pool, AgentBalance performs query-conditioned role-backbone matching. Backbone descriptors are built from three textual profiles for each model: a performance profile, a PTP profile, and a type profile indicating reasoning or non-reasoning. A lightweight encoder produces $P_i$3, $P_i$4, and $P_i$5. These are fused into a backbone representation $P_i$6. On the role side, the query embedding $P_i$7, the role prompt embedding $P_i$8, and a global pool context $P_i$9 are combined into a role representation $\mathcal{R}_i = v_i(P_i).$0. Compatibility is then scored by the dot product $\mathcal{R}_i = v_i(P_i).$1, and the matching probability is the product of per-role softmax distributions over candidate backbones:

$\mathcal{R}_i = v_i(P_i).$2

This makes backbone assignment simultaneously role-aware, query-conditioned, and budget-sensitive. The ablations indicate that random pool selection and random role-backbone matching substantially reduce cost-effectiveness on MATH and MMLU, increasing token-cost and latency while lowering performance (Cai et al., 12 Dec 2025).

4. Adaptive topology generation and latency-aware coordination

Once heterogeneous agents have been instantiated, AgentBalance synthesizes a per-query communication topology. Because agents differ in model family, model size, reasoning type, role, and query context, the topology generator first learns a unified representation $\mathcal{R}_i = v_i(P_i).$3 for each agent. The final form is

$\mathcal{R}_i = v_i(P_i).$4

where the attention inputs are built from role/query prototypes and transformed backbone descriptors. This representation encodes role semantics, query context, and backbone performance-cost-type information jointly. An ablation using role embeddings only worsens cost-effectiveness, indicating that topology generation benefits from explicit backbone-aware representations (Cai et al., 12 Dec 2025).

The next step is agent gating, which removes roles that are unnecessary for a particular query. With pooled context

$\mathcal{R}_i = v_i(P_i).$5

the keep probability for agent $\mathcal{R}_i = v_i(P_i).$6 is

$\mathcal{R}_i = v_i(P_i).$7

A Bernoulli sample produces the gate decision $\mathcal{R}_i = v_i(P_i).$8, and the retained set is

$\mathcal{R}_i = v_i(P_i).$9

The training implementation uses Gumbel-Sigmoid or Concrete relaxation, and the system enforces at least two active agents by turning on the highest-probability ones if necessary. The paper treats gating primarily as an efficiency mechanism: removing it leaves performance roughly similar in some settings but drastically increases token-cost and latency (Cai et al., 12 Dec 2025).

On the retained agents, topology is generated by learning query-conditioned edge probabilities

$E \in \{0,1\}^{|V|\times |V|}$0

and sampling adjacency entries with Gumbel-Sigmoid. Because latency in MAS is strongly affected by communication depth rather than only graph density, AgentBalance also predicts a hop limit. From pooled context,

$E \in \{0,1\}^{|V|\times |V|}$1

and the hop limit is

$E \in \{0,1\}^{|V|\times |V|}$2

where $E \in \{0,1\}^{|V|\times |V|}$3. If the longest path $E \in \{0,1\}^{|V|\times |V|}$4 in the sampled graph exceeds $E \in \{0,1\}^{|V|\times |V|}$5, the system iteratively removes the lowest-probability edge on the current critical path until the constraint is satisfied. During training, the model penalizes violations with

$E \in \{0,1\}^{|V|\times |V|}$6

The result is a topology policy that is explicitly query-adaptive and latency-aware, rather than a fixed sparse graph reused across queries (Cai et al., 12 Dec 2025).

A plausible implication is that AgentBalance treats communication depth as the main structural latency variable, whereas backbone selection defines the coarse cost-performance regime. This division of labor is consistent with the framework’s core claim that topology should be optimized only after backbone heterogeneity has been resolved (Cai et al., 12 Dec 2025).

5. Empirical evaluation, ablations, and transfer

AgentBalance is evaluated on MMLU, MATH, and HumanEval, using 14 candidate LLM backbones drawn from the Qwen and DeepSeek families. The main baselines are AgentPrune, AgentDropout, G-Designer, and MasRouter; these are built on a Complete Graph framework. For single-LLM baselines, the evaluation uses the upper envelope over budget-performance points rather than a single arbitrarily chosen backbone (Cai et al., 12 Dec 2025).

Benchmark	Token-cost budgets (USD)	Latency budgets (s)
MMLU	0.07, 0.13, 0.32, 0.75	19.0, 42.0, 63.0, 135.0
HumanEval	0.03, 0.04, 0.12, 0.60	9.0, 11.0, 40.0, 100.0
MATH	0.08, 0.17, 0.50, 1.05	28.0, 51.0, 100.0, 210.0

Under these matched budgets, the abstract reports up to 10% gains under token-cost budgets and up to 22% gains under latency budgets. On MMLU, AgentBalance is best across all reported token and latency budget points, with $E \in \{0,1\}^{|V|\times |V|}$7 versus 61.80 for G-Designer and 61.45 for AgentDropout, $E \in \{0,1\}^{|V|\times |V|}$8 versus 86.77 for AgentPrune, $E \in \{0,1\}^{|V|\times |V|}$9, $E_{ij}=1$0 versus 59.69 for G-Designer and 51.59 for MasRouter, and $E_{ij}=1$1. On HumanEval, it achieves $E_{ij}=1$2, $E_{ij}=1$3, and $E_{ij}=1$4, with the caveat that $E_{ij}=1$5 is slightly below G-Designer’s 1.885 even though $E_{ij}=1$6 is the best. On MATH, it is again consistently best, with $E_{ij}=1$7 versus 63.96 for AgentPrune, $E_{ij}=1$8 versus 78.98, $E_{ij}=1$9, and $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$00 (Cai et al., 12 Dec 2025).

The ablation study evaluates six variants: random LLM pools, random pool selection, random role-backbone matching, using role embeddings only in topology generation, removing agent gating, and dense communication topology. The pattern is modular and consistent. Random pool construction or selection reduces performance and often worsens cost or latency. Random role-backbone matching sharply reduces cost-effectiveness, directly validating heterogeneous matching. Removing query and backbone information from topology generation harms performance and increases cost and latency. Removing gating substantially increases token-cost and latency while leaving performance roughly similar in some settings, which confirms that gating mainly serves efficiency. Dense topology significantly increases latency, supporting the need for explicit topology sparsification and hop-depth control (Cai et al., 12 Dec 2025).

AgentBalance is also evaluated as a plug-in for existing MAS frameworks. On MMLU, replacing the original configurators of Layered Graph and AutoGen with AgentBalance improves both tight-budget performance and AUC. For Layered Graph, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$01 improves from 69.26 to 73.20, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$02 from 1.414 to 1.430, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$03 from 45.38 to 73.20, and $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$04 from 165.93 to 173.61. For AutoGen, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$05 improves from 69.20 to 72.55, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$06 from 1.449 to 1.473, $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$07 from 58.97 to 72.55, and $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$08 from 155.13 to 160.55. The framework is further tested without retraining under backbone-set shifts: removing the thinking and non-thinking versions of Qwen3-235B-A22B lowers the frontier and AUC, while adding Qwen3-Next-80B-A3B, DeepSeek-V3, and DeepSeek-R1 improves them, which the paper interprets as evidence that the profiling-based backbone representations generalize to unseen LLMs (Cai et al., 12 Dec 2025).

6. Limitations, interpretive extensions, and broader uses of “balance” in agent research

AgentBalance is explicit about several limitations. Its latency model is simplified: pool construction uses model-level proxies based on activated parameters, and topology control uses longest-path depth rather than a full system model for parallelism, batching, queueing, or provider-side variability. All agents for a query must come from a single selected pool, which simplifies budget control but restricts some cross-tier heterogeneous combinations. Training uses policy gradient over discrete design decisions, and the paper does not deeply discuss variance reduction or optimization stability. The benchmark scope is limited to three standard benchmarks, and several formulas in the paper are noted as typeset imperfectly or compressed (Cai et al., 12 Dec 2025).

In adjacent literature, however, “agent balance” functions as a broader design motif rather than a single named framework. In Agentic Entropy-Balanced Policy Optimization, balance refers to managing entropy in both rollout and policy update so that exploration diversity and gradient stability are preserved simultaneously (Dong et al., 16 Oct 2025). In AOrchestra, balance is expressed as a performance-cost trade-off over dynamically synthesized sub-agents represented by the tuple $$v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.$$09, where instruction, context, tools, and model are allocated at runtime (Ruan et al., 3 Feb 2026). In Device-Native Autonomous Agents for Privacy-Preserving Negotiations, balance denotes the simultaneous management of autonomy vs. privacy, local execution vs. reasoning capability, trust vs. convenience, and latency vs. security (Roy, 1 Jan 2026). In Design and Implementation of Agentic Orchestrations and Orchestration of Agents, balance is the explicit trade-off between autonomy and process-level robustness, tractability, traceability, reactivity, and correctness assurance (Rinderle-Ma et al., 30 Jun 2026).

A broader implication is that AgentBalance belongs to a family of work that treats agent design as multi-objective systems engineering rather than raw capability maximization. That interpretation is reinforced by AgentArch, which reports that no single enterprise architecture dominates across models and workflows and that architectural choices must balance orchestration, prompting style, memory, and thinking-tool integration (Bogavelli et al., 13 Sep 2025), and by Agentomics, which models deployment as coalition formation and values agents by workflow-level surplus after accounting for deployment cost, reliability, and expected failure loss (Zhu, 9 Jun 2026). Within that broader research landscape, AgentBalance is specifically the budget-aware instantiation of this principle for heterogeneous LLM-based MAS: backbone choice defines the economically feasible region, and topology synthesis refines coordination within it (Cai et al., 12 Dec 2025).