---
title: 'AgentBalance: Cost-Aware Multi-Agent LLM Design'
url: https://www.emergentmind.com/topics/agentbalance
type: topic
---

# AgentBalance: Cost-Aware Multi-Agent LLM Design

AgentBalance is a framework for constructing cost-effective large language model (LLM)-based multi-agent systems (MAS) under explicit token-cost and latency budgets through a **backbone-then-topology** design. Its central claim is that, when budgets are binding, the dominant design decision is often which backbones are assigned to which agent roles; only after that should inter-agent communication be optimized. The framework therefore first performs **backbone-oriented agent generation** through LLM pool construction, pool selection, and role-backbone matching, and then performs **adaptive MAS topology generation** through agent representation learning, gating, and latency-aware topology synthesis. Evaluated on MMLU, MATH, and HumanEval with **14 candidate LLM backbones**, AgentBalance reports **up to 10%** and **22%** performance gains under matched token-cost and latency budgets, respectively, along with strong AUC on performance-versus-budget curves, plug-in improvements for existing MAS, and generalization to unseen LLMs [2512.11426].

## 1. Definition, scope, and design intuition

AgentBalance is formulated for deployment settings in which MAS must operate under explicit **token-cost** and **latency** budgets rather than unconstrained accuracy maximization. The motivating applications listed for this setting include **web search**, **social-network analytics**, **online customer support**, **ride-hailing dispatch**, and **real-time agents**. In these settings, performance gains from deeper reasoning, more inter-agent interaction, or larger reasoning-heavy backbones are operationally constrained because those same choices increase both token expenditure and end-to-end latency [2512.11426].

The framework is positioned against two families of prior approaches. The first family, described as **topology-first methods**, includes **AgentPrune**, **G-Designer**, and **AgentDropout**; these methods mainly improve efficiency by changing inter-agent communication while usually assuming a single strong homogeneous backbone. The second family includes **MasRouter**, which incorporates multiple backbones but is still described as effectively topology-first. The critique is twofold: backbone choice moves the cost-performance frontier more than topology changes do, and the best topology depends on the chosen backbones. This suggests that topology optimization performed before backbone selection can be structurally misaligned with the actual computational and reasoning characteristics of the instantiated agents [2512.11426].

The system models a MAS as a directed acyclic graph
$$
G=(V,E),
$$
where each node is an agent
$$
v_i=\langle backbone_i,\ role_i,\ state_i,\ plugin_i \rangle \in V.
$$
Here, \(backbone_i\) is the assigned LLM, \(role_i\) is the role-specific instruction, \(state_i\) is accumulated history or knowledge, and \(plugin_i\) denotes optional tools or APIs. Each agent receives a prompt \(P_i\) and outputs
$$
\mathcal{R}_i = v_i(P_i).
$$
The communication structure is encoded by an adjacency matrix \(E \in \{0,1\}^{|V|\times |V|}\), where \(E_{ij}=1\) means agent \(v_i\) sends a message to \(v_j\) [2512.11426].

## 2. Optimization objective and formal problem formulation

The formal problem is to maximize expected task performance under explicit token-cost and latency budgets. Given a base MAS framework \(M=(G^\ast,\mathcal{B})\), AgentBalance optimizes
$$
\begin{aligned}
\max_{\theta}\quad & \mathbb{E}_{Q\sim\mathcal{D}}\!\left[\mathrm{Perf}\big(F_\theta(M,Q),Q\big)\right] \\
\text{s.t.}\quad & \mathbb{E}_{Q\sim\mathcal{D}}\!\left[\mathrm{Tok}\big(F_\theta(M,Q),Q\big)\right] \le B_{\mathrm{tok}}, \\
& \mathbb{E}_{Q\sim\mathcal{D}}\!\left[\mathrm{Lat}\big(F_\theta(M,Q),Q\big)\right] \le B_{\mathrm{lat}} .
\end{aligned}
$$
Here, \(Q\) is a query sampled from the task distribution \(\mathcal{D}\), \(F_\theta(M,Q)\) is the instantiated MAS for that query, \(\mathrm{Perf}\) is task performance, \(\mathrm{Tok}\) is token-cost, and \(\mathrm{Lat}\) is latency. The budget parameters are \(B_{\mathrm{tok}}\) and \(B_{\mathrm{lat}}\) [2512.11426].

The base framework contains a template graph \(G^\ast=(T,E^\ast)\) and a candidate backbone set \(\mathcal{B}\). Each template is
$$
\tau=\langle \mathrm{role},\,\mathrm{plugin}\rangle,
$$
with backbone unspecified. For a query \(Q\), the configurator returns
$$
F_\theta(M,Q)=(V_Q,E_Q),
$$
with
$$
V_Q = \big\{\langle b_Q(\tau),\,\tau.\mathrm{role},\,\tau.\mathrm{plugin}\rangle:\ \tau\in T_Q\big\},
$$
where \(T_Q\subseteq T\) is the selected subset of templates, \(b_Q:T_Q\to\mathcal{B}\) is the backbone assignment mapping, and \(E_Q\subseteq E^\ast\) is the chosen communication edge set. This decomposition makes three decisions explicit: which roles are active, which backbone is assigned to each role, and how those instantiated agents communicate [2512.11426].

The constrained optimization is implemented through a Lagrangian-style reward
$$
R(G_Q,Q)=\mathrm{Perf}\!\big(F_\theta(M,Q),Q\big)-\lambda_{\mathrm{tok}}\mathrm{Tok}\!\big(F_\theta(M,Q),Q\big)-\lambda_{\mathrm{lat}}\mathrm{Lat}\!\big(F_\theta(M,Q),Q\big),
$$
where \(\lambda_{\mathrm{tok}}\ge 0\) and \(\lambda_{\mathrm{lat}}\ge 0\) weight token-cost and latency penalties. For each query, the discrete decision tuple is
$$
\mathcal{D}=\{p,\{m_i\},\{g_i\},E_Q\},
$$
where \(p\) is the selected LLM pool, \(\{m_i\}\) are per-role backbone assignments, \(\{g_i\}\) are gating decisions, and \(E_Q\) is the synthesized topology. The policy factorizes as
$$
p_\theta(\mathcal{D}\mid Q)=
p_{\mathrm{sel}}\cdot
p_{\mathrm{match}}\cdot
p_{\mathrm{gate}}\cdot
p_{\mathrm{topo}}.
$$
Training minimizes
$$
\mathcal{L}=-R(G_Q,Q)\,\log p_\theta(\mathcal{D}\mid Q)+\lambda_{\mathrm{len}}\mathrm{Pen}_{\mathrm{len}}\!\big(E_Q,L_{\max}\big),
$$
where \(\lambda_{\mathrm{len}}\) weights a hop-length regularizer and \(L_{\max}\) is the learned hop limit [2512.11426].

## 3. Backbone-oriented agent generation

The first stage of AgentBalance constructs heterogeneous agents before topology is optimized. It begins from a candidate backbone set \(\mathcal{B}\), and for each model \(m\) builds a profile triple
$$
z_m=\langle \mathrm{Perf}_m,\ \mathrm{TokCost}_m,\ \mathrm{Lat}_m\rangle.
$$
Here, \(\mathrm{Perf}_m\) is aggregated benchmark performance, \(\mathrm{TokCost}_m\) is per-token price times token count with an extra multiplier for reasoning models, and \(\mathrm{Lat}_m\) is a latency proxy derived from activated parameters and optionally calibrated by small local runs. The paper notes that conventional LLMs are assumed to have similar completion lengths, while reasoning models are assumed to produce longer outputs by a task-dependent factor \(\gamma_{\text{task}}>1\) [2512.11426].

Models are then filtered by the **3D Pareto frontier** over \((\mathrm{Perf}_m,\mathrm{TokCost}_m,\mathrm{Lat}_m)\). One explicit example is that **Qwen2.5-72B** is discarded because it is slower and costlier yet underperforms **Qwen3-32B**. The surviving backbones are clustered with \(k\)-medoids over
$$
\phi(m)=\big[\mathrm{Perf}_m,\ \log \mathrm{TokCost}_m,\ \log \mathrm{Lat}_m\big].
$$
Each cluster becomes an **LLM pool**. The experiments use **four pools**, and all agents for a given query are drawn from one selected pool, which serves as a query-level budget control mechanism [2512.11426].

Backbone selection is conditioned on query difficulty. A lightweight estimator \(f_{\mathrm{diff}}\) predicts query difficulty from text; the appendix specifies that it is built from **MPNet plus a small MLP**, pretrained on **RouterBench**, with supervision given by the fraction of LLMs that solve an example. The paper defines an ease score
$$
p_i=\tfrac{1}{M}\sum_{m=1}^{M}\mathbb{I}\{\text{LLM}_m \text{ answers } x_i \text{ correctly}\}\in[0,1],
$$
and predicts a score \(s_i\in[0,1]\). In the main pipeline, the predicted difficulty is
$$
d=f_{\mathrm{diff}}(Q)\in[0,1],
$$
which is shifted by a user-controlled offset \(\delta\):
$$
d_{\mathrm{eff}} = d + \delta.
$$
Pool selection uses a cost-aware logit
$$
\ell_p = W_P e_p - \alpha\,\bar{c}_p,
$$
where \(e_p\) is the pool embedding and \(\bar{c}_p\) is a strictly increasing normalized cost curve over pool indices. After masking disallowed pools and applying softmax, a smoothed bucketizer maps \(d_{\mathrm{eff}}\) to a categorical pool-selection probability [2512.11426].

Within the selected pool, AgentBalance performs **query-conditioned role-backbone matching**. Backbone descriptors are built from three textual profiles for each model: a **performance profile**, a **PTP profile**, and a **type profile** indicating reasoning or non-reasoning. A lightweight encoder produces \(e_m^{\mathrm{perf}}\), \(e_m^{\mathrm{ptp}}\), and \(e_m^{\mathrm{type}}\). These are fused into a backbone representation \(u_m\). On the role side, the query embedding \(q\), the role prompt embedding \(r_i\), and a global pool context \(g\) are combined into a role representation \(v_i\). Compatibility is then scored by the dot product \(\langle v_i,u_m\rangle\), and the matching probability is the product of per-role softmax distributions over candidate backbones:
$$
p_{\mathrm{match}}=\prod_{i=1}^{|V|}\operatorname{softmax}_m\!\big(\langle v_i,u_m\rangle\big).
$$
This makes backbone assignment simultaneously **role-aware**, **query-conditioned**, and **budget-sensitive**. The ablations indicate that **random pool selection** and **random role-backbone matching** substantially reduce cost-effectiveness on MATH and MMLU, increasing token-cost and latency while lowering performance [2512.11426].

## 4. Adaptive topology generation and latency-aware coordination

Once heterogeneous agents have been instantiated, AgentBalance synthesizes a per-query communication topology. Because agents differ in model family, model size, reasoning type, role, and query context, the topology generator first learns a unified representation \(h_i\) for each agent. The final form is
$$
h_i = r_i^{h} + \gamma\,W_{\mathrm{ctx}}\,\mathrm{attn}(\mathbf{q}_i,\mathbf{k}_i,\mathbf{v}_i),
$$
where the attention inputs are built from role/query prototypes and transformed backbone descriptors. This representation encodes role semantics, query context, and backbone performance-cost-type information jointly. An ablation using **role embeddings only** worsens cost-effectiveness, indicating that topology generation benefits from explicit backbone-aware representations [2512.11426].

The next step is **agent gating**, which removes roles that are unnecessary for a particular query. With pooled context
$$
\bar h=\tfrac{1}{|V_Q|}\sum_i h_i,
$$
the keep probability for agent \(i\) is
$$
p_i=\sigma\!\big(W_g[\,h_i;\bar h\,]\big).
$$
A Bernoulli sample produces the gate decision \(g_i\in\{0,1\}\), and the retained set is
$$
V_Q^{\mathrm{gated}}=\{v_i\in V_Q: g_i=1\}.
$$
The training implementation uses Gumbel-Sigmoid or Concrete relaxation, and the system enforces at least two active agents by turning on the highest-probability ones if necessary. The paper treats gating primarily as an efficiency mechanism: removing it leaves performance roughly similar in some settings but drastically increases token-cost and latency [2512.11426].

On the retained agents, topology is generated by learning query-conditioned edge probabilities
$$
p_{ij}=\sigma(\langle W_a h_i,\; W_a h_j\rangle),
$$
and sampling adjacency entries with Gumbel-Sigmoid. Because latency in MAS is strongly affected by communication depth rather than only graph density, AgentBalance also predicts a **hop limit**. From pooled context,
$$
\pi_{\mathrm{hop}}=\mathrm{softmax}(W_L\bar h),
$$
and the hop limit is
$$
L_{\max}=1+\sum_{k=1}^{N-1} k\,\pi_{\mathrm{hop}}(k),
$$
where \(N=|V_Q^{\mathrm{gated}}|\). If the longest path \(\ell(E_Q)\) in the sampled graph exceeds \(L_{\max}\), the system iteratively removes the **lowest-probability edge on the current critical path** until the constraint is satisfied. During training, the model penalizes violations with
$$
\mathrm{Pen}_{\mathrm{len}}=\operatorname{ReLU}\!\big(\ell(E_Q)-L_{\max}\big).
$$
The result is a topology policy that is explicitly **query-adaptive** and **latency-aware**, rather than a fixed sparse graph reused across queries [2512.11426].

A plausible implication is that AgentBalance treats communication depth as the main structural latency variable, whereas backbone selection defines the coarse cost-performance regime. This division of labor is consistent with the framework’s core claim that topology should be optimized only after backbone heterogeneity has been resolved [2512.11426].

## 5. Empirical evaluation, ablations, and transfer

AgentBalance is evaluated on **MMLU**, **MATH**, and **HumanEval**, using **14 candidate LLM backbones** drawn from the Qwen and DeepSeek families. The main baselines are **AgentPrune**, **AgentDropout**, **G-Designer**, and **MasRouter**; these are built on a **Complete Graph** framework. For single-LLM baselines, the evaluation uses the upper envelope over budget-performance points rather than a single arbitrarily chosen backbone [2512.11426].

| Benchmark | Token-cost budgets (USD) | Latency budgets (s) |
|---|---:|---:|
| MMLU | 0.07, 0.13, 0.32, 0.75 | 19.0, 42.0, 63.0, 135.0 |
| HumanEval | 0.03, 0.04, 0.12, 0.60 | 9.0, 11.0, 40.0, 100.0 |
| MATH | 0.08, 0.17, 0.50, 1.05 | 28.0, 51.0, 100.0, 210.0 |

Under these matched budgets, the abstract reports **up to 10% gains under token-cost budgets** and **up to 22% gains under latency budgets**. On **MMLU**, AgentBalance is best across all reported token and latency budget points, with \(P@T_1 = 71.90\) versus **61.80** for G-Designer and **61.45** for AgentDropout, \(P@T_4 = 88.02\) versus **86.77** for AgentPrune, \(AUC_{\text{tok}} = 1.297\), \(P@L_1 = 71.90\) versus **59.69** for G-Designer and **51.59** for MasRouter, and \(AUC_{\text{lat}} = 250.0\). On **HumanEval**, it achieves \(P@T_1 = 87.94\), \(P@T_3 = 92.20\), and \(P@L_4 = 95.46\), with the caveat that \(AUC_{\text{tok}} = 1.880\) is slightly below **G-Designer’s 1.885** even though \(AUC_{\text{lat}} = 476.5\) is the best. On **MATH**, it is again consistently best, with \(P@T_1 = 66.46\) versus **63.96** for AgentPrune, \(P@T_4 = 79.38\) versus **78.98**, \(AUC_{\text{tok}} = 3.523\), and \(AUC_{\text{lat}} = 602.4\) [2512.11426].

The ablation study evaluates six variants: **random LLM pools**, **random pool selection**, **random role-backbone matching**, **using role embeddings only in topology generation**, **removing agent gating**, and **dense communication topology**. The pattern is modular and consistent. Random pool construction or selection reduces performance and often worsens cost or latency. Random role-backbone matching sharply reduces cost-effectiveness, directly validating heterogeneous matching. Removing query and backbone information from topology generation harms performance and increases cost and latency. Removing gating substantially increases token-cost and latency while leaving performance roughly similar in some settings, which confirms that gating mainly serves efficiency. Dense topology significantly increases latency, supporting the need for explicit topology sparsification and hop-depth control [2512.11426].

AgentBalance is also evaluated as a **plug-in** for existing MAS frameworks. On MMLU, replacing the original configurators of **Layered Graph** and **AutoGen** with AgentBalance improves both tight-budget performance and AUC. For **Layered Graph**, \(P@T_1\) improves from **69.26** to **73.20**, \(AUC_{\text{tok}}\) from **1.414** to **1.430**, \(P@L_1\) from **45.38** to **73.20**, and \(AUC_{\text{lat}}\) from **165.93** to **173.61**. For **AutoGen**, \(P@T_1\) improves from **69.20** to **72.55**, \(AUC_{\text{tok}}\) from **1.449** to **1.473**, \(P@L_1\) from **58.97** to **72.55**, and \(AUC_{\text{lat}}\) from **155.13** to **160.55**. The framework is further tested without retraining under backbone-set shifts: removing the thinking and non-thinking versions of **Qwen3-235B-A22B** lowers the frontier and AUC, while adding **Qwen3-Next-80B-A3B**, **DeepSeek-V3**, and **DeepSeek-R1** improves them, which the paper interprets as evidence that the profiling-based backbone representations generalize to unseen LLMs [2512.11426].

## 6. Limitations, interpretive extensions, and broader uses of “balance” in agent research

AgentBalance is explicit about several limitations. Its latency model is simplified: pool construction uses model-level proxies based on activated parameters, and topology control uses longest-path depth rather than a full system model for parallelism, batching, queueing, or provider-side variability. All agents for a query must come from a **single selected pool**, which simplifies budget control but restricts some cross-tier heterogeneous combinations. Training uses policy gradient over discrete design decisions, and the paper does not deeply discuss variance reduction or optimization stability. The benchmark scope is limited to **three standard benchmarks**, and several formulas in the paper are noted as typeset imperfectly or compressed [2512.11426].

In adjacent literature, however, “agent balance” functions as a broader design motif rather than a single named framework. In **Agentic Entropy-Balanced Policy Optimization**, balance refers to managing entropy in both rollout and policy update so that exploration diversity and gradient stability are preserved simultaneously [2510.14545]. In **AOrchestra**, balance is expressed as a performance-cost trade-off over dynamically synthesized sub-agents represented by the tuple \(\Phi=(I,C,T,M)\), where instruction, context, tools, and model are allocated at runtime [2602.03786]. In **Device-Native Autonomous Agents for Privacy-Preserving Negotiations**, balance denotes the simultaneous management of autonomy vs. privacy, local execution vs. reasoning capability, trust vs. convenience, and latency vs. security [2601.00911]. In **Design and Implementation of Agentic Orchestrations and Orchestration of Agents**, balance is the explicit trade-off between autonomy and process-level robustness, tractability, traceability, reactivity, and correctness assurance [2606.31518].

A broader implication is that AgentBalance belongs to a family of work that treats agent design as **multi-objective systems engineering** rather than raw capability maximization. That interpretation is reinforced by **AgentArch**, which reports that no single enterprise architecture dominates across models and workflows and that architectural choices must balance orchestration, prompting style, memory, and thinking-tool integration [2509.10769], and by **Agentomics**, which models deployment as coalition formation and values agents by workflow-level surplus after accounting for deployment cost, reliability, and expected failure loss [2606.14769]. Within that broader research landscape, AgentBalance is specifically the budget-aware instantiation of this principle for heterogeneous LLM-based MAS: backbone choice defines the economically feasible region, and topology synthesis refines coordination within it [2512.11426].

Source: https://www.emergentmind.com/topics/agentbalance