GBC: Gradient-Based Connections for Optimizing Multi-Agent Systems

Abstract: Multi-agent systems (MAS) built on LLMs provide a promising framework for solving complex tasks through role specialization and structured interaction. However, their performance is often limited by miscoordination and, more fundamentally, the lack of fine-grained credit assignment across agents. Existing approaches typically rely on coarse-grained feedback, making it difficult to identify which agents or interaction steps are responsible for errors. We propose Gradient-Based Connections (GBC), an approach for fine-grained attribution and optimization of multi-agent systems. GBC models a MAS as a computational graph and introduces gradient-based connection weights to quantify the influence of each agent's output on downstream agents at the token level. By constructing an attribution graph and propagating task-specific loss signals backward, our method enables precise identification of error sources and targeted prompt optimization. We further develop AgentChord, an efficient implementation that leverages prefix-based gradient computation. Experiments on MultiWOZ and τ-bench show that GBC improves multi-agent performance and outperforms strong single-agent and multi-agent baselines, and higher attribution quality is associated with greater optimization effectiveness. Code is available at: https://github.com/yxc-cyber/AgentChord.

• The paper introduces the GBC framework that applies token-level, gradient-based credit assignment to improve multi-agent system coordination.
• It leverages differentiable connection weights (mean/max L1 norm and gradient–input products) to quantify agent contributions and guide prompt updates.
• Experimental results demonstrate substantial gains, including JGA improvements from 28.9 to 54.4, validating GBC's effectiveness in enhancing MAS performance.

Gradient-Based Connections for Fine-Grained Optimization in Multi-Agent Systems

Problem Formulation and Motivation

Role-specialized multi-agent systems constructed from LLMs promise greater task decomposition and coordination but routinely encounter bottlenecks due to inadequate credit assignment and coarse-grained error feedback. Existing paradigms in MAS optimization—both architectural techniques and prompt-evolution methods—fail to localize failures at the agent, token, or interaction-step level, severely constraining system-level learning dynamics and interpretability. The GBC framework directly targets this attribution deficit by modeling MAS workflows as computation graphs and enabling token-level loss propagation that surfaces the contributions of upstream agents to downstream task errors.

Methodology: GBC and AgentChord Framework

GBC formally represents a multi-agent interaction as a directed acyclic graph, with agents acting as prompt-model pairs and edges encoding information flows. The core innovation is the introduction of differentiable, gradient-based connection weights that—at each step—quantify the sensitivity of downstream agent outputs with respect to upstream agent outputs at the token level. Four variants of connection weights are instantiated: (1) mean L1 norm, (2) max L1 norm, (3) mean gradient–input product, and (4) max gradient–input product, each providing a different tradeoff in attribution sparsity and focus.

On receiving a task-specific loss (articulated as a natural language loss function capable of incorporating eg. joint goal accuracy, inform, or action-level feedback), GBC computes backward attribution graphs. Traversing these graphs produces attribution trajectories that specify which agent outputs, and at what locus, most influenced system-level errors.

Prompt update is performed via an LLM-based optimizer that receives the current state (prompts, toolkits, trajectories, and optimization history) and triggers improvements, incorporating explicit warnings for recurring agent-level errors. To address scalability, a prefix-based gradient approach is integrated: gradients are only tracked for input tokens, while prompts are pre-encoded as fixed prefixes, minimizing memory cost.

Experimental Evaluation

MultiWOZ Task-Oriented Dialogue

GBC is evaluated on MultiWOZ 2.4 under a manager-worker-responder decomposition. Prior to optimization, raw MASs (LLama-3.3-70B-It and Qwen-3-32B variants) fail to exceed robust single-agent LLM baselines across JGA, slot F1, and success. Post-GBC optimization, especially with L1-norm-based connection weights, the Qwen-3-32B multi-agent system achieves best-in-class JGA (54.4 vs. 28.9 pre-optimization), slot F1 (91.4), and inform rate (99.0), all surpassing standalone single-agent performance. Attribution quality, as measured by correct localization of responsible agents within trajectories, is highly correlated with these performance gains.

Error analysis and update frequencies reveal that most remedied errors arise from cross-domain miscoordination and information omission—attribution-guided prompt optimization directly targets these multi-agent pathologies. Domain-specific workers receive most updates, confirming attribution’s precision. However, optimization plateaus on persistent error types such as cross-domain failures and certain forms of over-prediction, indicating limits inherent to current LLM backbone and MAS architecture.

T-bench Tool-Use Environments

The T-bench experiments, focused on multi-step tool-use in the retail domain, extend the GBC framework to environments with long-horizon action tracking. Here, Qwen-3-32B, when GBC-optimized (max L1 norm/product), advances overall reward from 13.0 (unoptimized) to 24.3, outperforming the single-agent baseline (22.6). Gains are driven predominantly by improvements in action reward (from 13.9 to 28.7), confirming that fine-grained credit assignment in tool-use decisions is especially impactful.

Error taxonomies extracted during optimization reveal that tool misuse and retrieval/identification failure are the principal residual failure modes. These, more so than prompt deficiencies, reflect the long-range dependency and grounding challenges present in MAS environments.

Theoretical and Practical Implications

The GBC framework substantiates the need for structural credit assignment in MAS: token-level, differentiable attribution is essential for breaking through MAS performance plateaus evident under legacy approaches. By leveraging GBC, one can move beyond black-box prompt search and RL strategies that are blind to intra-system causality, thereby achieving both greater system transparency and faster convergence in prompt space. The pragmatic memory optimization via prefix-based encoding ensures tractability, but at significant compute cost, especially as the number of agents and task horizon grow.

GBC’s operationalization via AgentChord provides a modular extension point for future research: integration with non-verbal losses, reinforcement learning regimes, or adaptive agent-graph topologies could further enhance both sample-efficiency and generalization. The finding that higher attribution accuracy predicts stronger optimization suggests future directions combining causal analysis with proactive topology redesign and agent specialization.

Limitations and Future Directions

GBC’s reliance on first-order gradient signals (and natural language losses) means it cannot fully capture high-order agent dependencies or implicit context sharing in highly entangled multi-agent workflows. Memory and runtime overhead, while addressed to a degree, remains a bottleneck, especially for large n-agent graphs or extended dialogue/tool-use trajectories. Furthermore, the methodology presupposes high-quality loss functions; errors in verbalized losses can misguide optimization.

Future work should pursue more efficient gradient/attribution computation (possibly via low-rank or surrogate approximations), more robust loss engineering (or automated loss discovery), and integration with adaptive/dynamic agent routing. Extensions into open-ended or self-supervised MAS environments, as well as code and agentic systems with structural ambiguity, will be critical to validate the generality of the GBC approach.

Conclusion

Gradient-Based Connections (GBC) introduce a principled, token-level, differentiable attribution and optimization mechanism for multi-agent LLM systems. By providing structured credit assignment and actionable attribution trajectories, GBC enables effective prompt refinement and system improvement, with empirical evidence demonstrating substantial gains over existing MAS and single-agent baselines on both dialogue and tool-use tasks. The framework surfaces remaining MAS challenges (e.g., cross-domain and context-tracking errors) and sets a foundation for future work in interpretable and optimizable agent architectures (2606.28187).