Can AI Agents Agree?

Abstract: LLMs are increasingly deployed as cooperating agents, yet their behavior in adversarial consensus settings has not been systematically studied. We evaluate LLM-based agents on a Byzantine consensus game over scalar values using a synchronous all-to-all simulation. We test consensus in a no-stake setting where agents have no preferences over the final value, so evaluation focuses on agreement rather than value optimality. Across hundreds of simulations spanning model sizes, group sizes, and Byzantine fractions, we find that valid agreement is not reliable even in benign settings and degrades as group size grows. Introducing a small number of Byzantine agents further reduces success. Failures are dominated by loss of liveness, such as timeouts and stalled convergence, rather than subtle value corruption. Overall, the results suggest that reliable agreement is not yet a dependable emergent capability of current LLM-agent groups even in no-stake settings, raising caution for deployments that rely on robust coordination.

• The paper demonstrates that LLM-driven agents struggle to achieve consensus, with only 41.6% valid outcomes in benign settings and even lower rates (15.8%) for the Qwen3-8B model.
• It employs Monte Carlo simulations using the A2A-Sim platform to evaluate consensus dynamics across varying group sizes, prompt conditions, and Byzantine fractions.
• It finds that even minimal Byzantine presence or adversary-aware prompts trigger liveness failures rather than value corruption, highlighting the fragility of current LLM architectures for distributed coordination.

LLM-Based Agents and the Limits of Consensus: An Analysis of "Can AI Agents Agree?"

Problem Statement and Motivation

The study systematically interrogates the capacity of modern LLM-driven agents to achieve consensus within a synchronous Byzantine fault-prone environment. The evaluation centers on the canonical scalar consensus problem, where agents iteratively broadcast proposals over rounds, but it is conducted in a zero-sum, no-stake scenario that isolates pure agreement from incentive-driven behaviors. Crucially, the analysis refuses to treat consensus as a solved property for LLM systems, explicitly examining whether robust agreement—central to distributed coordination and fault tolerance—emerges ex proprio motu for current LLM agents, in light of the stochastic generation and lack of deterministic guarantees. The methodology focuses on Qwen3-8B and Qwen3-14B models, varying both honest group size and Byzantine fraction.

Figure 1: Valid consensus rates for Qwen3-8B and Qwen3-14B under varying group sizes and prompt conditions, showing substantial performance drop with increased $N$ and any mention of adversaries.

Experimental Design and Simulator Architecture

The experimental platform, A2A-Sim, simulates synchronous all-to-all messaging, with agents’ state transitions governed by prompt-driven LLM outputs. Each round, agents receive a structured summary (scarcity-induced due to context window constraints) and generate not only proposal values but also justifications and explicit votes for termination. Consensus is declared if at least two-thirds of all agents decide to stop and the honest subset agrees on a value initially proposed by an honest peer. The Byzantine threat model is intentionally restricted: adversaries cannot equivocate nor suppress messages—every agent receives the same vector of messages, and identity is consistent. This mirrors a weakest-link analysis, quantifying the agreement fragility under minimal adversarial behavior and partial awareness.

Methodologically, each parameterization (model, group size, adversarial prompt, Byzantine fraction) is subjected to 25-run Monte Carlo evaluation, recording the rates of valid/invalid consensus and instances of liveness failure (timeouts, stalled convergence).

Empirical Results

Consensus Failure Even in Benign Configurations

Empirical results demonstrate that LLM-agent systems exhibit persistent failure modes even in the complete absence of Byzantine behavior. Across all Qwen3-8B/14B configurations, only 41.6% of non-adversarial runs achieved valid consensus, with Qwen3-8B attaining an especially poor 15.8% rate—indicative of a pronounced gap in emergent coordination ability at smaller scales.

Figure 1: Valid consensus without Byzantines as a function of group/model size and adversarial prompt context.

Increasing agent count to $N=16$ (from $N=4$) degraded the probability of successful consensus from 46.6% to 33.3%, evidencing that larger collectives exacerbate stochastic divergence and indecision even in idealized scenarios. Prompting agents to be “adversary-aware” (mentioning possible Byzantines) consistently worsened liveness, reducing consensus rates and doubling convergence times, a finding that underscores prompt-induced fragility.

Impact of Byzantine Agents

Introducing as few as a single Byzantine agent ($f = 1/9$) within groups of Qwen3-14B sharply reduced successful consensus events; the primary observable effect was not misdirection towards invalid outcomes, but outright liveness collapse—timeouts dominated failure cases, not value corruption. Even at the upper theoretical tolerable bound of $f=1/3$, valid consensus was nearly unattainable.

Figure 2: Distribution of consensus outcomes as the number of Byzantine agents increases, highlighting that timeouts (liveness failures) rather than value corruption dominate.

Notably, increases in the Byzantine fraction did not correspond to higher invalid consensus rates, which remained negligible. Instead, agent stalemates and premature terminations dominated as the main cause of consensus breakdown, implicating a lack of resilient collective convergence rather than susceptibility to explicit adversarial manipulation.

Proposal Trajectories and Convergence Dynamics

Examination of honest agent proposal trajectories further reveals the insufficiency of both model capacity and prompt engineering in guaranteeing reliable agreement. In threat-unaware conditions, convergence to a shared value occurs more frequently and in fewer rounds, but honest agents confronted with “Byzantine-aware” prompts or actual Byzantines display protracted and erratic trajectories, frequently oscillating without termination before final agreement.

Figure 3: Honest agent scalar proposals over rounds, comparing convergence dynamics with and without explicit adversary modeling or presence.

Interpretation and Broader Implications

These results collectively challenge the presumption that LLM-based agents, even at substantial parameter counts and under no-stake regimes, possess any inherent or robust emergent consensus capability. Critical Byzantine-tolerant properties like safety (validity) appear trivially maintained (due to the constrained Byzantine model), but liveness—progress towards agreement—is acutely fragile. The deleterious effect of prompt formulations that prime for adversary awareness suggests that current LLMs’ lack of protocol-level reasoning renders them hypersensitive to textual framing, with detrimental effects on group coordination even in optimal conditions.

Practically, these findings signal caution for all deployments where AI agents must synchronize state under decentralization, partial trust, or adversarial threat. The absence of liveness guarantees in even benign, controlled experiments implies that LLM agents are ill-suited, in their current form, for DAOs, decentralized optimization, or any critical distributed decision task requiring consensus. Theoretically, the results delineate a divergence point between stochastic LLM-driven agents and classical deterministic algorithms' proven properties in distributed systems, exposing the limitations of treating LLM policy composition as a surrogate for robust protocol synthesis.

Future Directions

The constrained threat model (non-equivocating, non-dropping Byzantines) and homogeneous agent instantiation leave open avenues for extension: testing against Byzantine strategies employing more adaptive or subversive tactics, introducing population heterogeneity, relaxing prompt synchronization, and evaluating with domain-adaptive LLMs or external memory architectures. There is also a clear imperative to architect or align agent-level inductive biases and interaction protocols that more closely approximate deterministic algorithmic consensus, rather than relying on naive self-organization.

Conclusion

The investigation establishes, through rigorous simulation and statistical analysis, that LLM-driven agents do not reliably achieve consensus, even in elementary agreement tasks with mild network or adversarial assumptions. The core failure results from systemic liveness collapse and prompt-induced indecision, not adversarial value injection, and worsens with scale and adversarial anticipation. As such, current LLM architectures and prompting paradigms must not be assumed to support dependable distributed agreement; explicit protocol embedding and new agent architectures are required before LLM-agent collectives can underpin safety-critical or adversarially robust multi-agent infrastructures.