Collective Hallucination in Multi-Agent LLMs:Modeling and Defense

Abstract: Hallucinations in LLMs create heightened risks in multi-agent settings, where recursive agent interactions can propagate, reinforce, and amplify unsupported claims. This paper models hallucination as a system-level, time-evolving process across a network of interacting LLM agents, where nodes represent agents and edges encode information exchange. The proposed formulation captures how hallucinated claims diffuse through communication topologies, intensify under adversarial perturbations, and affect collective reliability across reasoning rounds. To suppress error propagation, we introduce an interaction-aware control method that combines confidence-weighted aggregation, adaptive impact regulation, external claim verification, and selective isolation of unreliable agents. Experiments on TruthfulQA and TriviaQA show that the proposed method reduces hallucination by up to 39.0% relative to undefended multi-agent reasoning, improves factual accuracy from 0.79 to 0.87, and increases semantic consistency from 0.75 to 0.84. Under adversarial conditions, the method limits hallucination amplification to 1.08, compared with 1.45 without adaptive control, maintaining stable collective behavior across recursive interaction rounds. These results indicate that hallucination in multi-agent LLM systems is governed by both individual model reliability and system-level interaction dynamics, including communication topology, confidence coupling, and recursive information flow.

• The paper introduces a formal framework to model hallucination propagation as a time-evolving stochastic error in multi-agent LLMs using graph theory and spectral analysis.
• The study quantifies the impact of recursive interactions and adversarial conditions on factual accuracy, showing defense mechanisms that reduce hallucination by up to 39%.
• Adaptive controls integrating trust weighting, external verification, and agent isolation significantly enhance system stability, leading to improved factual accuracy and consistency.

Collective Hallucination Modeling and Defense in Multi-Agent LLM Systems

Problem Formulation and Theoretical Foundations

The paper "Collective Hallucination in Multi-Agent LLMs: Modeling and Defense" (2606.07941) introduces a formal framework for characterizing hallucination as a time-evolving, interaction-driven systemic error within multi-agent LLM architectures. The authors model the multi-agent LLM system as a directed graph, with nodes corresponding to heterogeneous LLM agents and edges representing information propagation channels. Recursive communication leads to closed-loop semantic diffusion, enabling claims—both factual and hallucinated—to be adopted, propagated, and reinforced through iterative context construction and confidence-weighted interaction.

The propagation of unsupported claims is treated as a stochastic perturbation, whose dynamics are determined by probabilistic claim adoption, interaction topology, agent-level confidence, semantic distance, and graph connectivity. The framework incorporates spectral stability analysis, information-theoretic divergence (via KL divergence between generated belief distributions and latent truth distributions), reproduction number analysis, and graph-theoretic risk metrics to profile attenuation, amplification, cascade formation, and stability in collective multi-agent reasoning.

Figure 1: Architecture of collective hallucination modeling in multi-agent LLM systems.

By decomposing agent outputs into atomic semantic claims and tracking hallucination rates, the model provides a macroscopic reliability signal for multi-agent interaction, enabling analysis beyond isolated output-level correctness.

Propagation Dynamics, Topology, and System Behavior

Hallucination propagation is shown to be critically dependent on communication topology and recursive interaction mechanics. Ring topologies achieve local attenuation of hallucination due to constrained diffusion, while fully connected and scale-free topologies facilitate global amplification and cascade formation, especially under adversarial conditions. The spectral radius of the impact matrix and the reproduction number $\mathcal{R}_0$ act as propagation risk indicators: values exceeding unity correspond to self-sustaining recursive amplification and instability.

Adaptive adversarial models targeting recursive context construction, confidence calibration, and verification pathways induce significant propagation amplification, as evidenced by increases in hallucination amplification factor (AF) and reductions in propagation resistance (HPR).

Figure 2: Accuracy--hallucination trade-off across scenarios and defense strategies. Bubble size indicates HPR.

Figure 3: Scenario-level hallucination sensitivity, with the strongest vulnerability under judge corruption.

These results demonstrate that even partial system compromise and localized attacks can trigger systemic failures, particularly when confidence signals are leveraged for trust propagation.

Confidence Dynamics, Entropy, and Agreement States

The paper rigorously analyzes the interaction between confidence, entropy, and reasoning consensus. While higher confidence typically correlates with improved accuracy and reduced hallucination, adversarial propagation and recursive feedback can lead to decoupling: high-confidence, incorrect outputs can drive collective agreement on false claims. The calibration analysis reveals notable gaps, with overconfident agents (e.g., GPT-5.3) increasing propagation risk and false adoption rates.

Figure 4: Defense impact on hallucination and confidence dynamics. (A) Round-level hallucination amplification. (B) Confidence evolution across rounds. (C) Scenario-level confidence distortion. (D) Confidence distribution for hallucinated and non-hallucinated outputs.

Figure 5: Entropy-confidence relationship. Higher collective entropy corresponds to lower confidence stability.

Figure 6: Agreement, diversity, and correctness across collective reasoning states. Low entropy generally aligns with the correct consensus, though adversarial interactions can induce incorrect agreement.

Figure 7: Entropy--hallucination relationship showing transition behavior at intermediate entropy levels. Recursive amplification of unsupported claims occurs in unstable collective-reasoning states.

Figure 8: Calibration behavior across confidence levels. Deviations from the diagonal indicate a mismatch between confidence and observed accuracy, highlighting overconfident, incorrect predictions.

Figure 9: Relationship between confidence and hallucination risk. High-confidence outputs may still contain hallucinations under adversarial conditions.

The collective entropy quantifies agreement and diversity collapse: low entropy may signify either correct consensus or synchronized high-confidence hallucination, depending on adversarial context.

Adaptive Control Mechanism and Defense Evaluation

To mitigate recursive propagation and cascade vulnerability, the authors propose an interaction-aware adaptive defense, integrating confidence-weighted trust reduction, external claim verification, and selective isolation of unreliable agents. Agents are dynamically weighted according to prior hallucination exposure; external verification acts as a corrective feedback layer; and isolation removes high-risk nodes from further propagation.

Across extensive empirical evaluation with TruthfulQA and TriviaQA, the adaptive mechanism demonstrates:

• Hallucination reduction up to 39.0% versus undefended multi-agent reasoning.
• Factual accuracy improvement from 0.79 to 0.87.
• Semantic consistency gains from 0.75 to 0.84.
• Limitation of hallucination amplification to 1.08 (vs. 1.45 without defense), maintaining robust propagation resistance.

The ablation study confirms the necessity of coordinated control components: trust weighting limits impact of unreliable agents; verification corrects semantic drift; isolation curtails cascade growth. Optimal thresholding balances accuracy, attenuation, and agent participation.

Numerical Results and Statistical Analysis

Key quantitative results validate the propagation-aware defense:

• In fully connected and scale-free topologies under attack, AF is reduced from 1.45 to 1.12.
• Under judge corruption, hallucination is reduced from 0.164 to 0.092, AF from 1.45 to 1.12, and $\mathcal{R}_0$ is lowered below propagation boundary.
• The defense achieves the highest effect sizes in accuracy, hallucination reduction, and AF reduction ($>$0.7), as established by rigorous statistical testing.
• Model calibration analysis identifies DeepSeek-V3 as the most stable agent, with minimal calibration gap and propagation risk.

Practical and Theoretical Implications

This work formally demonstrates that hallucination in multi-agent LLM systems is a network-driven phenomenon governed by propagation dynamics, interaction topology, and recursive confidence coupling. Single-model hallucination detection and mitigation approaches fail to control collective propagation, highlighting the necessity for propagation-aware, system-level controls.

Practical consequences include:

• Collective reliability in autonomous agent-based applications (healthcare, finance, legal decision support) cannot be guaranteed through isolated output-level diagnostics.
• Dense agent connectivity and inadequate trust regulation increase the risk of system-wide cascade failures in adversarial and unconstrained deployment scenarios.
• External verification and adaptive trust weighting are essential for robust defense, particularly in heterogeneous agent environments.

On the theoretical side, the paper's spectral and graph-theoretic analysis provides a quantitative foundation for cascade phase transitions and stability in networked LLM systems.

Future Directions

Extensions are warranted toward dynamic, adaptive interaction graphs, richer verification pipelines (e.g., retrieval augmentation or formal logic reasoning), scalability to large agent pools, and domain-specific modeling of hallucination error types. Control-theoretic and game-theoretic methodologies could further regularize agent impact and stabilize collective reasoning. Broader evaluation across multimodal reasoning and real-world heterogeneous deployments is advised for practical validation.

Conclusion

"Collective Hallucination in Multi-Agent LLMs: Modeling and Defense" (2606.07941) establishes hallucination propagation as a systemic risk in multi-agent LLM architectures. The study demonstrates that recursive interaction, confidence coupling, and communication topology drive cascade formation and amplification of unsupported claims, necessitating propagation-aware controls. The adaptive defense substantially improves factual reliability, stability, and propagation resistance, outperforming conventional detection baselines. These findings underscore the critical importance of structure-aware modeling and control for trustworthy multi-agent LLM systems, with significant implications for both theoretical research and practical deployment in high-stakes applications.