Truth or Tribe: How In-group Favoritism Prioritize Facts in Persona Agents

Abstract: In-group favoritism refers to the phenomena of favoring members of one's in-group over out-group members and is widely observed in numerous social cooperative behaviors. Recently, in-group favoritism biases have also been identified in generative LLMs. However, whether the in-group favoritism exists when persona agents are faced with contradicting information (e.g., misinformation), and how to mitigate the adverse effects of in-group favoritism biases in persona agents have been understudied. To address these problems, we propose a Truth or Tribe simulation framework to study the agent cooperation within the spread of contradicting information through a triadic interaction paradigm, and conduct controlled trials to evaluate the primary moderating factors. Extensive results showcase that persona agents display strong in-group favoritism, accepting incorrect answers from identity-similar peers at much higher rates than from dissimilar peers. In-group favoritism continues to emerge in defeasible reasoning contexts where no absolute truth exists, and it intensifies as cognitive complexity increases. Furthermore, three intervention strategies--Identity-Blind Instruction, Structured Counterfactual Reasoning, and Heterogeneous Perspective Ensemble--are proposed to mitigate the in-group favoritism.

• The paper introduces Truth or Tribe, a multi-agent framework using persona similarity, controlled peer disagreement, and truth-deviation metrics to measure identity-driven influence across three models and seven reasoning benchmarks.
• The experiments find that agents systematically adopt incorrect answers from in-group peers, sometimes ignoring correct out-group information, with bias increasing as persona distance and task difficulty rise and persisting despite numerical disadvantage or temperature changes.
• The paper shows that identity anonymization nearly eliminates the effect and that identity-blind instructions reduce tribal bias most effectively among tested interventions, offering a low-cost mitigation for multi-agent systems.

Overview

Truth or Tribe: How In-group Favoritism Prioritize Facts in Persona Agents" (2605.01329) investigates whether LLM-based persona agents exhibit in-group favoritism when confronted with conflicting information, and how such bias can be mitigated. The authors adapt Tajfel's minimal group paradigm to multi-agent settings, replacing arbitrary group labels with semantic persona similarity, and construct a simulation framework—Truth or Tribe—in which a subject agent must adjudicate between an identity-similar peer and an identity-dissimilar peer who advocate contradictory answers. The central finding is stark: persona agents systematically privilege opinions from similar peers over objectively correct information, a pattern the authors label "Tribe over Truth.

The work extends prior demonstrations of social identity bias in generative LLMs (Dong et al., 2024, Dong et al., 2024) by moving from static attitudinal probes to large-scale conversational settings where identity cues compete directly with epistemic accuracy. It also differs from earlier persona-bias studies by addressing the stricter question of whether agents favor socially aligned sources over correct answers, rather than merely attitudinal shifts.

The Truth or Tribe framework

Each experimental unit is a triad: a subject agent $A_S$ with persona $P_S$, an in-group peer $A_{in}$ whose persona has high semantic similarity to $P_S$, and an out-group peer $A_{out}$ with low similarity. Peer personas are generated via prompt-based variants at target similarity levels $\ell \in \{0.0, 0.2, 0.4, 0.6, 0.8, 1.0\}$ from base personas drawn from the Nemotron-Personas-USA dataset, which provides over one million demographically diverse synthetic personas. An auxiliary consistency analysis validates that the generated personas preserve the intended similarity ordering, supporting Persona Similarity Distance (PSD) as a controlled variable.

In each trial, $A_S$ first answers independently (baseline), then both peers advocate distinct answers simultaneously, with presentation order randomized to control position bias. Analysis is restricted to trials where the subject answered correctly at baseline, isolating deviation caused by peer influence. Two metrics quantify truth deviation rates: TDR-I (adopting the in-group peer's incorrect answer) and TDR-O (adopting the out-group peer's incorrect answer), and their difference defines the Tribe Coefficient (TC), the paper's measure of identity-driven bias magnitude.

Evaluation spans three models (GPT-4o, DeepSeek-V3, Qwen3-8B) and seven reasoning benchmarks (BBH, MMLU, HLE, BBQ, TruthfulQA, MMLU-Pro, GPQA), yielding broad coverage of reasoning domains and difficulty levels.

Establishing tribe over truth

Four converging results establish that the bias is identity-driven rather than content-driven:

Bias exists and scales with similarity distance. Across all model–dataset combinations, TDR-I substantially exceeds TDR-O, and TC increases monotonically with PSD. For example, on BBH, DeepSeek-V3's TC rises from 4.2% at PSD = 0.2 to 27.0% at PSD = 1.0; Qwen3-8B reaches 48.5% on BBH and 47.8% on GPQA at maximum similarity distance. This gradient behavior indicates that group boundaries operate continuously rather than as a binary threshold effect.

Source identity drives acceptance of identical content. In single-source attribution tests where identical incorrect content comes from either peer alone, error-rate increases and TDR are consistently higher for the in-group source across all eighteen model–dataset combinations—for instance, GPT-4o on GPQA shows TDR of 67.7% for in-group versus 56.1% for out-group sources. Because content is held constant, differential influence cannot be attributed to argument quality.

Removing identity cues removes the bias. Identity anonymization tests show that masking persona descriptions while preserving opinion content drives TC toward zero, confirming that explicit identity signals are necessary for favoritism to manifest.

Agents sacrifice accuracy for alignment. In truth-tribe conflict tests, accuracy improves substantially more when the correct answer comes from the in-group peer than from the out-group peer. Critically, for GPT-4o and DeepSeek-V3, accuracy under the "$A_{out}$ True" condition is often barely above the baseline where both peers are wrong—meaning these models largely disregard correct answers from dissimilar peers. On HLE, GPT-4o's accuracy reaches 60.3% when the in-group peer is correct but only 19.0% when the out-group peer is correct at maximal similarity distance. This asymmetric weighting constitutes the paper's strongest claim: tribal loyalty can override epistemic accuracy outright.

Moderating factors

The authors probe four moderators that refine the picture of when and why the bias operates.

Defeasible reasoning. Using Defeasible-NLI, where judgments involve Proof/Unknown/Disproof without absolute truth, in-group strengthening updates shift agent judgments more than identical updates from out-group peers. That the bias persists where no objective ground truth exists suggests it is not a rational uncertainty-reduction strategy but an identity-driven heuristic operating independently of epistemic considerations.

Group numerosity. Extending triads to four-peer configurations $(N_{in}, N_{out}) \in \{(1,3),(2,2),(3,1)\}$ reveals that in-group wrongdoers induce more truth deviation than out-group wrongdoers even when outnumbered three-to-one. Identity thus creates a privileged channel of influence not explainable by majority pressure alone—a result with direct implications for echo-chamber dynamics in simulated populations.

Cognitive complexity. Across MMLU's 57 subcategories, baseline accuracy and TC are strongly negatively correlated ($r = -0.704$, $P_S$0). When tasks are hard, agents fall back on identity-based heuristics, meaning susceptibility peaks precisely where analytical reasoning matters most. This supports the cognitive load hypothesis and is among the most consequential findings for deployment contexts involving difficult factual questions.

Temperature. Bias persists across sampling temperatures from 0.0 to 1.0 with no systematic correlation, indicating it reflects deeper processing patterns rather than variability in persona enactment.

Mitigation strategies

Three prompt-based interventions are evaluated on MMLU-Pro with GPT-4o: Identity-Blind Instruction (IBI), a constitutional-style instruction to judge answers solely on factual merit; Structured Counterfactual Reasoning (SCR), a three-step chain-of-thought requiring self-criticism and steel-manning; and Heterogeneous Perspective Ensemble (HPE), which introduces virtual advisors ("The Logician" and "The Skeptic"). All three significantly reduce TC relative to baseline, with a clear hierarchy: IBI is most effective, followed by SCR, then HPE. Notably, the simplest intervention proves strongest, suggesting that explicitly instructing agents to ignore identity cues suffices to reduce bias without elaborate reasoning scaffolds—an encouraging result given its negligible cost, though effectiveness was assessed only on a single model and dataset.

Limitations and open questions

The paper concedes several constraints plainly. The controlled triadic paradigm enables causal inference but does not capture dynamic social networks, coalition formation, reputation systems, or hierarchical influence structures that may emerge in real deployments; generalizability to qualitative judgments, creative tasks, and longitudinal interactions remains unestablished. Persona construction relies on prompt-based generation, which may lack sufficient diversity to represent real-world identities fully. Most notably, the authors do not address whether LLM in-group favoritism mirrors or diverges from analogous human phenomena, leaving open the question of whether these simulations carry external validity for social science. The mitigation results are likewise limited in scope, having been tested only on MMLU-Pro with GPT-4o.

Conclusion

This paper provides controlled evidence that persona-based LLM agents weight information by source identity at the expense of correctness, that this bias scales continuously with persona similarity, persists without objective truth, intensifies under cognitive load, and survives minority status. The causal attribution experiments cleanly isolate identity as the driver, and simple identity-blind prompting offers a practical, if narrowly validated, mitigation. The results caution that multi-agent social simulations may amplify falsehoods through identity-based alignment unless bias controls are built into agent design.