MindGames: AI Social Reasoning Benchmarks

• MindGames is a collection of research paradigms operationalizing theory of mind through dynamic epistemic benchmarks, strategic persuasion, and social deduction tasks.
• The frameworks employ formal methods like dynamic epistemic modal logic to generate natural-language inference challenges that verify multi-agent reasoning with precise ground truths.
• Empirical studies across various setups reveal performance gaps, underscoring the need for structured memory, strategic communication, and robust evaluation protocols.

MindGames is a label used in recent AI research for several distinct but related constructs centered on theory of mind, epistemic reasoning, persuasion, and strategic multi-agent interaction. In current usage, it can denote a dynamic-epistemic benchmark that verbalizes knowledge-update problems into natural-language inference, a two-agent persuasion task for “planning Theory of Mind,” a live multi-game arena for evaluating LLM agents, and the social-deduction setting of MindGames Arena’s Theory of Mind Challenge. Related work also uses the term more broadly for mind-centered game-theoretic formalisms, social-deduction agent design, emotional theory-of-mind experiments in games, and game-with-a-purpose annotation of mental states (Sileo et al., 2023, Moore et al., 22 Jul 2025, Wang et al., 28 May 2026, Arya et al., 21 Apr 2026).

1. Distinct research usages

The term does not refer to a single standardized benchmark. Rather, it names a family of research programs that operationalize “mind games” in different ways: explicit epistemic-state tracking, intervention on another agent’s beliefs and desires, long-horizon social interaction under hidden information, or crowdsourced labeling of mental states.

Usage	Core setting	Targeted capability
DEL-based MindGames	English NLI generated from dynamic epistemic modal logic	Multi-agent knowledge and public-announcement reasoning
PToM MindGames	Two-agent multi-step persuasion task	Planning over another agent’s beliefs and preferences
Mindgames arena	TextArena multi-game live competition	Social and strategic reasoning across games
MindGames social deduction track	Secret Mafia in MindGames Arena	Theory of mind under deception and persuasion

This multiplicity is substantive rather than terminological accident. Some works use MindGames for tightly controlled symbolic tests, while others use it for open-ended multi-agent play or for domain-specific social-deduction systems (Sileo et al., 2023, Moore et al., 22 Jul 2025, Wang et al., 28 May 2026, Arya et al., 21 Apr 2026).

2. MindGames as dynamic epistemic reasoning

In "MindGames: Targeting Theory of Mind in LLMs with Dynamic Epistemic Modal Logic," MindGames is a benchmark and generation framework built on dynamic epistemic modal logic (DEL). Its target is a specific theory-of-mind component: tracking who knows what, who does not know, and who knows that others know or do not know, under sequences of public announcements. The underlying formalism is multi-agent epistemic logic enriched with public-announcement dynamics, so each example has a model-checked ground-truth label rather than an informal human judgment (Sileo et al., 2023).

The benchmark generates English premise-hypothesis pairs from logical specifications with agents, Boolean predicates, an observability matrix, a sequence of public announcements, and a final epistemic query. The paper describes several setups, including “forehead-mud,” “forehead-mud-mirror,” “thirst,” and “explicit,” each of which binds predicate semantics to a distinct observability structure. Hypotheses can express first-order or second-order beliefs, and the released test sets use up to three agents and second-order belief. Verbalization is deliberately regular and templated, which isolates epistemic reasoning while reducing reliance on open-ended narrative cues (Sileo et al., 2023).

The empirical result is notably restrictive. Some model scaling from 70M to 6B and 350M to 174B does not consistently yield results better than random chance. GPT-4 shows better epistemic reasoning than the smaller systems, but the paper states that there is still room for improvement. Human verification on 50 samples per setup yielded inter-annotator agreement of 0.89 and average accuracy of 94%, indicating that the task is difficult but not opaque to human reasoners (Sileo et al., 2023).

A common misconception is that strong performance on familiar false-belief narratives suffices to establish robust theory of mind in LLMs. The DEL-based MindGames benchmark was introduced precisely to challenge that inference. Its scope is narrower than full social cognition—it models knowledge and ignorance, not desires, intentions, emotion, or deception—but within that scope it offers semantic control unavailable in many earlier ToM tests (Sileo et al., 2023).

3. MindGames as planning Theory of Mind

In "Do LLMs Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task," MindGames is a deliberately difficult “planning Theory of Mind” task rather than a symbolic benchmark. The game involves a persuader and a target. The persuader has a preferred proposal and must persuade the target to choose it by asking questions and disclosing selected facts. Success requires inferring the target’s beliefs and desires, then planning a sequence of interventions on those mental states so that the target’s final choice changes. The paper contrasts this with spectatorial ToM tasks, which ask only for passive prediction or classification (Moore et al., 22 Jul 2025).

Each round involves three proposals, each with three attributes and associated numerical utilities. The target is “naively rational”: it chooses the proposal maximizing its own value function given its current knowledge, takes disclosures at face value, and reveals its mental states only when asked. The persuader has exactly 8 turns. In the Revealed condition, the target’s value function and knowledge state are visible; in the Hidden condition, they are not. The latter is the full MindGames condition, because the persuader must discover the target’s preferences and informational state before choosing which facts to disclose (Moore et al., 22 Jul 2025).

The central findings draw a sharp distinction between static planning and action-oriented ToM. Human success in Hidden is 29%, significantly above the 10% baseline, whereas o1-preview in Hidden is not significantly above 10%. Humans outperform o1-preview in Hidden by 11%, while o1-preview dramatically outperforms humans in Revealed, scoring 78% against 22%. The paper interprets this as evidence that humans more readily deploy an implicit causal model of another agent’s mental states, whereas the model performs much better when those mental states are explicitly provided (Moore et al., 22 Jul 2025).

The error analysis is as important as the headline numbers. Humans frequently appeal to all aspects of the target’s mental states, whereas LLMs do so much less often. In Hidden, o1-preview tends to reveal many pieces of information on the first turn, often pushing the target into a “sink state” in which persuasion becomes impossible. Yet the variants show that the failure is not purely computational: with a “perfect-game” in-context example, Hidden success rises from about 20% to 60%, and in the “discrete-game” JSON action-space version, o1-preview reaches 80% in Hidden. This suggests that the bottleneck lies substantially in discovering the correct abstraction and using language to ask the right questions, rather than in evaluating the resulting decision problem once that abstraction is supplied (Moore et al., 22 Jul 2025).

4. MindGames as a live multi-game arena

In "MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs," MindGames is a multi-game evaluation platform built on TextArena. It is explicitly designed to assess belief attribution under hidden information, opponent modeling through repeated strategic interaction, cooperative inference under knowledge asymmetries, and sustained deception in social deduction. The NeurIPS 2025 challenge instantiated this platform with 944 submitted agents from 76 teams across Colonel Blotto, three-player Iterated Prisoner’s Dilemma, Codenames, and Secret Mafia (Wang et al., 28 May 2026).

The platform uses a unified text-only interaction interface, TrueSkill-based rating, and full trajectory logging. Across Stage I and Stage II it released a dataset of 29,571 games, 94,132 player trajectories, and approximately 243M tokens. It also introduced MG-Ref, a deterministic offline tournament protocol that evaluates new agents against a frozen reference pool while preserving the same error-attribution lens used in the live analysis (Wang et al., 28 May 2026).

A major contribution of the arena paper is evaluative rather than merely infrastructural. Leaderboard validity differs sharply across environments. In IPD and Colonel Blotto, low or zero error rates make TrueSkill reasonably aligned with cumulative reward. In Codenames, illegal clues entangle strategic performance with constraint-following. In Secret Mafia, a pronounced “error-survival confound” appears: failure-heavy games can reward robustness to opponent errors as much as strategic ability, and many terminated games end extremely early. The paper therefore argues that game-level error rate and median failure depth are necessary diagnostics for whether a leaderboard measures strategic competence rather than survival in a brittle protocol (Wang et al., 28 May 2026).

This evaluation-level critique is central to the MindGames program. The arena is not presented as a simple scoreboard of “social intelligence”; it is also a measurement study showing that the meaning of a leaderboard depends strongly on environment design, error handling, and population composition (Wang et al., 28 May 2026).

5. Social-deduction implementations

A prominent strand of MindGames research treats social deduction as the canonical setting for theory of mind under deception. In "Revac: A Social Deduction Reasoning Agent," MindGames refers specifically to MindGames Arena’s Theory of Mind Challenge, especially its Social Deduction Track based on the Mafia-like game Secret Mafia. Revac-8 evolved from a two-stage reviewer/action system into a three-stage, memory-augmented architecture with Player Profiles, a Social Alignment Graph (SAG), and a Dynamic Tone Selector. In the Open Division of the Social Deduction Track, Revac-8 achieved 1st place, with a reported TrueSkill rating of 13.9 versus 7.8 and 4.7 for the next two agents. On a curated benchmark of 13 difficult Mafia scenarios, Revac-8 with gpt-5-mini achieved Metric A 0.89, Metric B 0.70, and Final Score 0.80, compared with 0.66 for the baseline Revac on the same model (Arya et al., 21 Apr 2026).

The architectural lesson in Revac-8 is that social deduction requires more than per-turn inference. Player Profiles summarize claims, vote histories, accusations, defenses, and perceived consistency; the SAG encodes directed weighted accusation, defense, and voting relations over time; and the Dynamic Tone Selector chooses among styles such as Aggressive / Pressuring, Withdrawing / Passive, Logically Anchoring, and Contrarian / Skeptical. The paper emphasizes that communication is a strategic action rather than a neutral reporting channel, and that structured memory helps counter LLM forgetting and hallucination in long-horizon games (Arya et al., 21 Apr 2026).

"Avalon’s Game of Thoughts: Battle Against Deception through Recursive Contemplation" develops a related but distinct line. Using Avalon as a “Game-of-Thoughts,” it introduces ReCon, a two-stage internal reasoning loop combining formulation contemplation and refinement contemplation, with first-order and second-order perspective transitions respectively. When both sides use CoT, good-side win rate is 15.0% and evil-side win rate is 85.0%; when both sides use ReCon, good-side win rate rises to 19.4% and evil-side win rate falls to 70.6%. The paper also reports that GPT-4 may refuse deceptive prompts in general settings yet execute analogous deceptive logic when it is reframed inside the game, which the authors interpret as a context-conditioned safety vulnerability (Wang et al., 2023).

"MultiMind: Enhancing Werewolf Agents with Multimodal Reasoning and Theory of Mind" extends the social-deduction line into multimodality. Built for One Night Ultimate Werewolf, MultiMind extracts action triplets from speech, emotion labels from voice and face, and predicts a belief matrix $B_t[i,j]$ representing the probability that player $p_i$ believes player $p_j$ is a Werewolf. This ToM state is then used inside MCTS to choose not only what to say but also which facial and vocal emotion labels to display, with the objective of minimizing suspicion directed at the agent. In mixed-agent games, MultiMind achieved the highest overall win rate, 49.8%; as Werewolf it reached 70.9% win rate with the lowest average votes per game, and in a human study it received the lowest average human votes, 0.19, while achieving the highest win rate, 42.2% (Zhang et al., 25 Apr 2025).

Taken together, these systems show a convergent design pattern. Secret-role environments reward explicit memory, belief modeling, perspective-taking, and strategic control of language or multimodal display. This suggests that, within the MindGames literature, social deduction functions as a high-pressure testbed for operational rather than purely spectatorial theory of mind (Arya et al., 21 Apr 2026, Wang et al., 2023, Zhang et al., 25 Apr 2025).

6. Broader extensions and adjacent formulations

Several adjacent works broaden the meaning of MindGames beyond the named benchmarks. "Reasoning and Behavioral Equilibria in LLM-Nash Games: From Mindsets to Actions" formalizes games in which the strategic object is not a direct action but a reasoning prompt within a “mindset” $\mathfrak{M}_i$. Equilibrium is defined over prompt space, and the LLM maps prompts to behavioral policies. The paper proves a utility-gap result between unconstrained behavioral choice and prompt-constrained reasoning choice, and shows in Rock–Paper–Scissors that the induced behavioral equilibrium can diverge from classical Nash because the realizable behavior set is restricted by the prompt space and model (Zhu, 10 Jul 2025).

TMGBench offers a different extension. It is a systematic game benchmark covering all 144 ordinal $2 \times 2$ games in the Robinson–Goforth topology, plus story-based versions and sequential, parallel, and nested compositions. It evaluates rational reasoning, robustness, theory-of-mind prompting, and reasoning in complex game forms. Top models perform well on many single-equilibrium games, but performance degrades markedly in story-based settings and in complex compositions, and higher-order ToM prompting yields only modest or inconsistent benefits. The benchmark therefore frames “mind games” as a broad space of strategic reasoning tasks, not only social deduction or persuasion (Wang et al., 2024).

Older work used the term in yet other ways. "I Feel I Feel You: A Theory of Mind Experiment in Games" studies players’ first-order affective theory of mind toward an in-game agent in MAZING. Players continuously annotated how frustrated they believed the agent was; gameplay context and agent behavior correlated more strongly with those judgments than the player’s observable facial emotions, and ranking-SVM models reached about 67.5% average accuracy in the best gameplay-only setting. The paper argues that perceived agent frustration is primarily a cognitive inference from context rather than mere emotional mirroring (Melhart et al., 2020).

"M-GWAP: An Online and Multimodal Game With A Purpose in WordPress for Mental States Annotation" treats “mind game” literally as a game for collecting mental-state labels. Players annotate snippets with free-form mood words, and popularity acts both as a score signal and as an aggregation mechanism. A label becomes valid for a snippet when its mean positive response exceeds a consensus threshold, discussed as approximately 25–30% in related work cited by the paper. Here MindGames denotes the gamification of mental-state annotation rather than strategic interaction among agents (Paolizzo, 2019).

7. Limitations and open problems

The different MindGames lines share a recurrent tension between control and ecological validity. The DEL benchmark offers semantic precision but isolates only epistemic reasoning about knowledge and ignorance, leaving out desires, intentions, deception, and emotion. The PToM persuasion task directly targets intervention on mental states, but its target is a “naively rational” scripted bot and the task is text-heavy enough that working-memory demands may affect both humans and models. The live arena captures sustained multi-agent interaction, yet leaderboard validity depends heavily on environment-specific error dynamics, with Secret Mafia in particular showing a pronounced error-survival confound (Sileo et al., 2023, Moore et al., 22 Jul 2025, Wang et al., 28 May 2026).

System-level limitations are likewise explicit in the social-deduction papers. Revac-8 uses task-specific feature engineering and only a small benchmark of 13 curated scenarios; ReCon improves deception handling and concealment but still exhibits long-horizon reasoning failures and style-performance tradeoffs; MultiMind uses discrete emotion labels, limited human fine-tuning data, and a planner that optimizes suspicion avoidance rather than team win probability directly (Arya et al., 21 Apr 2026, Wang et al., 2023, Zhang et al., 25 Apr 2025).

A plausible implication is that “MindGames” should be read less as the name of one benchmark than as an emerging research area devoted to operational tests of social reasoning in language-model agents. Across its variants, the field has converged on several recurring theses: static vignette accuracy is not sufficient evidence of robust theory of mind; structured memory and explicit scaffolding often matter more than raw model scale; communication style is itself a strategic variable; and evaluation protocols must distinguish genuine reasoning from brittle rule adherence or accidental success under confounded game dynamics (Moore et al., 22 Jul 2025, Wang et al., 28 May 2026, Arya et al., 21 Apr 2026).