Theory of Mind Utility (ToM-U)

• Theory of Mind Utility is a computational framework that formally defines mental state inference via Local Epistemic World Models (LEWMs) to improve predictive and decision-making outcomes.
• It utilizes structured candidate evaluation and recursive mentalizing to enable proactive coordination in multi-agent, teaching, and HRI settings.
• Empirical applications demonstrate its utility across active inference, adaptive teaching, cooperative reinforcement learning, and user-agent interaction, highlighting role-dependent benefits.

Searching arXiv for the provided ids to verify availability and recency. arXiv search: (Gurney et al., 10 Jun 2026) Theory of Mind Utility (ToM-U) designates, in its strongest explicit formulation, a computational-level specification of mentalizing that infers another agent’s epistemic state by constructing and evaluating candidate Local Epistemic World Models (LEWMs) against observed behavior (Gurney et al., 10 Jun 2026). In adjacent research areas, the same expression is often implicit rather than explicit. Multi-agent active inference treats ToM as useful because it enables proactive coordination without explicit communication (Pitliya et al., 1 Aug 2025); adaptive teaching treats it as useful because a teacher can select demonstrations that maximise the learners' rewards while minimising teaching costs (Grislain et al., 2023); human-robot interaction treats it as useful insofar as it helps a robot infer and respond to human mental states, adapt its internal model to user behavior, and generate explanations that are more intuitive, user-aligned, interpretable, and predictive of robot behavior, provided those explanations are also faithful to the robot’s actual internal reasoning (Bauer et al., 29 Dec 2025). Taken together, these lines of work support a broad but technically precise interpretation: ToM-U concerns when and how explicit modeling of other minds improves prediction, coordination, explanation, teaching, or decision-making.

1. Conceptual scope and definition

The term “Theory of Mind Utility” is not used uniformly across the literature. Some papers explicitly define or formalize it, whereas others provide a paper-grounded notion of utility without introducing the term itself. The most explicit formulation appears in “The Theory of Mind Utility: Formal Specification of a Mentalizing Mechanism,” which treats ToM-U as a formal specification of epistemic state inference rather than as a reward bonus, a prompt template, or a benchmark score (Gurney et al., 10 Jun 2026). By contrast, “Theory of Mind for Explainable Human-Robot Interaction” states that it does not explicitly define a construct called “Theory of Mind Utility (ToM-U),” but it does provide enough material to infer a paper-grounded notion of ToM’s utility in HRI (Bauer et al., 29 Dec 2025).

Across these uses, a stable core emerges. Theory of Mind is useful when the behavior of one agent depends on another agent’s beliefs, desires, intentions, goals, percepts, or false beliefs, and when modeling those states changes the focal agent’s action selection, explanation strategy, or intervention choice. In the active-inference formulation, the focal agent explicitly separates “what I believe about the world” from “what I believe the other believes about the world,” enabling first-order belief attribution and false-belief-like divergences in principle (Pitliya et al., 1 Aug 2025). In the adaptive-teaching formulation, the same general idea appears as learner-relative demonstration choice under posterior uncertainty over learner goal and sensory capacity (Grislain et al., 2023). In the HRI formulation, utility depends not only on user-centeredness but also on fidelity: plausible explanations must also correspond to the robot’s actual internal reasoning (Bauer et al., 29 Dec 2025).

This suggests that ToM-U is best understood as a family of formalisms and evaluations centered on a single question: under what conditions does explicit mental-state modeling produce better epistemic or behavioral outcomes than non-ToM baselines?

2. Formal specification at the computational level

The most detailed formal account specifies ToM-U at Marr’s computational level. Its central object is the Local Epistemic World Model, defined as a directed typed graph

$$W = \langle A,\, O,\, E,\, \text{BLS},\, \text{obs},\, \text{cred} \rangle$$

where $A$ is a finite set of agent nodes, $O$ is a finite set of state nodes, $E$ is a set of directed typed edges, $\text{BLS}$ maps each edge to a belief-like state type and scalar value, $\text{obs}$ assigns observability, and $\text{cred}$ assigns accumulated inferential credibility (Gurney et al., 10 Jun 2026). The framework does not presuppose belief states. Instead, it derives belief-like state estimates from ordered information access history, source credibility, and observability constraints.

Candidate LEWMs are evaluated against observed behavior, and confidence accumulates across candidate evaluations according to

$C_n = 1 - \prod_{j=1}^{n}(1 - f_j),$

where $f_j \in [0,1]$ is the fit score of the $j$-th candidate configuration (Gurney et al., 10 Jun 2026). The framework also defines a bounded proliferation mechanism for recursive mentalizing through the expected marginal coherence gain

$A$0

together with stopping thresholds $A$1 and $A$2 (Gurney et al., 10 Jun 2026). Rejected candidates are not discarded as null events; they leave a structured inferential trace through a residue function. This is a notable departure from accounts in which failed mentalizing attempts simply vanish from the formalism.

A related but distinct formalization appears in “A Causal Model of Theory of Mind in Conflict for Artificial Intelligence,” which shifts emphasis from what mentalizing computes to when mentalizing should be engaged at all (Gurney, 15 Jun 2026). There, ToM is modeled as a selectively activated mechanism whose value depends on situational and agent-level conditions. Engagement is triggered by a weighted combination of information asymmetry, low accessible tractability, and miscalibrated relative sophistication:

$A$3

Acceptance of the ToM output is then modeled separately:

$A$4

with $A$5 taking values $A$6, $A$7, or $A$8 (Gurney, 15 Jun 2026). The primary outcome is epistemic accuracy rather than behavior policy. This makes ToM-U explicitly resource-rational: mentalizing is warranted when it has useful causal work to do, and not otherwise.

3. Mechanisms by which ToM produces utility

Several formal traditions implement ToM-U through distinct computational mechanisms.

In multi-agent active inference, utility arises because the focal agent maintains distinct internal representations for itself and for the other agent:

$A$9

The crucial distinction is between the focal agent’s own world beliefs and its beliefs about what the other agent believes about the world (Pitliya et al., 1 Aug 2025). Planning is lifted from a single-agent policy tree to a recursively expanded joint policy tree. At each horizon, the focal agent expands possible actions of the other agent, updates its own world beliefs using likelihood-message passing, expands observations for both agents, and performs a backward pass propagating recursive expected free energy values from leaves to root (Pitliya et al., 1 Aug 2025). The practical consequence is proactive coordination: the focal agent changes its own action because it anticipates how the other agent is likely to act.

In adaptive teaching, ToM-U is embedded directly in a utility-maximizing pedagogical decision rule. The teacher maintains a posterior over learner type $O$0, where $O$1 is the learner’s goal and $O$2 its observation function or sensory capacity, inferred from an observed trajectory in an observation environment (Grislain et al., 2023). The teacher then evaluates candidate demonstrations in a separate demonstration environment under the posterior over learner states. Utility is explicitly defined as learner reward after teaching minus demonstration cost, and the expected utility of a candidate demonstration is computed by posterior-weighting over learner hypotheses (Grislain et al., 2023). Here ToM is not only predictive; it is instrumentally necessary because the same demonstration has different utility for different latent learner states.

In cooperative reinforcement learning, “Theory of Mind with Guilt Aversion Facilitates Cooperative Reinforcement Learning” makes the agent’s effective utility belief-dependent. Agent $O$3 computes the expected material value experienced by agent $O$4,

$O$5

and incurs a psychological penalty if the realized outcome gives $O$6 less than this expected value:

$O$7

The shaped reward becomes

$O$8

so ToM-U here is literally a utility transformation driven by nested beliefs about another agent’s expectations (Nguyen et al., 2020).

A related but different mechanism appears in temporal and memory-augmented ToM models for LLMs and observer networks. “TimeToM” constructs a temporal space, builds a Temporal Belief State Chain for each character, separates self-world beliefs from social-world beliefs, and reduces some higher-order ToM questions to first-order questions over belief communication periods (Hou et al., 2024). “Memory-Augmented Theory of Mind Network” equips the observer with external memory and hierarchical attention so that distal, belief-relevant events can be selectively retrieved when predicting current beliefs and future behavior (Nguyen et al., 2023). In both cases, utility arises because the model preserves or reconstructs the informational history on which belief attribution depends.

Setting	ToM representation	Utility notion
Active inference (Pitliya et al., 1 Aug 2025)	Separate self/other belief states; joint policy tree	Better prediction of others, implicit coordination, less wasted effort
Adaptive teaching (Grislain et al., 2023)	Posterior over learner goal and observation function	Learner reward after teaching minus demonstration cost
Guilt-averse RL (Nguyen et al., 2020)	Zero-order and first-order beliefs	Reward shaping by disappointment relative to the other’s expected payoff
HRI/XAI (Bauer et al., 29 Dec 2025)	Inferred human mental states evaluated via VXAI	Interpretability, predictability, trust, with fidelity as a constraint

4. Empirical domains and demonstrated forms of utility

The empirical literature does not converge on a single task family. Instead, ToM-U has been demonstrated or argued for across cooperation, teaching, HRI, user-agent interaction, persuasion, and bargaining.

In the active-inference setting, the paper evaluates collision avoidance and apple foraging in a deterministic $O$9 grid world (Pitliya et al., 1 Aug 2025). In collision avoidance, non-ToM agents both choose the shortest path through the center and become permanently stuck, whereas the ToM agent predicts that the other agent is likely to choose the center route and therefore selects a longer route around the center. In apple foraging, non-ToM agents both exploit the known apple location, while the ToM-equipped agent predicts that the purple agent will go to the known apple and therefore explores a different orchard location whose reward is uncertain (Pitliya et al., 1 Aug 2025). The demonstrated utility is conflict avoidance and redundancy reduction without explicit communication or hand-coded coordination rules.

In adaptive teaching, the paper uses MiniGrid observation and demonstration environments in which a teacher first observes learner behavior and then chooses a demonstration. Learners taught by the ToM-equipped teacher are more efficient than those taught in a learner-agnostic way, and the effect gets stronger when the teacher’s model of the learner better aligns with the actual learner’s state, either using a more accurate prior or after accumulating observations of the learner’s behaviour (Grislain et al., 2023). This provides a direct link between posterior quality and realized teaching utility.

Human-robot interaction offers a different form of ToM-U. Here ToM is characterized as a user-friendly explanatory backend and as a form of XAI (Bauer et al., 29 Dec 2025). The VXAI evaluation applied to eight HRI studies uses seven desiderata—Parsimony, Plausibility, Coverage, Fidelity, Continuity, Consistency, and Efficiency—and reports that all studies satisfy Parsimony and Plausibility, no study satisfies Coverage, no study satisfies Fidelity, only Yuan and Angelopoulos satisfy Continuity and Consistency, and Efficiency is satisfied by most, but not all, studies (Bauer et al., 29 Dec 2025). The article’s implication is narrow but important: user-centered utility claims about trust, transparency, and understanding remain under-validated unless explanations are faithful to the robot’s internal reasoning.

User-agent interaction in LLM systems is treated in “Beyond Words: Evaluating and Bridging Epistemic Divergence in User-Agent Interaction via Theory of Mind,” which formalizes ToM as a mechanism for epistemic divergence detection and resolution (Ruan et al., 14 Feb 2026). The SynchToM benchmark contains 390 test instances, each with a 10-turn trajectory, across software engineering, preference modeling, education, and culture differences. The paper reports that injecting correct belief/profile information consistently improves the Solution score, while shuffled mental states degrade it, and that a GRPO-based reinforcement-learning procedure trained on a 6,522-instance trajectory-based ToM dataset improves Solution especially in multi-turn settings (Ruan et al., 14 Feb 2026). This is one of the clearest empirical demonstrations that ToM factors are causally linked to downstream task resolution rather than functioning merely as explanatory side outputs.

Negotiation provides yet another operationalization. In ultimatum-game simulations, ToM prompting is used to steer LLM agents toward human-aligned bargaining norms (Yadav et al., 30 May 2025). Across 2,700 simulations, the paper reports that first-order ToM is best for proposer alignment, Both ToM is best for responder acceptance, and zero-order ToM is best for responder rejection. Fair-Fair belief combinations resulted in the highest alignment with human norms, and reasoning models exhibit limited capability compared to models with ToM reasoning (Yadav et al., 30 May 2025). This indicates a task-dependent and role-dependent utility landscape rather than a monotonic “more ToM is better” pattern.

5. Evaluation, benchmarking, and measurement problems

A substantial part of the ToM-U literature concerns how utility should be measured. One influential pattern is the distinction between inferring mental states and using those inferred states for downstream social judgment. PersuasiveToM makes this distinction explicit through two benchmark tasks: ToM Reasoning and ToM Application (Yu et al., 28 Feb 2025). The benchmark shows that LLMs are relatively strong at easy or static recognition such as persuader desire, much weaker on dynamic persuadee desire and persuader intention, and only partially successful at ToM Application, especially strategy prediction. Human performance remains substantially higher on persuader intention and dialogue-level consistency (Yu et al., 28 Feb 2025). This suggests that ToM-U is harder than ToM-as-multiple-choice-recognition.

UniToMBench extends this logic by combining TOMBENCH task families with 1,025 custom dynamic scenarios, split into 500 multi-interaction tasks and 525 evolving story tasks (Thiyagarajan et al., 11 Jun 2025). The paper reports that GPT-4o and GPT-4o Mini show consistently high accuracy in tasks involving emotional and belief-related scenarios, with results usually above 80%, but that there is significant variability in their performance across knowledge-based tasks (Thiyagarajan et al., 11 Jun 2025). Perspective-taking helps selectively rather than uniformly. This matters because a utility notion based only on aggregate accuracy would obscure strong subskill asymmetries.

Two meta-level critiques sharpen the evaluation problem. “Rethinking Theory of Mind Benchmarks for LLMs: Towards A User-Centered Perspective” argues that what matters in practice may be less whether LLMs possess ToM reasoning capabilities and more about the type of downstream behaviors enabled by LLM’s ToM capabilities during human-AI interactions (Wang et al., 15 Apr 2025). It criticizes static, third-person, synthetic tasks and calls for dynamic, interactional evaluation that accounts for user preferences, needs, and experiences. The survey “Theory of Mind in LLMs: Assessment and Enhancement” makes a related point by distinguishing passive benchmarks from active settings, emphasizing “illusory ToM,” and arguing that useful ToM should be assessed through consistency across related questions, robustness across formats, broader mental-state coverage, and reasoning processes rather than answer-only correctness (Chen et al., 26 Apr 2025).

A plausible implication is that ToM-U should not be reduced to a single benchmark score. The literature repeatedly separates belief attribution, action prediction, explanation fidelity, dynamic context tracking, profile modeling, and applied social judgment.

6. Limitations, controversies, and future directions

The literature is unusually explicit about its own limitations. One recurring controversy is whether current systems are genuinely reasoning about minds or merely exploiting benchmark regularities. “Does ChatGPT have Theory of Mind?” reports that ChatGPT-4 answers 224 of 270 questions correctly, or 83.0%, with $E$0, but also emphasizes prompt sensitivity, stochasticity, and cases in which correct answers are reached on the basis of false assumptions or invalid reasoning (Holterman et al., 2023). The paper’s strongest caution is that correct outputs can be produced for the wrong reasons, which constrains any direct identification of benchmark success with ToM-U.

A second limitation is scale and ecological validity. The active-inference cooperation framework is demonstrated only in dyadic, small discrete grid worlds with deterministic dynamics, short planning horizons, known or assumed preferences, and first-order ToM only (Pitliya et al., 1 Aug 2025). The adaptive-teaching framework assumes a discrete learner-type space and a fixed candidate demonstration set (Grislain et al., 2023). HRI work is conceptually rich but weak on fidelity and coverage (Bauer et al., 29 Dec 2025). Benchmarks such as PersuasiveToM and UniToMBench remain largely multiple-choice and offline (Yu et al., 28 Feb 2025, Thiyagarajan et al., 11 Jun 2025). These limits do not nullify ToM-U, but they constrain claims about robustness and generality.

A third issue is whether ToM should be always on. The causal framework for conflict argues strongly against that assumption. It treats ToM as selectively activated through a tractability pathway, a reasoning-depth pathway, and an enabling-cause pathway, with information asymmetry as the enabling cause and epistemic accuracy as the primary outcome (Gurney, 15 Jun 2026). This suggests a future synthesis in which ToM-U is not just the value of mentalizing, but also the value of deciding when not to mentalize.

The forward direction of the field is therefore twofold. One direction seeks richer formalization: explicit epistemic state inference via LEWMs, residue, bounded proliferation, and candidate evaluation (Gurney et al., 10 Jun 2026). The other seeks broader validation: multi-turn interaction, unknown goals, multiple ToM agents, richer environments, process-level evaluation, and stronger links between inferred mental states and downstream success (Ruan et al., 14 Feb 2026, Wang et al., 15 Apr 2025). A cautious general conclusion is warranted. Explicit ToM-style modeling can improve cooperation, teaching, explanation, and user-agent interaction when outcomes depend on hidden beliefs and informational asymmetries. What remains unresolved is how far that utility scales beyond bounded tasks, prompt scaffolds, and toy social worlds.