The Theory of Mind Utility: Formal Specification of a Mentalizing Mechanism

Abstract: Inferring others' beliefs requires more than reading surface signals; it requires tracking who told them what, in what order, and how credibly. The Theory of Mind Utility (ToM-U) formalizes this epistemic state inference problem at the computational level of analysis, specifying what mentalizing computes and why without commitment to algorithmic or neural implementation. ToM-U achieves this by constructing Local Epistemic World Models (LEWMs) -- directed typed graphs that represent agents, state nodes, and the epistemic relationships among them -- and evaluating discrete candidate LEWMs against observed behavior until one achieves sufficient confidence. Five formal definitions specify the LEWM structure, agent node properties including ordered information access history, a bounded proliferation mechanism for recursive mentalizing, three inference procedures, and a residue function that captures the structured trace left by failed mentalizing attempts. ToM-U differs from Bayesian Theory of Mind and adjacent formal accounts, which presuppose rather than derive belief states, and from simulation theory and theory-theory, which lack a formal apparatus for epistemic state inference. The architecture generates directional, falsifiable predictions about mentalizing failure that follow from structural properties of the model rather than auxiliary assumptions, and positions ToM-U as a domain-agnostic mechanism upstream of goal inference and other downstream social cognitive processes.

• The paper presents a formal specification that constructs belief-like states using ordered information and source credibility via Local Epistemic World Models (LEWMs).
• The paper details three inference procedures—backward inference, self-projection, and mutual reconciliation—to derive agents’ mental states without presupposing them.
• The paper introduces a residue mechanism that adjusts future inferences based on earlier mentalizing failures, providing testable predictions about ToM deficits.

The Theory of Mind Utility: Formal Specification of a Mentalizing Mechanism

Overview and Motivation

This paper introduces the Theory of Mind Utility (ToM-U), a computational-level formalization of mentalizing—specifically, the process by which agents infer the epistemic states of others from information access histories, source credibility, and observability constraints. Unlike Bayesian Theory of Mind (BToM), simulation theory, or theory-theory accounts—which either presuppose epistemic states or lack formal apparatus for their derivation—ToM-U specifies how belief-like states (BLSs) are constructed via Local Epistemic World Models (LEWMs). ToM-U characterizes mentalizing as a structured, domain-agnostic mechanism, independent of algorithmic or neural implementation, and proposes five formally defined components governing the representation and inference of other agents' mental states.

The conceptual advance is the explicit modeling of epistemic provenance—that is, capturing not just what information an agent has, but when and from whom it was acquired, as well as how credible and observable each piece is. This is operationalized through LEWMs, directed typed graphs representing the epistemic scenario from the focal agent's perspective, recursively proliferated to enable bounded-depth reasoning about nested mental states.

Formal Specification: Components and Mechanisms

Local Epistemic World Models (LEWMs)

LEWMs serve as the formal substrate for mentalizing within ToM-U. Each LEWM is a directed typed graph comprising agent nodes, state nodes, and epistemic edges characterized by BLSs, observability ($\text{obs}$), and credibility ($\text{cred}$) functions. Agent nodes possess an information access history ($H_i$), a bounded sophistication parameter ($S_i$), and a snapshot timestamp. Epistemic edges encode specific relationships (e.g., belief, trust) and are structurally distinguished such that, for example, absent and false beliefs yield different graph properties.

Bounded Proliferation

To accommodate recursive mentalizing (e.g., "I think that you know that I know..."), ToM-U employs a proliferation mechanism generating a bounded branching tree of LEWMs, with each node at depth $k$ treated as a constructed other—circumventing infinite regress and avoiding the homuncular fallacy. The depth of this tree is capped by the sophistication parameter $S_1$ of the focal agent; proliferation halts when either the tree's marginal coherence gain becomes negligible or the ceiling is reached.

Inference Procedures

ToM-U defines three domain-general inference operations:

1. Backward Inference: Attributing belief states to a target agent by generating and evaluating candidate LEWMs against the target's observed behavior, conditional on reconstructed information access history and observability. Stopping can occur on either exceeding a confidence threshold or exhausting the sophistication limit.
2. Self-Projection: Projecting the focal agent's own state as it appears from another's perspective (a degenerate 2-node case), incorporating anchoring and adjustment dynamics consistent with empirical data on egocentrism in ToM reasoning.
3. Mutual Reconciliation: An iterative, multi-agent extension seeking a LEWM that coherently accounts for the observed behaviors and mutually inferred states of all agents, with a joint confidence measure governed by the least-supported inference direction.

These procedures are generate-and-filter rather than distributional: candidate LEWMs are enumerated, evaluated for fit, and accepted or rejected discretely.

Residue Mechanism

A key architectural innovation is the residue function. When LEWM-based inference fails (i.e., when no candidate achieves sufficient confidence and the output is rejected), a trace is left on relevant epistemic edges. This residue dynamically attenuates $\text{cred}$ on implicated relationships in future invocations, modulated by degree of disconfirmation and time decay, with partial rehabilitation possible via new evidence. Thus, failed mentalizing attempts are not neutral but shape subsequent inference structurally—a property not addressed by previous formal accounts.

Empirical and Theoretical Implications

ToM-U’s core claim is that belief states must be derived, not presupposed, from historically ordered, credence-weighted information. This contrasts with BToM’s assumption that belief states are given inputs to an inverse-planning model. Adjacent approaches (DEL, AGM frameworks) similarly treat epistemic states as primitive, lacking machinery for their derivation or for handling inference failure.

The formal specification yields strong predictions and claims:

• Distinct LEWM failures: Absence of an edge (no exposure) and presence of an incorrectly valued edge (false belief) are structurally (and empirically) dissociable, leading to specific, testable predictions about error patterns in ToM reasoning.
• Directionality of ToM deficits: Systematic errors in reconstructing $H_i$ or source credibility will yield directionally biased, not random, mentalizing failures.
• Residue-sensitive learning: Agents should exhibit a history-dependent resistance to belief updating over relationships implicated in prior inference failures, even in the face of new evidence—a prediction inaccessible to snapshot Bayesian or DEL-based models.
• Sophistication-bounded nesting: Over- or underestimation of a target's sophistication parameter leads to miscalibrated recursive reasoning; individual and cultural variation in $S_i$ predicts ToM error patterns.
• Separation of certainty and content errors: Errors from poor observability (low $\text{obs}$) do not conflate with content inference errors. These are structurally separated in the model.

These commitments are empirically falsifiable; for instance, evidence that belief attribution is invariant to order or source credibility would undermine ToM-U's foundations.

Extensions and Limitations

ToM-U is intentionally specified at Marr's computational level, abstracting over implementation details:

• The choice of fit scores in the generate-and-filter architecture and the detailed functional form of residue scheduling are deferred to further algorithmic-level work.
• LEWMs handle only the epistemic inference subproblem; downstream goal inference, action prediction, or behavioral generation are assumed to consume but not shape this output directly.
• There is no treatment of non-veridicality in $\text{cred}$0 or explicit modeling of reconstructive memory, although extensions are suggested.
• The architecture is parameterized for further cross-cultural and context-sensitive calibration (e.g., varying sophistication parameter $\text{cred}$1, threshold levels), as ToM development is empirically heterogeneous across populations.
• The model can be enriched by introducing multi-typed epistemic edges, veridicality scores for memory, and more complex handling of collective sensemaking or clinical ToM impairments.

Worked Example: The Popcorn Bag Task

The paper demonstrates ToM-U's machinery with variants of the unexpected contents ("popcorn bag") task. Ordered information access history ($\text{cred}$2: "friend's testimony, then bag label") is critical: only if the trusted friend's input precedes the label and is highly credible does the focal agent infer that Sam's belief diverges from appearance (i.e., she believes "popcorn", not "chocolate"). LLMs and naive ToM models that neglect the ordering or omit trust relationships systematically fail this attribution, highlighting the distinction between ToM-U and models that rely on surface cues.

Position in the Theoretical Landscape

ToM-U bridges gaps left by established cognitive and formal models. Its explicit separation of computational, algorithmic, and implementational concerns aligns with the rational analysis tradition, while adopting explicit graph-theoretic apparatus for belief modeling. The bounded proliferation mechanism avoids infinite regress without unsatisfying recursion-stopping loopholes, and the residue function provides a formal account for learning from mentalizing failures—a feature relevant for domains ranging from clinical psychology to strategic AI interaction.

The model’s generate-and-filter approach also resonates with abductive inference frameworks, emphasizing hypothesis generation and coherence evaluation over full probabilistic reasoning. It therefore challenges probabilistic/Bayesian orthodoxy—providing fertile ground for further theoretical debate and empirical scrutiny.

Conclusion

The Theory of Mind Utility provides a rigorous, formal structure for the epistemic inference problem underlying mentalizing. By specifying the construction and evaluation of LEWMs, operationalizing order and credibility of inputs, and formalizing both the content and aftermath of mentalizing failures, ToM-U advances the formal landscape of human and artificial ToM modeling. Its architecture makes strong, testable predictions about the structure and content of ToM failures and provides principled inputs to downstream social-cognitive systems. Future work will refine the model’s algorithmic realization, extend its applicability to culturally or developmentally diverse populations, and connect its predictions to empirical data in both human and artificial agents.