Expert Mental Models (EMM) Overview

• Expert Mental Models (EMMs) are structured, computable representations capturing tacit expertise and reasoning to support analysis and intervention.
• EMM frameworks range from sequential explanation models based on observable user understanding to abstract decompositions of knowledge and skills with layered architectures.
• Practical implementations of EMMs include systems for interactive learning, bias management, and clinical evaluations that improve interpretability and decision-making processes.

Searching arXiv for the cited EMM-related papers to ground the article in published sources. Searching for "Sequential Explanations with Mental Model-Based Policies" and related EMM formulations. Searching for "The Expertise Level" and expert cognition frameworks connected to expert mental models. Searching for recent expert-cognition and EMM-related architectures. In current research usage, an Expert Mental Model (EMM) is not a single standardized construct. Depending on the problem setting, it can denote the explainee’s current mental model as an observable state for sequential explanation selection, an abstract Model of Expertise that separates what an expert knows from what an expert can do, a structured representation of expert procedural, semantic, and decision knowledge for AI-mediated learning, or an externalized substrate of tacit reasoning used for retrieval, evaluation, or prompt engineering (Yeung et al., 2020, Fulbright, 2022, Yuan et al., 2 May 2026). Across these formulations, the common objective is to make expertise, understanding, or judgment sufficiently explicit that it can be analyzed, supported, compared, or operationalized computationally.

1. Conceptual scope and core definitions

In the sequential-explanation literature, the relevant “mental model” is the explainee’s current understanding of a black-box model at time $t$. It is defined as “any observations of [the explainee’s] interpretation of the black-box model,” and in practice is represented through measurable proxies rather than an explicit hidden theory in the user’s head. The operational state includes satisfaction with the prior explanation and local simulatability of the model’s behavior on each classification possibility (Yeung et al., 2020).

In the expertise-modeling literature, by contrast, an EMM is an abstract representation of expertise that combines Knowledge Level and Expertise Level descriptions. The Knowledge Level describes what an expert knows; the Expertise Level, introduced as a level above Newell’s Knowledge Level, describes what an expert does. The resulting Model of Expertise is explicitly implementation-independent and treats expertise as the interaction of knowledge stores and skills rather than as either one in isolation (Fulbright, 2022).

In AI-for-learning formulations, the EMM becomes a computable representation of expert cognition. The AI Expert Twin models expertise through three layers—procedural actions, semantic concepts, and decision processes—with explicit attention to heuristics, cues, trade-offs, uncertainty, and value-laden preferences. This representation is intended to be transparent and pedagogically usable rather than merely predictive (Yuan et al., 2 May 2026).

Research use	What EMM denotes	Representative papers
Sequential explanation	Observable user-understanding state	(Yeung et al., 2020)
Abstract expertise theory	Combined knowledge-and-skill model	(Fulbright, 2022)
Practice-based learning	Computable expert cognition model	(Yuan et al., 2 May 2026)
Bias management	Externalized and interruptible expert reasoning	(Whitehead et al., 2022)
Knowledge preservation	Queryable expert cognition artifacts	(Cervera, 15 Mar 2026)

This diversity suggests that “EMM” functions as an umbrella term for formalizations of expertise or understanding that are sufficiently structured to support intervention. The unifying theme is not a shared ontology, but a shared move from tacit cognition toward inspectable representation.

2. Representational architectures and formal structures

The most explicit abstract decomposition appears in the Model of Expertise, which combines 12 fundamental skills—recall, apply, evaluate, understand, analyze, create, extract, teach, perceive, learn, alter, and act—with 14 knowledge stores, including generic knowledge, domain-specific knowledge, common-sense knowledge, episodic knowledge, generic and domain-specific models, generic and domain-specific task models, generic and domain-specific problem-solving models, goals, utility values, actions, and perceived environmental states (Fulbright, 2022). The paper also treats expert behaviors as composite processes; for example, justify is given as recall + analyze + understand, while predict is recall + analyze + evaluate + apply.

The AI Expert Twin adopts a different but related three-layer representation. Its procedural layer captures task decomposition, workflows, ordered actions, dependencies, and variations in practice; its semantic layer captures concepts, entities, materials, tools, constraints, and relationships; and its decision layer captures rules, heuristics, cues, trade-offs, context-sensitive choice, and uncertainty. The paper introduces a formal modulation device, the personality/tension vector

$T = \{R, I, G, E, C\}$

where $R$ is risk preference, $I$ innovation drive, $G$ rigor, $E$ empathy or human focus, and $C$ orientation toward tradition or change (Yuan et al., 2 May 2026).

A more explicitly cognitive and interpretive account appears in the Expert Identity Cognition Model (EICM), which defines expert cognition as “an identity-structured process operating within situational constraints, in which internal tensions between identity commitments are resolved into stable value structures that guide action.” EICM formalizes three layers—Constraint, Tension, and Value—with the chain

$T = f(C, I), \quad V = g(T), \quad A = h(V),$

and therefore

$A = h(g(f(C, I))).$

In this framework, tension is the central cognitive mechanism connecting situational structure to judgment formation, and values are treated as stabilized decision structures rather than reward functions or static preferences (Yuan, 12 May 2026).

These frameworks differ in granularity and purpose, but they converge on a layered view of expert cognition. Knowledge, action, interpretation, and value are repeatedly separated analytically and then re-linked as interacting levels. A plausible implication is that recent EMM research increasingly treats expertise as structured mediation between world state and action, rather than as mere outcome regularity.

3. Sequential, interactive, and team-based EMMs

The sequential-explanation framework of "Sequential Explanations with Mental Model-Based Policies" models explanation as a feedback loop in which the explainer selects the next explanation based on the explainee’s current state. The state is $s_t$, the policy is $T = \{R, I, G, E, C\}$0, and the next explanation is chosen by

$T = \{R, I, G, E, C\}$1

The reward is an interpretability proxy,

$T = \{R, I, G, E, C\}$2

and the experiments use a greedy immediate-reward setting with $T = \{R, I, G, E, C\}$3 (Yeung et al., 2020). The empirical implementation used online human-in-the-loop experiments on Amazon Mechanical Turk, a binary classification task on Kuzushiji-49, a CNN trained for 300 epochs with Adam and binary cross-entropy, and a balanced-test-set accuracy of 0.85. Participants completed a baseline iteration and then five experimental iterations; each experimental iteration involved explanation selection, satisfaction rating, and local simulatability tasks over 12 images comprising 3 instances each of true positives, true negatives, false positives, and false negatives. Across 488 participants who participated fully or partially across the three experiments, mental model-based policies generally improved simulatability more than random selection baselines, especially in the prototype and combined explanations conditions, while the saliency map policy was not much different from the random baseline (Yeung et al., 2020).

A different interactive elicitation strategy appears in Thought Bubbles, which embeds repeated open-ended prompts into gameplay as a diegetic mechanism for eliciting evolving mental models. The method is analyzed through Situation Awareness (SA) levels—perception, comprehension, and projection—and applied in two supply-chain studies involving 250 participants total. Study 1 used 115 logistics and supply-chain students across six conditions defined by disruption location and information sharing; Study 2 used 135 Prolific participants and examined Hoarders, Reactors, and Followers. The reported associations include disruption-location effects on SA distributions, information-sharing effects, and profile-specific differences, such as the finding that Hoarders showed more uncertainty-driven projection and more proactive stockpiling when information was available (Mohaddesi et al., 2023).

At the team level, "Are you with me?" shifts from individual cognition to discrepancy detection in dialogue. The framework defines four discrepancy types—unsupported beliefs, false beliefs, belief contradictions, and omissions—and uses transcribed dialogue plus task context to infer mental-model misalignment in real time. In 20 dyad teams performing collaborative object identification across four sequential levels, the study reports 1,447 discrepancies total, with team totals ranging from 44 to 176. For Level 4 prediction, averaging historical discrepancy counts yielded a correlation of $T = \{R, I, G, E, C\}$4 with $T = \{R, I, G, E, C\}$5 under a uniform-weighting exploratory baseline. The paper is explicit that dialogue-derived signals are proxies rather than the mental model itself, but argues that they provide finer temporal resolution than retrospective shared-mental-model assessment (Kowalyshyn et al., 4 May 2026).

Taken together, these studies treat EMMs as dynamic state representations rather than static knowledge snapshots. In one case the state is directly consumed by a policy; in another it is elicited through open-ended self-report; in a third it is inferred from communicative discrepancies. This suggests that a major current trajectory is from post-hoc description toward temporally local monitoring of understanding.

4. Externalizing, preserving, and engineering expert cognition

One research line treats EMMs as objects that should be made visible, inspectable, and interruptible in organizational workflows. The Modular Interface for Cognitive bias in Experts (MICE) was proposed in geoscience precisely because expert judgment is indispensable yet vulnerable to anchoring and adjustment, availability, confirmation, overconfidence, representativeness, herding, and single-expert over-dominance. MICE is organized into four module types—monitoring, output, feedback, and action—and centers on action modules that force a stop-and-perform step into interpretation tasks. Its starter-pack includes training modules and tool modules such as pre-mortem, slow-down breaks, ask-again-later, seek advice or knowledge, devil’s advocate, visualisation, explicit knowledge elicitation, expert profiling, risk attitude profiling, deconstruct task, reword task, and forced anonymity (Whitehead et al., 2022).

A preservation-oriented formulation appears in Expert Mind, a retrieval-augmented architecture for the energy sector. It is described as a four-layer architecture: multimodal knowledge capture, LLM-driven processing and indexing, vector-store persistence, and conversational query interface. Expert cognition is elicited through 60–90 minute structured interviews guided by cognitive task analysis, think-aloud sessions on real or simulated operational problems, and text corpus ingestion from technical reports, email correspondence, annotated procedures, and internal knowledge-base entries. Audio/video is transcribed with OpenAI Whisper; artifacts are extracted with Claude API into four types—factual claims, decision criteria and heuristics, anomaly recognition patterns, and best practices and lessons learned—then validated by the originating expert, embedded with a dense text encoder, and stored in Pinecone or ChromaDB with provenance metadata (Cervera, 15 Mar 2026). The system is explicitly governed by written informed consent, knowledge-licensing considerations, provenance metadata, and a complete erasure pathway consistent with GDPR.

Another line uses EMM not as storage but as a formal decision scaffold for prompt engineering. "LLM Enhancement with Domain Expert Mental Model to Reduce LLM Hallucination with Causal Prompt Engineering" defines an expert mental model $T = \{R, I, G, E, C\}$6 for expert $T = \{R, I, G, E, C\}$7 and scenario $T = \{R, I, G, E, C\}$8, with binary or ordinal $T = \{R, I, G, E, C\}$9-valued outputs. Its four-step algorithm consists of factor identification, hierarchical structuring of factors, generating a generalized expert mental model specification, and generating a detailed generalized expert mental model from that specification. The paper emphasizes monotone Boolean and ordinal $R$0-valued functions as mechanisms for pruning expert-query complexity and illustrates the approach with a call-for-proposals decision problem, where the root question “Should we respond to the RFP?” is decomposed into a tree of branch and leaf factors (Kovalerchuk et al., 13 Sep 2025).

These works share a commitment to externalization. In MICE, the goal is bias management through procedural friction; in Expert Mind, it is preservation of tacit judgment as retrievable artifacts; in EMM-based prompt engineering, it is tractable formalization of tacit decision logic. In all three, the EMM is valuable precisely because expertise is otherwise too private, too perishable, or too implicit for reliable organizational reuse.

5. Clinical and mental-health instantiations

Mental-health AI has become a major domain in which EMM-like constructs are operationalized. One approach grounds inference directly in expert symptom descriptions. The Multi-Head Siamese network (MHS) for mental disorder detection models each disorder as a set of symptom heads derived from DSM-5 diagnostic criteria and clinical self-test questionnaires, with questions mapped to symptom criteria under a psychology researcher’s guidance. Each head outputs a scalar similarity score $R$1 between user text and symptom descriptions, and the stacked vector $R$2 is passed to a final classifier. The strongest reported results are for RoBERTa-MHS: MDD F1 89.6, AUC 93.8; Bipolar F1 90.4, AUC 93.4; GAD F1 91.5, AUC 94.3; BPD F1 90.8, AUC 94.0. The paper argues that interpretability follows from symptom-level heads, symptom scores, and final-layer weights tied to named clinical concepts (Song et al., 2023).

A second approach uses multi-expert aggregation rather than symptom matching. Stacked Multi-Model Reasoning (SMMR) is a layered framework in which early layers consist of coequal initial experts, middle layers refine aggregated outputs, and a final long-context model consolidates the result. The formulation is explicitly staged,

$R$3

followed by iterative refinement and final consolidation. On DAIC-WOZ, which the paper describes as 187 labeled interviews, SMMR improved multiple metrics over single-model baselines. For GPT-3.5-turbo on the test set, Accuracy improved from 0.55 to 0.76, F1 from 0.57 to 0.70, MAE from 6.04 to 4.22, and RMSE from 6.81 to 5.54 (Tang et al., 20 Jan 2025).

A third approach emphasizes behavior alignment for clinician support. coTherapist, built on LLaMA 3.2-1B-Instruct, combines domain-adaptive pretraining, LoRA style tuning, self-instruction tuning, RAG with top-$R$4 retrieval via FAISS, and an agentic loop with Planner, Retriever, Reasoner, Critic / Self-refiner, and Finalizer. Evaluation includes automatic NLG metrics, the T-BARS / COTHERF rubric, psychometric profiling, and blind human assessment by 5 licensed clinical psychologists and 15 clinical psychology students. Reported T-BARS composite scores are 1.6 for the base model, 2.5 for TG-RAG, 3.2 for coTherapist, and 3.5 for the human average; clinicians reportedly described coTherapist as resembling a “well-trained trainee” (Adhikary et al., 15 Jan 2026).

Clinical evaluation itself has also been formalized as an expert mental model. CounselBench uses 100 licensed or professionally trained mental health practitioners in the U.S. to rate responses along six clinically grounded dimensions—Overall Quality, Empathy, Specificity, Medical Advice, Factual Consistency, and Toxicity—with written rationales and span-level annotations. In CounselBench-EVAL, 2,000 expert evaluations were collected; in CounselBench-ADV, 120 clinician-authored adversarial questions were used to generate 2,880 QA pairs across eight LLMs. The paper reports that LLaMA-3.3 led on five of six dimensions, yet 14% of its outputs were flagged for unauthorized medical advice; it also reports that LLM judges systematically overrate responses relative to human experts and often miss precisely the safety issues experts identify (Li et al., 10 Jun 2025).

Finally, Chain of Alignment uses expert intelligence not to define values but to translate public normative objectives into operational rules. In three mental-health prompt domains, the public objectives reached 96%–98% ± 2% overall support, while expert-derived rules yielded a rule-based reward whose alignment with expert judgments reached Pearson’s $R$5 and AUC = 0.964. Here the EMM-like object is a set of expert rules such as “Always immediately acknowledge the user’s distress” or “Never tell a user that a recommendation ‘will’ improve their symptom(s),” which function as a compressed model of domain-appropriate behavior (Konya et al., 2024).

Across these clinical systems, EMMs appear in at least four forms: symptom schemata, layered panels of model “second opinions,” therapist-behavior rubrics, and expert-derived safety rules. The commonality is not architecture but clinical grounding: each system attempts to anchor computation in structured expert judgment rather than unconstrained text generation.

6. Limits, controversies, and research implications

A persistent limitation is that many EMM formulations remain conceptual, proxy-based, or implementation-independent. The Expertise Level framework explicitly does not specify algorithmic or neural implementation, and the EICM paper presents a theoretical and illustrative rather than empirically validated model (Fulbright, 2022, Yuan, 12 May 2026). The AI Expert Twin case study in a jade carving cultural heritage workshop demonstrates feasibility of collection, segmentation, and annotation, but is explicit that it is not yet a completed Expert Twin and not yet evaluated with learners (Yuan et al., 2 May 2026).

Even empirical systems often access EMMs only indirectly. In sequential explanation, the mental model is operationalized through satisfaction and simulatability proxies rather than latent-model estimation (Yeung et al., 2020). In team dialogue, discrepancy patterns are derived from speech and task context; the paper explicitly cautions that dialogue is a proxy, not a direct measure, and that false-belief detection depends on the availability of authoritative ground truth (Kowalyshyn et al., 4 May 2026). In Thought Bubbles, free-text responses are coded through Situation Awareness categories, again treating expressed cognition as evidence rather than identity (Mohaddesi et al., 2023).

Another recurrent issue is narrowness. The Expertise Level paper emphasizes that many artificial systems exhibit only narrow or weak expertise relative to human expertise (Fulbright, 2022). coTherapist is framed as a support tool, not a replacement expert, and is limited by being unlicensed, English-only, and focused on mainstream modalities (Adhikary et al., 15 Jan 2026). Expert Mind proposes performance targets such as response accuracy: >85%, weekly query volume: >50 queries/week, and onboarding reduction: >20%, but these are preliminary criteria rather than demonstrated outcomes (Cervera, 15 Mar 2026).

A further controversy concerns what counts as the “real” expert mental model. Some works identify it with a user state, some with an abstract skill-and-knowledge decomposition, some with tacit reasoning artifacts, some with value-sensitive decision structures, and some with the criteria experts use to judge model outputs. This suggests that EMM research is currently organized less by a settled definition than by a family of representational strategies for making expertise usable. The practical consequence is productive but also methodologically difficult: comparisons across EMM systems are often cross-paradigm rather than like-for-like.

The broader implication is that EMM research has moved from static expertise encoding toward state-aware explanation, inspectable tacit-knowledge capture, expert-grounded clinical evaluation, and value-sensitive models of judgment. What remains unresolved is whether these should ultimately be unified into a common representational framework, or whether “Expert Mental Model” will remain a domain-bridging label for multiple, only partially commensurable, approaches to formalizing expert cognition.