---
title: Expert Mental Models (EMM) Overview
url: https://www.emergentmind.com/topics/expert-mental-model-emm
type: topic
---

# Expert Mental Models (EMM) Overview

Searching arXiv for the cited EMM-related papers to ground the article in published sources.
Searching for "Sequential Explanations with Mental Model-Based Policies" and related EMM formulations.
Searching for "The Expertise Level" and expert cognition frameworks connected to expert mental models.
Searching for recent expert-cognition and EMM-related architectures.
In current research usage, an **Expert Mental Model (EMM)** is not a single standardized construct. Depending on the problem setting, it can denote the explainee’s **current mental model** as an observable state for sequential explanation selection, an abstract **Model of Expertise** that separates what an expert knows from what an expert can do, a structured representation of expert procedural, semantic, and decision knowledge for AI-mediated learning, or an externalized substrate of tacit reasoning used for retrieval, evaluation, or prompt engineering [2007.09028] [2212.10435] [2605.01401]. Across these formulations, the common objective is to make expertise, understanding, or judgment sufficiently explicit that it can be analyzed, supported, compared, or operationalized computationally.

## 1. Conceptual scope and core definitions

In the sequential-explanation literature, the relevant “mental model” is the explainee’s **current understanding** of a black-box model at time $t$. It is defined as “any observations of [the explainee’s] interpretation of the black-box model,” and in practice is represented through measurable proxies rather than an explicit hidden theory in the user’s head. The operational state includes **satisfaction** with the prior explanation and **local simulatability** of the model’s behavior on each classification possibility [2007.09028].

In the expertise-modeling literature, by contrast, an EMM is an abstract representation of expertise that combines **Knowledge Level** and **Expertise Level** descriptions. The Knowledge Level describes what an expert knows; the Expertise Level, introduced as a level above Newell’s Knowledge Level, describes what an expert does. The resulting **Model of Expertise** is explicitly implementation-independent and treats expertise as the interaction of knowledge stores and skills rather than as either one in isolation [2212.10435].

In AI-for-learning formulations, the EMM becomes a **computable representation of expert cognition**. The AI Expert Twin models expertise through three layers—**procedural actions**, **semantic concepts**, and **decision processes**—with explicit attention to heuristics, cues, trade-offs, uncertainty, and value-laden preferences. This representation is intended to be transparent and pedagogically usable rather than merely predictive [2605.01401].

| Research use | What EMM denotes | Representative papers |
|---|---|---|
| Sequential explanation | Observable user-understanding state | [2007.09028] |
| Abstract expertise theory | Combined knowledge-and-skill model | [2212.10435] |
| Practice-based learning | Computable expert cognition model | [2605.01401] |
| Bias management | Externalized and interruptible expert reasoning | [2208.05996] |
| Knowledge preservation | Queryable expert cognition artifacts | [2603.14541] |

This diversity suggests that “EMM” functions as an umbrella term for formalizations of expertise or understanding that are sufficiently structured to support intervention. The unifying theme is not a shared ontology, but a shared move from tacit cognition toward inspectable representation.

## 2. Representational architectures and formal structures

The most explicit abstract decomposition appears in the **Model of Expertise**, which combines **12 fundamental skills**—recall, apply, evaluate, understand, analyze, create, extract, teach, perceive, learn, alter, and act—with **14 knowledge stores**, including generic knowledge, domain-specific knowledge, common-sense knowledge, episodic knowledge, generic and domain-specific models, generic and domain-specific task models, generic and domain-specific problem-solving models, goals, utility values, actions, and perceived environmental states [2212.10435]. The paper also treats expert behaviors as composite processes; for example, **justify** is given as recall + analyze + understand, while **predict** is recall + analyze + evaluate + apply.

The AI Expert Twin adopts a different but related three-layer representation. Its **procedural layer** captures task decomposition, workflows, ordered actions, dependencies, and variations in practice; its **semantic layer** captures concepts, entities, materials, tools, constraints, and relationships; and its **decision layer** captures rules, heuristics, cues, trade-offs, context-sensitive choice, and uncertainty. The paper introduces a formal modulation device, the **personality/tension vector**  
$$
T = \{R, I, G, E, C\}
$$
where $R$ is risk preference, $I$ innovation drive, $G$ rigor, $E$ empathy or human focus, and $C$ orientation toward tradition or change [2605.01401].

A more explicitly cognitive and interpretive account appears in the **Expert Identity Cognition Model (EICM)**, which defines expert cognition as “an identity-structured process operating within situational constraints, in which internal tensions between identity commitments are resolved into stable value structures that guide action.” EICM formalizes three layers—**Constraint**, **Tension**, and **Value**—with the chain  
$$
T = f(C, I), \quad V = g(T), \quad A = h(V),
$$
and therefore  
$$
A = h(g(f(C, I))).
$$
In this framework, tension is the central cognitive mechanism connecting situational structure to judgment formation, and values are treated as stabilized decision structures rather than reward functions or static preferences [2605.11393].

These frameworks differ in granularity and purpose, but they converge on a layered view of expert cognition. Knowledge, action, interpretation, and value are repeatedly separated analytically and then re-linked as interacting levels. A plausible implication is that recent EMM research increasingly treats expertise as structured mediation between world state and action, rather than as mere outcome regularity.

## 3. Sequential, interactive, and team-based EMMs

The sequential-explanation framework of "Sequential Explanations with Mental Model-Based Policies" models explanation as a feedback loop in which the explainer selects the next explanation based on the explainee’s current state. The state is $s_t$, the policy is $\mu$, and the next explanation is chosen by  
$$
a_t \leftarrow \mu_{\theta}(s_t).
$$
The reward is an interpretability proxy,
$$
r_{t+1}(s_t, a_t) \gets p(f(x), a_t\,|\,s_t),
$$
and the experiments use a greedy immediate-reward setting with $\gamma = 0$ [2007.09028]. The empirical implementation used online human-in-the-loop experiments on Amazon Mechanical Turk, a binary classification task on **Kuzushiji-49**, a CNN trained for 300 epochs with Adam and binary cross-entropy, and a balanced-test-set accuracy of **0.85**. Participants completed a baseline iteration and then **five experimental iterations**; each experimental iteration involved explanation selection, **satisfaction** rating, and **local simulatability** tasks over **12 images** comprising **3 instances each** of true positives, true negatives, false positives, and false negatives. Across **488 participants** who participated fully or partially across the three experiments, mental model-based policies generally improved simulatability more than random selection baselines, especially in the **prototype** and **combined explanations** conditions, while the **saliency map** policy was not much different from the random baseline [2007.09028].

A different interactive elicitation strategy appears in **Thought Bubbles**, which embeds repeated open-ended prompts into gameplay as a diegetic mechanism for eliciting evolving mental models. The method is analyzed through **Situation Awareness (SA)** levels—**perception**, **comprehension**, and **projection**—and applied in two supply-chain studies involving **250 participants** total. Study 1 used **115** logistics and supply-chain students across six conditions defined by disruption location and information sharing; Study 2 used **135** Prolific participants and examined **Hoarders**, **Reactors**, and **Followers**. The reported associations include disruption-location effects on SA distributions, information-sharing effects, and profile-specific differences, such as the finding that **Hoarders** showed more uncertainty-driven projection and more proactive stockpiling when information was available [2301.13101].

At the team level, "Are you with me?" shifts from individual cognition to discrepancy detection in dialogue. The framework defines four discrepancy types—**unsupported beliefs**, **false beliefs**, **belief contradictions**, and **omissions**—and uses transcribed dialogue plus task context to infer mental-model misalignment in real time. In **20 dyad teams** performing collaborative object identification across **four sequential levels**, the study reports **1,447 discrepancies total**, with team totals ranging from **44 to 176**. For Level 4 prediction, averaging historical discrepancy counts yielded a correlation of **$r = 0.56$** with **$p = 0.01$** under a uniform-weighting exploratory baseline. The paper is explicit that dialogue-derived signals are proxies rather than the mental model itself, but argues that they provide finer temporal resolution than retrospective shared-mental-model assessment [2605.03149].

Taken together, these studies treat EMMs as **dynamic state representations** rather than static knowledge snapshots. In one case the state is directly consumed by a policy; in another it is elicited through open-ended self-report; in a third it is inferred from communicative discrepancies. This suggests that a major current trajectory is from post-hoc description toward temporally local monitoring of understanding.

## 4. Externalizing, preserving, and engineering expert cognition

One research line treats EMMs as objects that should be made visible, inspectable, and interruptible in organizational workflows. The **Modular Interface for Cognitive bias in Experts (MICE)** was proposed in geoscience precisely because expert judgment is indispensable yet vulnerable to anchoring and adjustment, availability, confirmation, overconfidence, representativeness, herding, and single-expert over-dominance. MICE is organized into four module types—**monitoring**, **output**, **feedback**, and **action**—and centers on **action modules** that force a stop-and-perform step into interpretation tasks. Its starter-pack includes training modules and tool modules such as pre-mortem, slow-down breaks, ask-again-later, seek advice or knowledge, devil’s advocate, visualisation, explicit knowledge elicitation, expert profiling, risk attitude profiling, deconstruct task, reword task, and forced anonymity [2208.05996].

A preservation-oriented formulation appears in **Expert Mind**, a retrieval-augmented architecture for the energy sector. It is described as a **four-layer architecture**: multimodal knowledge capture, LLM-driven processing and indexing, vector-store persistence, and conversational query interface. Expert cognition is elicited through **60–90 minute structured interviews** guided by **cognitive task analysis**, **think-aloud sessions** on real or simulated operational problems, and **text corpus ingestion** from technical reports, email correspondence, annotated procedures, and internal knowledge-base entries. Audio/video is transcribed with **OpenAI Whisper**; artifacts are extracted with **Claude API** into four types—**factual claims**, **decision criteria and heuristics**, **anomaly recognition patterns**, and **best practices and lessons learned**—then validated by the originating expert, embedded with a dense text encoder, and stored in **Pinecone** or **ChromaDB** with provenance metadata [2603.14541]. The system is explicitly governed by written informed consent, knowledge-licensing considerations, provenance metadata, and a complete erasure pathway consistent with GDPR.

Another line uses EMM not as storage but as a formal decision scaffold for prompt engineering. "LLM Enhancement with Domain Expert Mental Model to Reduce LLM Hallucination with Causal Prompt Engineering" defines an expert mental model $M(E,S)$ for expert $E$ and scenario $S$, with binary or ordinal $k$-valued outputs. Its four-step algorithm consists of **factor identification**, **hierarchical structuring of factors**, **generating a generalized expert mental model specification**, and **generating a detailed generalized expert mental model from that specification**. The paper emphasizes **monotone Boolean** and **ordinal $k$-valued functions** as mechanisms for pruning expert-query complexity and illustrates the approach with a call-for-proposals decision problem, where the root question “Should we respond to the RFP?” is decomposed into a tree of branch and leaf factors [2509.10818].

These works share a commitment to **externalization**. In MICE, the goal is bias management through procedural friction; in Expert Mind, it is preservation of tacit judgment as retrievable artifacts; in EMM-based prompt engineering, it is tractable formalization of tacit decision logic. In all three, the EMM is valuable precisely because expertise is otherwise too private, too perishable, or too implicit for reliable organizational reuse.

## 5. Clinical and mental-health instantiations

Mental-health AI has become a major domain in which EMM-like constructs are operationalized. One approach grounds inference directly in expert symptom descriptions. The **Multi-Head Siamese network (MHS)** for mental disorder detection models each disorder as a set of symptom heads derived from **DSM-5 diagnostic criteria** and **clinical self-test questionnaires**, with questions mapped to symptom criteria under a psychology researcher’s guidance. Each head outputs a scalar similarity score $d_i$ between user text and symptom descriptions, and the stacked vector $D \in \mathbb{R}^n$ is passed to a final classifier. The strongest reported results are for **RoBERTa-MHS**: **MDD** F1 **89.6**, AUC **93.8**; **Bipolar** F1 **90.4**, AUC **93.4**; **GAD** F1 **91.5**, AUC **94.3**; **BPD** F1 **90.8**, AUC **94.0**. The paper argues that interpretability follows from symptom-level heads, symptom scores, and final-layer weights tied to named clinical concepts [2306.02955].

A second approach uses multi-expert aggregation rather than symptom matching. **Stacked Multi-Model Reasoning (SMMR)** is a layered framework in which early layers consist of coequal initial experts, middle layers refine aggregated outputs, and a final long-context model consolidates the result. The formulation is explicitly staged,
$$
r_1^i \gets M_1^i(\mathcal{X}), \quad R_1 \gets \{r_1^1,\dots,r_1^n\},
$$
followed by iterative refinement and final consolidation. On **DAIC-WOZ**, which the paper describes as **187 labeled interviews**, SMMR improved multiple metrics over single-model baselines. For **GPT-3.5-turbo** on the test set, **Accuracy** improved from **0.55** to **0.76**, **F1** from **0.57** to **0.70**, **MAE** from **6.04** to **4.22**, and **RMSE** from **6.81** to **5.54** [2501.13951].

A third approach emphasizes behavior alignment for clinician support. **coTherapist**, built on **LLaMA 3.2-1B-Instruct**, combines domain-adaptive pretraining, LoRA style tuning, self-instruction tuning, RAG with **top-$k=3$** retrieval via **FAISS**, and an agentic loop with **Planner**, **Retriever**, **Reasoner**, **Critic / Self-refiner**, and **Finalizer**. Evaluation includes automatic NLG metrics, the **T-BARS / COTHERF** rubric, psychometric profiling, and blind human assessment by **5 licensed clinical psychologists** and **15 clinical psychology students**. Reported T-BARS composite scores are **1.6** for the base model, **2.5** for **TG-RAG**, **3.2** for **coTherapist**, and **3.5** for the human average; clinicians reportedly described coTherapist as resembling a **“well-trained trainee”** [2601.10246].

Clinical evaluation itself has also been formalized as an expert mental model. **CounselBench** uses **100 licensed or professionally trained mental health practitioners in the U.S.** to rate responses along six clinically grounded dimensions—**Overall Quality**, **Empathy**, **Specificity**, **Medical Advice**, **Factual Consistency**, and **Toxicity**—with written rationales and span-level annotations. In **CounselBench-EVAL**, **2,000 expert evaluations** were collected; in **CounselBench-ADV**, **120 clinician-authored adversarial questions** were used to generate **2,880 QA pairs** across eight LLMs. The paper reports that **LLaMA-3.3** led on five of six dimensions, yet **14%** of its outputs were flagged for unauthorized medical advice; it also reports that LLM judges systematically overrate responses relative to human experts and often miss precisely the safety issues experts identify [2506.08584].

Finally, **Chain of Alignment** uses expert intelligence not to define values but to translate public normative objectives into operational rules. In three mental-health prompt domains, the public objectives reached **96%–98% ± 2%** overall support, while expert-derived rules yielded a rule-based reward whose alignment with expert judgments reached **Pearson’s $r=0.841$** and **AUC = 0.964**. Here the EMM-like object is a set of expert rules such as “Always immediately acknowledge the user’s distress” or “Never tell a user that a recommendation ‘will’ improve their symptom(s),” which function as a compressed model of domain-appropriate behavior [2411.10534].

Across these clinical systems, EMMs appear in at least four forms: symptom schemata, layered panels of model “second opinions,” therapist-behavior rubrics, and expert-derived safety rules. The commonality is not architecture but **clinical grounding**: each system attempts to anchor computation in structured expert judgment rather than unconstrained text generation.

## 6. Limits, controversies, and research implications

A persistent limitation is that many EMM formulations remain **conceptual**, **proxy-based**, or **implementation-independent**. The Expertise Level framework explicitly does not specify algorithmic or neural implementation, and the EICM paper presents a theoretical and illustrative rather than empirically validated model [2212.10435] [2605.11393]. The AI Expert Twin case study in a **jade carving cultural heritage workshop** demonstrates feasibility of collection, segmentation, and annotation, but is explicit that it is **not yet a completed Expert Twin** and **not yet evaluated with learners** [2605.01401].

Even empirical systems often access EMMs only indirectly. In sequential explanation, the mental model is operationalized through satisfaction and simulatability proxies rather than latent-model estimation [2007.09028]. In team dialogue, discrepancy patterns are derived from speech and task context; the paper explicitly cautions that dialogue is a proxy, not a direct measure, and that false-belief detection depends on the availability of authoritative ground truth [2605.03149]. In Thought Bubbles, free-text responses are coded through Situation Awareness categories, again treating expressed cognition as evidence rather than identity [2301.13101].

Another recurrent issue is **narrowness**. The Expertise Level paper emphasizes that many artificial systems exhibit only narrow or weak expertise relative to human expertise [2212.10435]. coTherapist is framed as a support tool, not a replacement expert, and is limited by being **unlicensed**, **English-only**, and focused on **mainstream modalities** [2601.10246]. Expert Mind proposes performance targets such as **response accuracy: >85%**, **weekly query volume: >50 queries/week**, and **onboarding reduction: >20%**, but these are preliminary criteria rather than demonstrated outcomes [2603.14541].

A further controversy concerns what counts as the “real” expert mental model. Some works identify it with a user state, some with an abstract skill-and-knowledge decomposition, some with tacit reasoning artifacts, some with value-sensitive decision structures, and some with the criteria experts use to judge model outputs. This suggests that EMM research is currently organized less by a settled definition than by a family of representational strategies for making expertise usable. The practical consequence is productive but also methodologically difficult: comparisons across EMM systems are often cross-paradigm rather than like-for-like.

The broader implication is that EMM research has moved from static expertise encoding toward **state-aware explanation**, **inspectable tacit-knowledge capture**, **expert-grounded clinical evaluation**, and **value-sensitive models of judgment**. What remains unresolved is whether these should ultimately be unified into a common representational framework, or whether “Expert Mental Model” will remain a domain-bridging label for multiple, only partially commensurable, approaches to formalizing expert cognition.

Source: https://www.emergentmind.com/topics/expert-mental-model-emm