Dishonesty-Aware Psychiatric Assessment

• Dishonesty-aware psychiatric assessment is a computational framework that models latent honesty states to detect misreporting in mental health evaluations.
• It employs multi-agent architectures and adaptive interview protocols to dynamically adjust questioning based on contextual response inconsistencies.
• Empirical evaluations demonstrate high detection accuracy and improved diagnostic fusion, while highlighting future challenges like cultural bias and privacy safeguards.

Dishonesty-aware psychiatric assessment refers to computational and algorithmic strategies that explicitly account for the presence of misreporting—whether through concealment, exaggeration, or other intentional/unintentional distortions—when conducting mental health evaluations. The concept is motivated by the well-established challenges of unreliable self-reporting in clinical settings, where subjectivity, stigma, and privacy concerns frequently produce data artifacts that compromise both diagnosis and research fidelity. Recent developments leverage NLP, multi-agent simulation, and adaptive interview protocols to detect, model, and potentially mitigate dishonesty in digital psychiatric assessment workflows (Cai et al., 20 Oct 2025, Zhang et al., 14 Jan 2026).

1. Theoretical Foundations: Latent Honesty States and Topic Dependency

Central to modern dishonesty-aware assessment frameworks is the explicit modeling of a patient’s latent honesty state at each interaction turn. In formal terms, the honesty state at turn $t$ is denoted as

$$h_t \in \{\mathsf{honest}, \mathsf{conceal}, \mathsf{exaggerate} \}$$

This state is not static but evolves as a discrete-time Markov process, conditioned on both the previous honesty state $h_{t-1}$ and the current clinical topic $z_t$:

$P(h_t \mid h_{t-1}, z_t)$

State transitions are parameterized to reflect empirically attested topic-dependent dishonesty, such as increased concealment during inquiries about suicidality and exaggeration in disability-related contexts (Zhang et al., 14 Jan 2026). The patient’s internal response vector, $$S_t = [\mathrm{Trust}_t, \mathrm{Stress}_t]^\top \in [0,1]^2$$, further modulates reporting style.

This mechanistic modeling of honesty is foundational for both synthetic data generation and for the design of adaptive real-world assessment agents. It provides a formal substrate for simulating and understanding the interplay between psychological state, social context, and reporting fidelity.

2. Multi-Agent Architectural Paradigms and Dialogue Flow

Dishonesty-aware assessment systems typically deploy explicit multi-agent frameworks. One prominent instantiation, MentalAED, implements a closed-loop workflow with four interacting roles (Zhang et al., 14 Jan 2026):

• Assessor Agent $\mathcal{A}$: Selects assessment scales based on the patient profile.
• Patient Agent $\mathcal{P}$: Simulates human responses under variable honesty and stress/trust conditions.
• Evaluator Agent $\mathcal{E}$: Conducts the semi-structured interview, maintains a suspicion score $\xi_t$, and performs follow-up probing.
• Diagnostician Agent $\mathcal{D}$: Integrates all collected evidence (self-reports, clinician ratings, transcripts, final suspicion) into a diagnostic summary.

The framework operates iteratively over $T_{\max}$ dialogue turns, with the Patient Agent’s output conditioned by its current honesty state, and the Evaluator Agent adaptively updating its suspicion score and modifying its probing strategy based on detected inconsistencies. This yields a dynamic, context-sensitive interrogation protocol capable of surfacing and adapting to misreporting behaviors in real time.

The detailed high-level pseudocode formalizes these interactions, including scale selection, patient state evolution, and feedback-driven question selection.

3. Dishonesty Detection and Consistency Scoring

Dishonesty detection within this paradigm is realized via both continuous suspicion scoring and consistency-based metrics. For each user response $u_t^P$, the Evaluator Agent updates its suspicion score

$\xi_t = \sigma(\xi_{t-1} + \gamma \cdot S(u_t^P))$

where $S(u_t^P) \in [-1,1]$ quantifies the deviation from normative affective, temporal, or logical response patterns, and $\sigma$ is a squashing function (typically sigmoid or tanh).

When $\xi_t$ exceeds a threshold $\theta_{\mathrm{susp}}$, an investigative probing policy is triggered:

$$d_t = \begin{cases} \mathsf{Investigate} & \xi_t > \theta_{\mathrm{susp}} \ \mathsf{Proceed} & \text{otherwise} \end{cases}$$

Specialized question templates are employed to cross-check previous answers for internal consistency and to elicit clarification where semantic or temporal discrepancies are detected. A complementary approach, outlined as a candidate module in (Cai et al., 20 Oct 2025), proposes a chain-level consistency score $C$ based on the mean semantic distance between actual and expected paraphrases of user responses; low $C$ values would trigger additional clarification prompts.

4. Integration into Assessment Workflows and Diagnostic Fusion

Dishonesty-aware strategies are embedded in interactive assessment protocols, typically using adaptive questioning logic and real-time feedback. For instance, Cai et al. (Cai et al., 20 Oct 2025) describe an LLM-driven system that employs a dynamic assessment engine to track symptom intensity and adapt questioning based on emergent evidence of distress or inconsistency.

In the diagnostic fusion stage, the final diagnosis is explicitly modulated by the suspicion score:

$$\mathrm{Score}_k = w_{\mathrm{self},k}\,s_{\mathrm{self},k} + w_{\mathrm{clin},k}\,s_{\mathrm{clin},k} - w_{\mathrm{susp}}\,\xi_T$$

where, for each diagnostic dimension $k$, the weights $w$ down-weight self-report scores when concealment is suspected, and clinician scores under exaggeration (Zhang et al., 14 Jan 2026). This systematic adjustment aims to limit the impact of unreliable inputs on final clinical determinations.

5. Empirical Evidence and Evaluation Metrics

Quantitative assessment of dishonesty-aware methods relies on a combination of classification metrics and alignment tests. In one evaluation on synthetic data, status and severity classification achieved accuracy of 86.8% and 76.2%, macro-F1 of 0.65 and 0.50, Cohen’s $\kappa$ of 0.54 and 0.31, and Matthews Correlation Coefficient (MCC) of 0.55 and 0.33, respectively. Dishonesty discrimination yielded an AUC-ROC of 0.945, with a Pearson $r=0.291$ with expert suspicion ratings ($p=0.0553$) (Zhang et al., 14 Jan 2026).

Ablation studies comparing passive versus chain-of-thought (CoT) evaluators demonstrated substantial improvements in severity grading accuracy (+25.9%) and status classification (+6.9%). These findings suggest that explicit modeling of honesty and adaptive probing directly enhances the fidelity and granularity of psychiatric assessment on both synthetic and real-data benchmarks.

By contrast, Cai et al. (Cai et al., 20 Oct 2025) conducted qualitative expert interviews only, with perceived empathy, coherence, and clinical applicability rated subjectively. No quantitative trials or ground-truth misreporting studies were reported, but expert consensus suggested that natural language interaction could improve data authenticity.

6. Cross-Cultural, Bias, and Privacy Considerations

Both major works identify significant risk factors around cultural, linguistic, and algorithmic bias. Variations in language style across regions and age groups, as well as stigma-influenced under-reporting in certain communities, necessitate template localization and continual fairness auditing. Training data dominated by urban, educated college populations risks poor generalization to rural or international populations; system designs must allow for clinician-injected culture-specific prompts and per-group performance monitoring (Cai et al., 20 Oct 2025).

Privacy protection is recognized as a critical design challenge. While neither system as yet implements a formal privacy module, future directions include the integration of differential privacy or federated learning to safeguard personal data. The proposed approach is to perturb feature vectors with Gaussian noise:

$\tilde{f} = f + \mathcal{N}(0, \sigma^2 I)$

to attain $\varepsilon$-DP guarantees.

7. Limitations and Future Directions

Current dishonesty-aware assessment systems face several limitations. Many elements—such as comorbidity modeling and large-scale empirical validation—remain to be addressed. Human evaluation of suspicion scores displays moderate inter-rater reliability (ICC ≈ 0.3–0.4), motivating richer rubric development and expansion to larger multi-site samples (Zhang et al., 14 Jan 2026).

Future research includes:

• Clinical trials with quantifiable ground-truth indicators of misreporting to benchmark sensitivity, specificity, and test–retest reliability relative to standard tools (Cai et al., 20 Oct 2025).
• Integration of multimodal signals (e.g., voice, facial affect) to augment deception detection, particularly in hybrid or telemedicine contexts.
• Development of more nuanced probabilistic honesty models $P(h_t)$, informed by empirical patient behavioral data.
• Extension to fairness-aware, multilingual deployments with culture-localized probes and A/B testing of prompt banks.

A plausible implication is that successful deployment of dishonesty-aware psychiatric assessment tools will depend on rigorous validation, robust privacy/bias defenses, and dynamic adaptation to diverse user subgroups. These innovations lay a foundation for more reliable, context-sensitive, and ethically grounded mental health technology (Cai et al., 20 Oct 2025, Zhang et al., 14 Jan 2026).