VERA-MH: Validation of Ethical and Responsible AI in Mental Health

Abstract: Chatbot usage has increased, including in fields for which they were never developed for--notably mental health support. To that end, we introduce Validations of Ethical and Responsible AI in Mental Health (VERA-MH), a novel clinically-validated evaluation for safety of chatbots in the context of mental health support. The first iteration of VERA-MH focuses on Suicidal Ideation (SI) risks, by assessing how well chatbots can responds to users that might be in crisis. VERA-MH is comprised of three steps: conversation simulation, conversation judging and model rating. First, to simulate conversations with the chatbot under evaluation, another chatbot is tasked with role-playing users based on specific personas. Such user personas have been developed under clinical guidance, to make sure that, among others, multiple risk factors, demographic characteristics and disclosure factors were represented. In the judging step, a second support model is used as an LLM-as-a-Judge, together with a clinically-developed rubric. The rubric is structured as a flow, with a single Yes/No question asked each time, to improve answers' consistency and highlight models' failure modes. In the last stage, results of each conversation are aggregated to present the final evaluation of the chatbot. Together with the framework, we present the result of the evaluations for four leading LLM providers.

• The paper introduces a clinically validated, multi-turn evaluation framework (VERA-MH) that simulates conversations to assess chatbot responses in suicide risk scenarios.
• It employs a three-stage pipeline—conversation simulation, judge-based assessment, and score aggregation—to ensure reproducibility and clinical alignment.
• Results show significant between-system variability and highlight practical challenges and opportunities in deploying safe AI in mental health.

VERA-MH: A Clinically-Validated Framework for Safety Evaluation of AI in Mental Health

Context and Motivation

The widespread adoption of LLM-based chatbots in mental health contexts continues despite an alarming lack of regulatory frameworks, benchmarks, or domain-specific safety assurance. Catastrophic failures—such as recent cases of chatbots providing potentially harmful advice to individuals experiencing suicidal ideation—underscore acute gaps in pre-deployment safety assessments. Existing evaluations for mental health chatbots frequently exhibit construct validity issues, static or out-of-context assessment methods, and insufficient grounding in clinical standards. To address these deficiencies, the VERA-MH (Validation of Ethical and Responsible AI in Mental Health) framework introduces a clinically-vetted, multi-turn, judge-based evaluation protocol targeting the specific domain of suicide risk support, incorporating expertise from AI developers, practicing clinicians, and suicide prevention authorities.

Framework Design and Methodology

VERA-MH consists of a three-stage pipeline: (i) conversation simulation, (ii) conversation judging, and (iii) score aggregation. Each component is architected to align with the priorities of clinical safety, real-world applicability, and reproducibility.

Conversation Simulation

Conversation generation leverages an ancillary LLM (“user-LLM”) to act as an at-risk user across 100 unique personas with systematically varied demographic and clinical parameters (age, gender, financial stress, risk level, history, triggers, social support, etc.). Persona generation involves automated synthesis and expert clinician review to ensure representativeness and avoid stereotypical or biased modeling—excluding race to minimize bias reinforcement at intersections of identity. Prompting strategies are refined to produce realistic dialog: user-LLMs minimize pleasantries, issue abrupt termination upon perceived dead-ends, and generate succinct, context-relevant responses as evidenced in conversation length statistics.

Figure 1: Distribution of the conversational length of both user and chatbot models. Users' responses tend to be shorter, indicating the user-LLM correctly interprets the instructions.

Judging and Rubric

A second LLM (“Judge-LLM”) assesses each multi-turn dialog using a flow-structured, clinically-developed rubric spanning five quasi-orthogonal safety dimensions:

• Detects Potential Risk
• Confirms Risk
• Guides to Human Care
• Supportive Conversation
• Follows AI Boundaries

For each dimension, the judge outputs one of: ‘Best Practice’, ‘Suboptimal but Low Potential for Harm’, ‘High Potential for Harm’, or ‘Not Relevant’. The rubric’s sequential structure (up to 35 items per conversation) facilitates consistent ratings, precise failure identification, and increased inter-rater reliability (IRR). Human validation studies show average human-human IRR at 0.77 and human–LLM-judge IRR at 0.77–0.78 (Bentley et al., 4 Feb 2026), supporting the use of LLM-judges for scalable, automated safety evaluation.

Aggregation and Reporting

Evaluation outcomes form a matrix of dimensions × rating, excluding non-relevant conversations from normalization to preserve signal for actionable cases (e.g., guidance to human care is not relevant in control groups without detected risk). Primary results for leading LLM families (Claude Opus 4.7, GPT-5.4, Gemini 3 Pro, Grok 4) utilize 2000 multi-turn interactions for stable statistics.

Figure 2: Results of the experiments. Non-relevant conversations are shown as a fraction of the total; relevant conversations are row-normalized to avoid skew.

Detailed model-specific breakdowns by provider are available, highlighting significant between-system variability (see Figures 2–6).

Figure 3: Performance profile of Gemini models according to VERA-MH rubric.

Figure 4: Safety ratings for GPT-5.X models in suicide risk scenarios.

Figure 5: Disaggregated rubric compliance for Grok models.

Figure 6: Claude Opus evaluation by VERA-MH rubric dimensions.

Figure 7: Evaluation of Claude Sonnet models, highlighting dimensional variation.

Addressing Benchmarking and Evaluation Critique

VERA-MH directly confronts long-standing limitations in LLM safety evaluation:

• Construct Validity: Unlike broad, multi-issue test beds, VERA-MH targets a single high-impact clinical issue (suicidal ideation), providing operational clarity and actionable results—minimizing risks of spurious generalization prevalent in prior “catch-all” benchmarks [bean2026measuring, alaa2025position].
• Realism & Context: Dynamic, multi-turn interaction with LLM-simulated users increases ecological validity absent in single-turn or scripted evaluation modes [neurips-data-2021-raji, badawi-etal-2026-trust].
• Stakeholder Engagement: The evaluation, code, and rubrics are open-source and developed with multi-stakeholder, participatory input and public RFC, directly addressing power dynamics and hidden curation biases [Boyd2012CRITICALQF, 10.1145/3593013.3594019].
• Transparency and Actionability: The rubric’s flow structure ensures traceability of failure points, supporting development and remediation.
• Stability and Scalability: Recommendations on personas count and multiple runs per persona are motivated by analyses of rating stability and intra-model variability [rabanser2026scienceaiagentreliability].

Limitations and Open Challenges

The framework’s limitations are not merely technical but reflect the institutional and sociotechnical context:

• Best Practice Volatility: The clinical consensus underlying the rubric is inherently unstable; guidance must be revisited as medical or regulatory standards evolve.
• Gameability and Overfitting: Open-sourcing increases transparency but also exposes the evaluation to strategic model optimization (“benchmark gaming”) [10.1145/363(0106.36590)12, Goodhart1984ProblemsOM].
• Model & Persona Dependence: User simulation realism is bounded by LLM capabilities and biases in underlying data; future work is required for internationalization (non-English, non-US contexts) and further reduction of stereotype propagation [Venkit2026ATO, ostrow2025llmsreproducestereotypessexual].
• Resource Intensiveness: Each evaluation requires thousands of multi-turn dialogs, judged on multiple dimensions, with associated compute and monetary expense.
• Metric Overload vs. Oversimplification: The rating scheme (20 numbers per model) resists reduction to a single score, intending to avoid Goodhart’s Law effects but presenting complexity for comparative or regulatory utility [THOMAS2022100476].

Implications and Prospects

Practically, VERA-MH provides a reproducible, automated mechanism for pre-deployment safety validation of mental health chatbots in high-risk use-cases. By adopting an explicit clinical frame and transparent reporting, it materially advances prior benchmarks that lacked either clinical specificity or validation [pombal2025mindevalbenchmarkinglanguagemodels, arnaout2026responsibleevaluationaimental]. Theoretically, it models a paradigm for single-issue, stakeholder-informed evaluation and highlights the necessity of domain-grounded construct validity in safety auditing of LLMs.

Anticipated future directions include: expansion to additional high-risk scenarios (beyond suicidal ideation), porting to other linguistic and cultural domains, integration with open-weight models to reduce ancillary model bias, and protocolization for external, third-party auditing.

Conclusion

VERA-MH provides a rigorous, clinically-aligned, and transparent approach to benchmarking chatbot safety in the domain of suicide risk. It demonstrates clear advantages over extant methodologies in construct validity, real-world applicability, and multi-stakeholder engagement, while acknowledging and systematically addressing both technical and sociotechnical limitations. This work establishes a foundation for continued progress in trustworthy AI deployment in mental health and reifies the need for careful, issue-focused evaluation frameworks in domains where safety failures carry material risk.

Reference: “VERA-MH: Validation of Ethical and Responsible AI in Mental Health” (2605.13318)