---
title: 'VERA-MH: AI Safety for Mental Health Chatbots'
url: https://www.emergentmind.com/topics/vera-mh
type: topic
---

# VERA-MH: AI Safety for Mental Health Chatbots

VERA-MH, short for **Validation of Ethical and Responsible AI in Mental Health**, is a clinically grounded, open-source, fully automated safety evaluation for chatbots used in mental-health contexts, with its first iteration focused on **suicidal ideation (SI)** and **suicide-risk detection and response**. It was developed in response to the rapid uptake of general-purpose generative AI systems for psychological support, despite those systems not being designed, tested, or regulated for that purpose. VERA-MH is explicitly **multi-turn**, **model-agnostic**, and **multidimensional**: rather than scoring a single prompt-response pair, it simulates extended conversations, evaluates them with a clinically developed rubric, and aggregates results into structured safety profiles for the chatbot under test [2605.13318].

## 1. Scope, rationale, and problem definition

VERA-MH addresses a specific question: whether an AI chatbot responds **safely** when a user may be at risk of suicide. The framework is not presented as an efficacy benchmark for psychotherapy or counseling quality. The authors instead distinguish **safety evaluation** from **efficacy evaluation**, arguing that generic chatbot benchmarks and single-turn prompts are inadequate in mental-health settings because crisis disclosure may be delayed, indirect, or only legible across several turns [2605.13318].

The motivating harms are concrete. The papers describe concern that chatbots may **romanticize suicidal thoughts**, **fail to recognize crisis language**, **encourage harmful dependence on the AI**, **provide unsafe or method-related information**, or **miss opportunities to redirect users to human help**. This narrow scoping to SI is a design choice intended to improve construct validity: rather than claiming to measure “mental health safety” in general, VERA-MH evaluates one high-risk, clinically important domain in depth. The concept paper frames this orientation in “do no harm” terms and states that future iterations may broaden beyond suicide risk to other harms such as **harm to others or psychosis** [2510.15297].

A recurrent misconception addressed by the framework is that crisis safety can be reduced to a detection problem. VERA-MH rejects that reduction. A chatbot may recognize potentially suicidal language yet still fail by not asking clarifying questions, not escalating toward human support, using invalidating language, overstepping AI boundaries, or giving harmful content. This broader view underlies the framework’s multidimensional rubric and its emphasis on full-interaction evaluation rather than isolated utterance classification [2602.05088].

## 2. System architecture and evaluation pipeline

At the system level, VERA-MH is organized as a **three-stage pipeline**: **conversation simulation**, **conversation judging**, and **model rating / aggregation**. These stages can be run in sequence or independently. In the first stage, an ancillary LLM acts as a **user-agent**, role-playing a help-seeking user from a clinically designed persona. In the second, a separate LLM functions as an **LLM-as-a-Judge**, applying the rubric to the completed dialogue. In the third, conversation-level judgments are aggregated into a final evaluation for the chatbot under test [2605.13318].

This architecture was designed to replace three alternatives that the authors explicitly reject: **scripted conversations**, **single prompts**, and **human-run role-play**. Scripted dialogues are too rigid, single-turn prompts do not capture sequential safety behavior, and human role-play is ethically and logistically difficult to scale. The resulting benchmark is therefore fully automated but structured to preserve clinically meaningful interactional context. The concept paper summarizes the workflow as follows: “**A user-agent is created using clinically developed personas via system-prompts.**” Then, “**The user-agent and the AI tool (chatbot) being assessed simulate a conversation.**” Finally, “**The judge-agent scores the simulated conversation based on the clinically developed rubric and response options.**” [2510.15297]

The framework’s automation is paired with model-agnosticism. In the reliability study, simulated conversations were generated between LLM-based users and three general-purpose chatbot provider-agents—**GPT-4o**, **GPT-5.0**, and **Gemini 3-pro-preview**—while the user-agents were instantiated with **GPT-5.0**, **Claude Opus 4.1**, and **Gemini 3**. The automated judge was implemented primarily with **GPT-4o**, with secondary judges including **GPT-5.2**, **Claude Sonnet 4.5**, and **Gemini 2.5 Flash**. This separation of user-agent, provider-agent, and judge-agent is central to VERA-MH’s design as an evaluation framework rather than a benchmark tied to one model family [2602.05088].

## 3. Simulated users, persona design, and realism controls

VERA-MH’s simulated conversations are driven by clinician-informed personas. The more mature 2026 evaluation paper describes **100 personas**, each with both demographic and clinical characteristics. Demographic characteristics explicitly include **age**, **gender**, and **financial stress**. Clinical and risk-related characteristics include **suicide risk level**, **history of suicidal thoughts**, **disclosure**, **access to mental health care**, **social isolation**, and **recent triggers**. Clinicians selected the suicide-risk distribution as **30% high risk**, **30% medium risk**, **30% low risk**, and **10% no-risk** as a control group, with the remaining characteristics assigned **randomly and independently** [2605.13318].

The persona-construction workflow is itself hybrid. Clinicians defined the persona structure and risk distribution; **GPT-5.2** generated biography elements and one seed phrase suggestion; **Grok 4** generated a second phrase suggestion; and clinicians performed a final review for accuracy and representativeness. The papers also describe safeguards against stereotyping: there is **no race category**, and **names are not passed in the system prompt**, specifically to reduce biasing the model. The authors note, however, that no finite persona inventory can capture the full diversity of human experience [2605.13318].

Earlier VERA-MH iterations used smaller persona sets. The concept paper describes **10 personas** spanning a **continuum of suicide risk**, including acutely suicidal presentations, no-current-risk controls, and indirect or tone-mediated risk disclosures, with **five independent conversations** per persona. The reliability paper likewise used **10 user-agent profiles**, varying in demographics, suicide-risk level, disclosure style, and mental-health context, to generate **90 simulated conversations** [2510.15297].

Clinician realism judgments were a central part of validation. In the concept paper’s initial human study, clinicians rated the simulated user-agents at an average realism score of **4.1 on a 5-point scale**. In the later reliability study, clinicians judged presentation realism at around **4/5** (“mostly realistic”) and communication realism at around **3/5** (“somewhat realistic”). A key limitation identified in that study was repetition in some user-agent behavior; the authors state that the simulation process was later improved by giving the user-agent LLM memory of prior statements to reduce this problem [2602.05088].

## 4. Rubric structure, judging logic, and safety dimensions

The clinical core of VERA-MH is its rubric. In the current formulation, the rubric comprises five quasi-independent dimensions: **Detects Potential Risk**, **Confirms Risk**, **Guides to Human Care**, **Supportive Conversation**, and **Follows AI Boundaries**. Each dimension is rated as **Best Practice**, **Suboptimal but Low Potential for Harm**, **High Potential for Harm**, or **Not Relevant**. The rubric is severity-based: the **most severe applicable criterion determines the final rating** [2602.05088].

The judge does not generate unconstrained narrative commentary. Instead, it follows a structured flow. In the 2026 evaluation paper, a fresh judge instance receives the conversation and proceeds through a **sequence of up to 35 multiple-choice questions**, most of them **Yes/No**, with some **Yes/No/Not Relevant**. The questions are ordered from **more severe to less severe** within each dimension. A “Yes” generally indicates a problematic or suboptimal behavior; a **Best Practice** rating is possible only if all questions in that dimension are exhausted without detecting a failure. The concept paper describes the same logic in earlier form through a **30-item rating form** used by clinicians and the judge-agent [2605.13318].

The framework operationalizes suicide-risk best practice as a staged response sequence: detect potentially suicidal language, ask direct clarifying questions to confirm risk and immediacy, provide human or crisis support, remain supportive and validating, and maintain appropriate AI boundaries. Failure modes encoded in the rubric include **failing to recognize or respond to risk**, **false positives in benign contexts**, **rigid or repetitive safety questioning**, **overwhelming or poorly targeted crisis guidance**, **invalidating or robotic tone**, **encouraging reliance on AI over humans**, **diagnosing the user**, **glamorizing suicide**, and **giving harmful content** [2605.13318].

The concept paper uses slightly different dimension labels—**Detects risk**, **Probes risk**, **Takes appropriate actions**, **Validates and collaborates**, and **Maintains safe boundaries**—but the underlying structure is continuous: recognition, confirmation, escalation, supportive interaction, and boundedness. The evolution in terminology suggests rubric refinement rather than a change in the benchmark’s basic clinical model [2510.15297].

A specific warning appears in the later paper regarding **Detects Potential Risk**: it should **not** be interpreted alone as a standalone detection metric. This reflects the benchmark’s design philosophy. Correct SI handling requires both recognition and appropriate action, and VERA-MH’s judgment logic is constructed to evaluate that sequence rather than a single classifier-like endpoint [2605.13318].

## 5. Aggregation, validation, and empirical results

VERA-MH does not collapse safety to a single scalar score. Instead, it produces a structured matrix over **dimensions × ratings**. In the concept paper, each cell indicates the proportion of times a given score was assigned to that dimension across all simulated conversations, and the appendix states that the values in each row add up to 1. The later 2026 framework modifies this slightly: **Not Relevant** conversations are counted separately, and the remaining relevant conversations are normalized to sum to 1 within each dimension. The authors note that this handling can appear counterintuitive, but it prevents irrelevant cases from skewing the safety assessment [2510.15297].

The recommended evaluation scale in the later paper is **100 personas**, **2 conversations per persona**, and up to **30 turns**, described as yielding roughly **2000 judged data points per evaluation**. In the reliability study, by contrast, the validation dataset consisted of **90 simulated conversations**, each capped at a fixed length or **4,000 words**, with each conversation ending on a provider response so the chatbot had an opportunity to address late risk disclosure [2605.13318].

The strongest evidence for VERA-MH’s validity comes from clinician–judge agreement studies. In the reliability paper, **six licensed mental health clinicians** underwent didactic orientation, interactive training, and **three rounds of practice coding** on 40 simulated conversations. In the main study, **3 clinicians independently rated each conversation**, and clinician consensus was defined as the modal rating, with the expert clinician’s rating used to break splits. Agreement among clinicians was strong: **74.7%** of ratings were unanimous, **25.1%** were 2–1 splits, and only **one** dimension-rating was a true 1-1-1 split [2602.05088].

Using **Krippendorff’s alpha** as the chance-corrected inter-rater reliability statistic, the study reports  
$$\alpha_{\text{clinician-clinician}} = 0.77 \quad (95\%\,CI:\,0.73\text{–}0.80),$$  
$$\alpha_{\text{LLM vs clinician consensus}} = 0.81 \quad (95\%\,CI:\,0.77\text{–}0.86),$$  
$$\alpha_{\text{LLM vs expert clinician}} = 0.80 \quad (95\%\,CI:\,0.76\text{–}0.84),$$  
and  
$$\alpha_{\text{LLM + all clinicians}} = 0.76 \quad (95\%\,CI:\,0.73\text{–}0.80).$$  
The paper states that these values exceed the commonly used **0.70 acceptability threshold**, supporting the benchmark’s reliability. Raw agreement was also high: **83%** clinician-clinician, **86%** LLM-clinician individual, and **75%** LLM-clinician consensus [2602.05088].

Dimension-level reliability varied. Among clinicians, reported alphas were **0.55** for **Detects Potential Risk**, **0.73** for **Confirms Risk**, **0.63** for **Guides to Human Care**, **0.45** for **Supportive Conversation**, and **0.74** for **Follows AI Boundaries**. For LLM versus clinician consensus, they were **0.83**, **0.75**, **0.69**, **0.91**, and **0.56**, respectively. The study interprets this as especially strong judge alignment on **Supportive Conversation**, with weaker agreement on dimensions that involved greater subjectivity or category imbalance, particularly **Follows AI Boundaries** [2602.05088].

The 2026 evaluation paper also reports benchmark results on four leading provider models—**Claude Opus 4.7**, **GPT-5.4**, **Gemini 3 Pro Preview**, and **Grok 4**—and concludes that the models differ in safety profile across the five dimensions and that **none is perfect**. The judge temperature was set to **0** to reduce variation, and for GPT-5.X models `max_tokens` had to be set to **5000** because the default was insufficient. The paper’s emphasis is not on a single winner, but on exposing model-specific failure configurations in realistic SI-related interactions [2605.13318].

## 6. Limitations, interpretation, and future development

VERA-MH is presented as a clinically meaningful benchmark, but its authors are explicit about its limitations. The first is scope: the benchmark currently centers on **suicide-risk safety**, not mental-health chatbot behavior in general. This narrowness is intentional, but it also means that broader domains—such as psychosis, harm to others, or other high-risk presentations—remain outside the first iteration’s formal coverage [2510.15297].

A second limitation concerns simulated-user coverage. Earlier versions used only **10 personas**, and even the larger 100-persona design cannot exhaustively represent real users. The concept paper notes that youth were intentionally excluded in the first version, while the reliability paper states that the benchmark does **not yet include youth under 18**. The later validation study also found that disclosure style prompts were not always faithfully reproduced: clinicians’ inferred disclosure levels matched the original prompts only **36.5%** of the time, although the resulting conversations still spanned meaningful disclosure variability [2602.05088].

A third limitation is interpretive. The multidimensional matrix output preserves clinically relevant nuance, but the concept paper states that the current **5×4 matrix** “**can make it challenging to compare different models or fully capture the trade-offs among dimensions or models**.” The authors are therefore exploring more actionable and interpretable representations without sacrificing the benchmark’s underlying dimensional structure [2510.15297].

Finally, VERA-MH is designed as an **iterative evaluation**, not a static benchmark. Planned developments include more systematic clinician-versus-judge comparison, further rubric refinement, training and calibration sessions to improve clinician consistency, iteration on the judge prompt, experimentation with **multiple judge-agents** for a single conversation, improved user-agent memory and disclosure realism, expansion to more user profiles and longer conversations, and extension to additional mental-health risk domains [2510.15297].

In that form, VERA-MH occupies a specific place in the evaluation landscape: it treats mental-health chatbot safety as a structured, sequential clinical task rather than as generic harmful-content detection or as a single-turn benchmark. Its main contribution is therefore methodological as much as empirical: a clinically anchored framework for auditing whether AI systems detect risk, confirm it appropriately, guide users toward human care, sustain supportive conversation, and remain within safe AI boundaries under multi-turn conditions [2602.05088].

Source: https://www.emergentmind.com/topics/vera-mh