VERA-MH: AI Safety for Mental Health Chatbots
- VERA-MH is a clinically grounded, open-source framework that evaluates safety in mental health chatbots, focusing on suicide-risk detection and response.
- It simulates multi-turn conversations using clinician-informed personas and a structured, multidimensional rubric to assess response safety.
- The framework’s three-stage pipeline—conversation simulation, judging, and result aggregation—provides actionable safety profiles in realistic mental health scenarios.
VERA-MH, short for Validation of Ethical and Responsible AI in Mental Health, is a clinically grounded, open-source, fully automated safety evaluation for chatbots used in mental-health contexts, with its first iteration focused on suicidal ideation (SI) and suicide-risk detection and response. It was developed in response to the rapid uptake of general-purpose generative AI systems for psychological support, despite those systems not being designed, tested, or regulated for that purpose. VERA-MH is explicitly multi-turn, model-agnostic, and multidimensional: rather than scoring a single prompt-response pair, it simulates extended conversations, evaluates them with a clinically developed rubric, and aggregates results into structured safety profiles for the chatbot under test (Belli et al., 13 May 2026).
1. Scope, rationale, and problem definition
VERA-MH addresses a specific question: whether an AI chatbot responds safely when a user may be at risk of suicide. The framework is not presented as an efficacy benchmark for psychotherapy or counseling quality. The authors instead distinguish safety evaluation from efficacy evaluation, arguing that generic chatbot benchmarks and single-turn prompts are inadequate in mental-health settings because crisis disclosure may be delayed, indirect, or only legible across several turns (Belli et al., 13 May 2026).
The motivating harms are concrete. The papers describe concern that chatbots may romanticize suicidal thoughts, fail to recognize crisis language, encourage harmful dependence on the AI, provide unsafe or method-related information, or miss opportunities to redirect users to human help. This narrow scoping to SI is a design choice intended to improve construct validity: rather than claiming to measure “mental health safety” in general, VERA-MH evaluates one high-risk, clinically important domain in depth. The concept paper frames this orientation in “do no harm” terms and states that future iterations may broaden beyond suicide risk to other harms such as harm to others or psychosis (Belli et al., 17 Oct 2025).
A recurrent misconception addressed by the framework is that crisis safety can be reduced to a detection problem. VERA-MH rejects that reduction. A chatbot may recognize potentially suicidal language yet still fail by not asking clarifying questions, not escalating toward human support, using invalidating language, overstepping AI boundaries, or giving harmful content. This broader view underlies the framework’s multidimensional rubric and its emphasis on full-interaction evaluation rather than isolated utterance classification (Bentley et al., 4 Feb 2026).
2. System architecture and evaluation pipeline
At the system level, VERA-MH is organized as a three-stage pipeline: conversation simulation, conversation judging, and model rating / aggregation. These stages can be run in sequence or independently. In the first stage, an ancillary LLM acts as a user-agent, role-playing a help-seeking user from a clinically designed persona. In the second, a separate LLM functions as an LLM-as-a-Judge, applying the rubric to the completed dialogue. In the third, conversation-level judgments are aggregated into a final evaluation for the chatbot under test (Belli et al., 13 May 2026).
This architecture was designed to replace three alternatives that the authors explicitly reject: scripted conversations, single prompts, and human-run role-play. Scripted dialogues are too rigid, single-turn prompts do not capture sequential safety behavior, and human role-play is ethically and logistically difficult to scale. The resulting benchmark is therefore fully automated but structured to preserve clinically meaningful interactional context. The concept paper summarizes the workflow as follows: “A user-agent is created using clinically developed personas via system-prompts.” Then, “The user-agent and the AI tool (chatbot) being assessed simulate a conversation.” Finally, “The judge-agent scores the simulated conversation based on the clinically developed rubric and response options.” (Belli et al., 17 Oct 2025)
The framework’s automation is paired with model-agnosticism. In the reliability study, simulated conversations were generated between LLM-based users and three general-purpose chatbot provider-agents—GPT-4o, GPT-5.0, and Gemini 3-pro-preview—while the user-agents were instantiated with GPT-5.0, Claude Opus 4.1, and Gemini 3. The automated judge was implemented primarily with GPT-4o, with secondary judges including GPT-5.2, Claude Sonnet 4.5, and Gemini 2.5 Flash. This separation of user-agent, provider-agent, and judge-agent is central to VERA-MH’s design as an evaluation framework rather than a benchmark tied to one model family (Bentley et al., 4 Feb 2026).
3. Simulated users, persona design, and realism controls
VERA-MH’s simulated conversations are driven by clinician-informed personas. The more mature 2026 evaluation paper describes 100 personas, each with both demographic and clinical characteristics. Demographic characteristics explicitly include age, gender, and financial stress. Clinical and risk-related characteristics include suicide risk level, history of suicidal thoughts, disclosure, access to mental health care, social isolation, and recent triggers. Clinicians selected the suicide-risk distribution as 30% high risk, 30% medium risk, 30% low risk, and 10% no-risk as a control group, with the remaining characteristics assigned randomly and independently (Belli et al., 13 May 2026).
The persona-construction workflow is itself hybrid. Clinicians defined the persona structure and risk distribution; GPT-5.2 generated biography elements and one seed phrase suggestion; Grok 4 generated a second phrase suggestion; and clinicians performed a final review for accuracy and representativeness. The papers also describe safeguards against stereotyping: there is no race category, and names are not passed in the system prompt, specifically to reduce biasing the model. The authors note, however, that no finite persona inventory can capture the full diversity of human experience (Belli et al., 13 May 2026).
Earlier VERA-MH iterations used smaller persona sets. The concept paper describes 10 personas spanning a continuum of suicide risk, including acutely suicidal presentations, no-current-risk controls, and indirect or tone-mediated risk disclosures, with five independent conversations per persona. The reliability paper likewise used 10 user-agent profiles, varying in demographics, suicide-risk level, disclosure style, and mental-health context, to generate 90 simulated conversations (Belli et al., 17 Oct 2025).
Clinician realism judgments were a central part of validation. In the concept paper’s initial human study, clinicians rated the simulated user-agents at an average realism score of 4.1 on a 5-point scale. In the later reliability study, clinicians judged presentation realism at around 4/5 (“mostly realistic”) and communication realism at around 3/5 (“somewhat realistic”). A key limitation identified in that study was repetition in some user-agent behavior; the authors state that the simulation process was later improved by giving the user-agent LLM memory of prior statements to reduce this problem (Bentley et al., 4 Feb 2026).
4. Rubric structure, judging logic, and safety dimensions
The clinical core of VERA-MH is its rubric. In the current formulation, the rubric comprises five quasi-independent dimensions: Detects Potential Risk, Confirms Risk, Guides to Human Care, Supportive Conversation, and Follows AI Boundaries. Each dimension is rated as Best Practice, Suboptimal but Low Potential for Harm, High Potential for Harm, or Not Relevant. The rubric is severity-based: the most severe applicable criterion determines the final rating (Bentley et al., 4 Feb 2026).
The judge does not generate unconstrained narrative commentary. Instead, it follows a structured flow. In the 2026 evaluation paper, a fresh judge instance receives the conversation and proceeds through a sequence of up to 35 multiple-choice questions, most of them Yes/No, with some Yes/No/Not Relevant. The questions are ordered from more severe to less severe within each dimension. A “Yes” generally indicates a problematic or suboptimal behavior; a Best Practice rating is possible only if all questions in that dimension are exhausted without detecting a failure. The concept paper describes the same logic in earlier form through a 30-item rating form used by clinicians and the judge-agent (Belli et al., 13 May 2026).
The framework operationalizes suicide-risk best practice as a staged response sequence: detect potentially suicidal language, ask direct clarifying questions to confirm risk and immediacy, provide human or crisis support, remain supportive and validating, and maintain appropriate AI boundaries. Failure modes encoded in the rubric include failing to recognize or respond to risk, false positives in benign contexts, rigid or repetitive safety questioning, overwhelming or poorly targeted crisis guidance, invalidating or robotic tone, encouraging reliance on AI over humans, diagnosing the user, glamorizing suicide, and giving harmful content (Belli et al., 13 May 2026).
The concept paper uses slightly different dimension labels—Detects risk, Probes risk, Takes appropriate actions, Validates and collaborates, and Maintains safe boundaries—but the underlying structure is continuous: recognition, confirmation, escalation, supportive interaction, and boundedness. The evolution in terminology suggests rubric refinement rather than a change in the benchmark’s basic clinical model (Belli et al., 17 Oct 2025).
A specific warning appears in the later paper regarding Detects Potential Risk: it should not be interpreted alone as a standalone detection metric. This reflects the benchmark’s design philosophy. Correct SI handling requires both recognition and appropriate action, and VERA-MH’s judgment logic is constructed to evaluate that sequence rather than a single classifier-like endpoint (Belli et al., 13 May 2026).
5. Aggregation, validation, and empirical results
VERA-MH does not collapse safety to a single scalar score. Instead, it produces a structured matrix over dimensions × ratings. In the concept paper, each cell indicates the proportion of times a given score was assigned to that dimension across all simulated conversations, and the appendix states that the values in each row add up to 1. The later 2026 framework modifies this slightly: Not Relevant conversations are counted separately, and the remaining relevant conversations are normalized to sum to 1 within each dimension. The authors note that this handling can appear counterintuitive, but it prevents irrelevant cases from skewing the safety assessment (Belli et al., 17 Oct 2025).
The recommended evaluation scale in the later paper is 100 personas, 2 conversations per persona, and up to 30 turns, described as yielding roughly 2000 judged data points per evaluation. In the reliability study, by contrast, the validation dataset consisted of 90 simulated conversations, each capped at a fixed length or 4,000 words, with each conversation ending on a provider response so the chatbot had an opportunity to address late risk disclosure (Belli et al., 13 May 2026).
The strongest evidence for VERA-MH’s validity comes from clinician–judge agreement studies. In the reliability paper, six licensed mental health clinicians underwent didactic orientation, interactive training, and three rounds of practice coding on 40 simulated conversations. In the main study, 3 clinicians independently rated each conversation, and clinician consensus was defined as the modal rating, with the expert clinician’s rating used to break splits. Agreement among clinicians was strong: 74.7% of ratings were unanimous, 25.1% were 2–1 splits, and only one dimension-rating was a true 1-1-1 split (Bentley et al., 4 Feb 2026).
Using Krippendorff’s alpha as the chance-corrected inter-rater reliability statistic, the study reports and The paper states that these values exceed the commonly used 0.70 acceptability threshold, supporting the benchmark’s reliability. Raw agreement was also high: 83% clinician-clinician, 86% LLM-clinician individual, and 75% LLM-clinician consensus (Bentley et al., 4 Feb 2026).
Dimension-level reliability varied. Among clinicians, reported alphas were 0.55 for Detects Potential Risk, 0.73 for Confirms Risk, 0.63 for Guides to Human Care, 0.45 for Supportive Conversation, and 0.74 for Follows AI Boundaries. For LLM versus clinician consensus, they were 0.83, 0.75, 0.69, 0.91, and 0.56, respectively. The study interprets this as especially strong judge alignment on Supportive Conversation, with weaker agreement on dimensions that involved greater subjectivity or category imbalance, particularly Follows AI Boundaries (Bentley et al., 4 Feb 2026).
The 2026 evaluation paper also reports benchmark results on four leading provider models—Claude Opus 4.7, GPT-5.4, Gemini 3 Pro Preview, and Grok 4—and concludes that the models differ in safety profile across the five dimensions and that none is perfect. The judge temperature was set to 0 to reduce variation, and for GPT-5.X models max_tokens had to be set to 5000 because the default was insufficient. The paper’s emphasis is not on a single winner, but on exposing model-specific failure configurations in realistic SI-related interactions (Belli et al., 13 May 2026).
6. Limitations, interpretation, and future development
VERA-MH is presented as a clinically meaningful benchmark, but its authors are explicit about its limitations. The first is scope: the benchmark currently centers on suicide-risk safety, not mental-health chatbot behavior in general. This narrowness is intentional, but it also means that broader domains—such as psychosis, harm to others, or other high-risk presentations—remain outside the first iteration’s formal coverage (Belli et al., 17 Oct 2025).
A second limitation concerns simulated-user coverage. Earlier versions used only 10 personas, and even the larger 100-persona design cannot exhaustively represent real users. The concept paper notes that youth were intentionally excluded in the first version, while the reliability paper states that the benchmark does not yet include youth under 18. The later validation study also found that disclosure style prompts were not always faithfully reproduced: clinicians’ inferred disclosure levels matched the original prompts only 36.5% of the time, although the resulting conversations still spanned meaningful disclosure variability (Bentley et al., 4 Feb 2026).
A third limitation is interpretive. The multidimensional matrix output preserves clinically relevant nuance, but the concept paper states that the current 5×4 matrix “can make it challenging to compare different models or fully capture the trade-offs among dimensions or models.” The authors are therefore exploring more actionable and interpretable representations without sacrificing the benchmark’s underlying dimensional structure (Belli et al., 17 Oct 2025).
Finally, VERA-MH is designed as an iterative evaluation, not a static benchmark. Planned developments include more systematic clinician-versus-judge comparison, further rubric refinement, training and calibration sessions to improve clinician consistency, iteration on the judge prompt, experimentation with multiple judge-agents for a single conversation, improved user-agent memory and disclosure realism, expansion to more user profiles and longer conversations, and extension to additional mental-health risk domains (Belli et al., 17 Oct 2025).
In that form, VERA-MH occupies a specific place in the evaluation landscape: it treats mental-health chatbot safety as a structured, sequential clinical task rather than as generic harmful-content detection or as a single-turn benchmark. Its main contribution is therefore methodological as much as empirical: a clinically anchored framework for auditing whether AI systems detect risk, confirm it appropriately, guide users toward human care, sustain supportive conversation, and remain within safe AI boundaries under multi-turn conditions (Bentley et al., 4 Feb 2026).
