---
title: 'MentaLLaMA: Interpretable Mental Health LLM'
url: https://www.emergentmind.com/topics/mentallama
type: topic
---

# MentaLLaMA: Interpretable Mental Health LLM

MentaLLaMA is an open-source large language model (LLM) series tailored for interpretable mental health analysis on social media and other short-form text corpora. Building directly on LLaMA-2 architectures, it incorporates instruction tuning with expert-validated explanations, task labels, and psychological best practices, with the aim of generating both accurate predictions (e.g., depression, stress, suicide risk) and human-interpretable rationales. MentaLLaMA has been widely adopted for evidence extraction, risk summarization, and pragmatic reasoning in clinical natural language processing (NLP) and forms the basis of downstream systems for suicide risk assessment and emotional support chatbots [2309.13567][2403.15478][2507.23247].

## 1. Model Architecture and Variants

MentaLLaMA is based on LLaMA-2, a Transformer decoder model with standard architectural hyperparameters for the 7B and 13B scales: 32–40 transformer layers, hidden sizes of 4,096–5,120, and approximately 32 attention heads. No architectural modifications are made; all improvements derive from domain- and task-specific instruction tuning [2309.13567][2403.15478]. The main variants are:

| Model Variant          | Base | Size  | Instruction Tuning         |
|-----------------------|------|-------|---------------------------|
| MentaLLaMA-7B         | LLaMA2         | 7B    | IMHI dataset           |
| MentaLLaMA-chat-7B    | LLaMA2-chat    | 7B    | IMHI dataset           |
| MentaLLaMA-chat-13B   | LLaMA2-chat    | 13B   | IMHI dataset           |

The pretraining corpus includes web text (e.g., CommonCrawl, The Pile) augmented with dialogue and code. The core pretraining objective is autoregressive maximum likelihood:
$$L = -\sum_{i=1}^N \log p_\theta(x_i | x_{<i}).$$
Instruction tuning on the interpretable mental health instruction (IMHI) dataset employs conditional text generation with cross-entropy objectives [2309.13567].

## 2. IMHI Dataset: Multi-Task, Multi-Source Foundation

The IMHI dataset underpins MentaLLaMA’s instruction tuning. It combines 10 existing corpora spanning 8 mental health analysis tasks (e.g., depression detection, stress, suicide risk, wellness dimensions, risk factors) on Reddit, Twitter, and SMS data. Altogether, the dataset comprises ∼105,000 (post, label, expert-prompt, explanation) instances. Each is generated using few-shot ChatGPT prompting grounded in expert-written templates. Labels and explanations cover a spectrum of disorders, causes, and risk factors [2309.13567].

Instructions and sample explanations were validated for correctness (agreement with gold labels), consistency (task-specific classifier accuracy), and quality (BART-score vs. gold rationales), with median expert evaluations ≥2.5/3 for consistency and ≥2.0/3 for reliability, professionality, and overall [2309.13567].

## 3. Instruction-Tuning and Optimization

MentaLLaMA is fine-tuned on IMHI via a conditional generation framework. The training objective minimizes
$$
\mathcal{L}(\phi) = - \sum_{(q,r)\in \mathcal{D}} \sum_{j=1}^{|r|} \log P_\phi(r_j | q, r_{<j}) + \lambda \|\phi\|_2^2
$$
where $q$ is the prompt (task+post), $r$ the rationale+answer, and $\lambda$ the regularization coefficient. Optimization uses AdamW, batch size 256, max sequence length 2048, and linear warmup. Hardware includes 4×A100 GPUs with Flash-Attention [2309.13567]. No auxiliary classification heads, ranking losses, or other architectural changes are introduced.

## 4. Evaluation: Correctness and Explanation Quality

MentaLLaMA and its chat variants are evaluated on a 10-dataset IMHI-held-out benchmark comprising stress, depression, suicide risk, loneliness, stress causes, and risk factors. Evaluation metrics are:

- **Classification correctness**: weighted $F_1$ (labels extracted from rationales via MentalBERT for non-templatic outputs).
- **Explanation quality**: BART-score and human expert ratings.

Comparative results demonstrate that MentaLLaMA-chat-13B matches or is within 5 $F_1$ points of discriminative SOTA (MentalRoBERTa) on 7/10 tasks, and consistently outperforms T5/BART/ChatGPT generative baselines in explanation quality by 0.2–0.5 BART-score [2309.13567]. Human scoring reflects high fluency and coherence, though a residual gap in “professionality” versus ChatGPT remains.

## 5. Empirical Performance Across Downstream Applications

MentaLLaMA serves as the generative engine in high-stakes clinical NLP pipelines:
- In suicide risk evidence summarization, MentaLLaMA is combined with BERT-based risk extraction and phrase dictionaries. The integrated system achieves recall $R=0.944$ and precision $P=0.906$ for highlight extraction, ranking 1st for recall in CLPsych 2024 shared task [2403.15478].
- In pragmatic reasoning benchmarks (P-ReMe), MentaLLaMA-7B underperforms more generalist instruction-tuned 7B LLMs (Mistral, Qwen) in agreement detection, implicature, and presupposition NLI, with maximum accuracy 0.52 (vs. Mistral/Qwen ≥0.90) and limited benefit from chain-of-thought prompting. This is attributed to possible overspecialization in surface empathy and data distribution mismatch [2507.23247].

## 6. Extensions, Critiques, and Best Practices

Domain-specialized instruction tuning enables state-of-the-art explainability, but limitations persist:
- MentaLLaMA’s professionality lags models trained with explicit clinical corpora or additional retrieval (e.g., PHQ-9, psychiatry notes). Automatic metrics (e.g., BART-score) only moderately correlate with human judgment. Broader instruction-tuning curricula may improve pragmatic inference [2507.23247].
- The architecture does not currently incorporate multi-modal context, longitudinal user modeling, or external retrieval. Future directions include continual pretraining on clinical guidelines and expansion to multi-modal and longitudinal signals [2309.13567].
- Ethical guidelines stress that MentaLLaMA-derived assistants (e.g., Sólo Escúchame for Spanish emotional support) are only supplemental and not substitutes for licensed professionals [2408.01852].

## 7. Open Resources and Community Impact

All MentaLLaMA code, checkpoint weights, and the IMHI dataset are publicly released for transparency and reproducibility. This positions MentaLLaMA as a research platform for computational psychologists, early-warning public health tools, and large-scale mental health discourse monitoring—always with clear human-readable rationales [2309.13567].

## References

- [MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models, arXiv:2309.13567](https://arxiv.org/abs/2309.13567)
- [Integrating Supervised Extractive and Generative Language Models for Suicide Risk Evidence Summarization, arXiv:2403.15478](https://arxiv.org/abs/2403.15478)
- [P-ReMIS: Pragmatic Reasoning in Mental Health and a Social Implication, arXiv:2507.23247](https://arxiv.org/abs/2507.23247)
- [Sólo Escúchame: Spanish Emotional Accompaniment Chatbot, arXiv:2408.01852](https://arxiv.org/abs/2408.01852)

Source: https://www.emergentmind.com/topics/mentallama