---
title: 'MentalQA: Arabic Mental Health QA Corpus'
url: https://www.emergentmind.com/topics/mentalqa
type: topic
---

# MentalQA: Arabic Mental Health QA Corpus

MentalQA is an annotated Arabic corpus of mental-health question–answer interactions and, subsequently, a benchmark track for Arabic mental-health question answering. It was introduced to address a specific resource gap: Arabic mental-health NLP had been dominated by one-way texts such as tweets, especially for depression detection, whereas practical support systems require data that reflects how patients ask for help and how professionals respond. The resource therefore centers on conversational patient–doctor exchanges and was framed around three tasks: question type classification, answer strategy classification, and, later, generative Arabic mental-health QA [2405.12619][2508.20047].

## 1. Origins, scope, and source material

MentalQA was constructed from **Altibbi.com**, an Arabic medical platform offering articles, glossaries, and question–answer services. The collection process drew on QA posts from **2020–2021** together with the platform’s most popular posts, initially yielding **53,402 unique QA pairs**. Filtering to the **Mental Health** section produced **2,621 unique question–answer pairs**, each consisting of one patient question and one doctor answer. At the corpus level, the paper reports **84 distinct doctors**, an average question length of **30 words**, and an average answer length of **31 words** [2405.12619].

The resource was explicitly positioned against prior Arabic work focused on social-media classification rather than interactive support. Its domain scope is broader than datasets centered on a single disorder such as depression, suicidality, self-harm, or anxiety. Instead, MentalQA covers multiple mental-health information needs and multiple professional response styles, which makes it applicable to intent recognition, answer retrieval, response-style modeling, and Arabic conversational support systems for early support, triage, and patient education [2405.12619].

A second stage in the resource’s history occurred when **AraHealthQA 2025** defined a dedicated **MentalQA track** as part of a shared task on Arabic health question answering. In that setting, MentalQA was presented as the benchmark component focused specifically on Arabic mental-health QA and described as targeting responses that are not only accurate, but also supportive, culturally aware, and contextually appropriate [2508.20047].

## 2. Corpus construction and annotation procedure

Although the filtered corpus contains **2,621** mental-health QA pairs, the annotation study covered **500 QA posts**, producing **500 question annotations + 500 answer annotations = 1,000 annotations** in the authors’ accounting. The annotators were **three** native Arabic speakers with experience in **biomedical text** and **natural language processing**. The paper does not state that they were clinicians or psychiatrists [2405.12619].

The annotation workflow was staged. It began with **small batches of 20 questions**, later increasing gradually to **100 questions** as familiarity improved. The first **20 questions** formed a **trial batch** annotated in parallel by all annotators and then reviewed for quality. For the first **200 questions**, all three annotators worked collaboratively on the same items; before each new batch, they held group meetings to discuss disagreements and document agreed resolutions. For the remaining **300 questions**, each annotator was assigned **100 questions independently**. Final labels were produced by **majority voting** [2405.12619].

The paper also records several procedural constraints and ambiguities. It confirms that **200 posts** were annotated by all three annotators, which implies that the reliability analysis was performed on the overlapping subset rather than the full 500-item annotation pool. It does not provide a train/dev/test split, benchmark models, tokenization details, normalization rules, or detailed preprocessing beyond filtering to the mental-health category and anonymization. For ethics and privacy, the authors state that personally identifiable information was removed, the dataset was anonymized, and data security measures were applied [2405.12619].

## 3. Annotation schema and task formulation

MentalQA’s schema combines and adapts a health-question taxonomy and answer-strategy labels from prior work. The original paper describes an internal inconsistency in the question taxonomy: the prose mentions an **Other** category and Table 1 reports **7 question categories**, while the abstract and main narrative foreground **6 core question types**. The later benchmarking paper operationalizes question classification as a **seven-label** multi-label task with **Diagnosis, Treatment, Anatomy, Epidemiology, Healthy Lifestyle, Health Provider Choice, and Other** [2405.12619][2406.15966].

| Layer | Labels | Notes |
|---|---|---|
| Question types | Diagnosis; Treatment; Anatomy and physiology; Epidemiology; Healthy lifestyle; Provider choice; Other | Original analysis emphasizes six core types; later benchmark uses seven labels |
| Answer strategies | Information; Direct Guidance; Emotional Support | Multi-label; strategy labels, not quality labels |

The six core question types in the original dataset paper are defined as follows. **Diagnosis** covers interpretation of symptoms, tests, disease manifestations, or diagnostic criteria. **Treatment** covers treatment options, medication use, side effects, contraindications, and how to use a drug. **Anatomy and physiology** covers basic medical knowledge such as tissues, organs, metabolism, or bodily and functional explanations relevant to mental-health concerns. **Epidemiology** covers disease course, prognosis, sequelae, etiology, causation, and risk factors. **Healthy lifestyle** covers diet, exercise, mood regulation, and related lifestyle factors. **Provider choice** covers recommendations about hospitals, departments, doctors, or the process of visiting and selecting a provider [2405.12619].

The answer taxonomy has three labels. **Information** includes factual information, explanations, resources, and even requests for more information from the patient; prior **Restatement** and **Interpretation** labels were merged into this category. **Direct Guidance** captures suggestions, instructions, and action-oriented advice. **Emotional Support** broadens a prior **Approval and Reassurance** label and includes reassurance and supportive care. **Self-disclosure** was excluded because all answers came from doctors rather than peers. The paper stresses that these labels characterize response strategy only; they do **not** evaluate clinical correctness, completeness, or whether the answer satisfied user needs [2405.12619].

## 4. Reliability, quantitative properties, and descriptive findings

Inter-annotator agreement was measured with **Fleiss’ Kappa**. The question-type value is consistent at **\(\kappa = 0.61\)**. For answer strategies, the paper contains an internal inconsistency: the abstract reports **\(\kappa = 0.98\)**, whereas the results section and Table 2 report **\(\kappa = 0.96\)**. Table 2 also reports **500 annotated questions**, **500 annotated answers**, **200 posts where all three annotators annotate**, **192** question posts where at least two annotators agreed on at least one category, **100** where all three agreed on at least one question category, **200** answer posts where at least two annotators agreed on at least one category, and **152** where all three agreed on at least one answer category. At the subset level, **82\%** of questions and **69\%** of answers had a single label, while **18\%** of questions and **31\%** of answers had multiple labels [2405.12619].

Among annotated posts where at least two annotators agreed, the most common question types were **Treatment: 296 (57\%)** and **Diagnosis: 286 (55\%)**. **Epidemiology** occurred **116 (22\%)**, **Healthy lifestyle** **125 (24\%)**, **Provider choices** **20 (4\%)**, and **Anatomy and physiology** **32** times. For answers, **Information** occurred **388 (75\%)**, **Direct guidance** **290 (56\%)**, and **Emotional support** **61 (11\%)**. Because MentalQA is multi-label, these percentages sum to more than 100 [2405.12619].

The paper adds several descriptive analyses. It reports a strong positive relationship between **Diagnosis** and **Treatment** questions and the answer strategies **Information** and **Direct Guidance**, while **Emotional Support** is especially associated with **Diagnosis** questions. For gender patterns, it reports that male patients most frequently asked **treatment** \((0.54)\) and **diagnosis** \((0.28)\) questions, while female patients most frequently asked **treatment** \((0.48)\) and **diagnosis** \((0.36)\) questions. The age analysis shows that treatment questions dominate across all groups, with reported values of **\(A = 0.34, B = 0.46\)** for users under 20, **\(A = 0.41, B = 0.52\)** for ages 20–30, **\(A = 0.28, B = 0.56\)** for ages 30–40, and **\(A = 0.00, B = 0.71, D = 0.07, E = 0.21\)** above 40, where \(A,B,D,E\) denote diagnosis, treatment, epidemiology, and healthy lifestyle respectively [2405.12619].

Exploratory sentiment analysis was performed with **CAMelBERT**. Patient questions were predominantly negative (**415** negative, **75** neutral, **10** positive), whereas doctor answers were more often neutral (**325** neutral, **139** negative, **36** positive). The paper also reports answer timing and length by question-type combinations, including **Diagnosis: 2.61 days, 22.33 words**, **Treatment: 11.73 days, 31.37 words**, and **Diagnosis + Treatment: 11.06 days, 47.30 words**, and concludes that diagnosis questions receive faster but shorter responses, treatment questions take longer and require more detail, and mixed-type questions often require longer replies [2405.12619].

## 5. Benchmarking studies and the AraHealthQA 2025 shared task

The first dedicated benchmark study on MentalQA treated both question classification and answer classification as **multi-label classification** tasks over a **60/20/20** split, yielding **300 training**, **100 validation**, and **100 test** examples for each task. It compared traditional sparse features with SVM, Arabic PLMs used as frozen feature extractors with SVM, fine-tuned Arabic PLMs, and prompted GPT models. The strongest reported question-classification result in Table 3 is **MARBERT** with **0.85 micro F1 / 0.85 weighted F1 / 0.80 Jaccard**. For answer classification, the best table result is again **MARBERT**, with **0.95 micro F1 / 0.95 weighted F1 / 0.94 Jaccard**. The same paper notes that classical sparse baselines are unexpectedly competitive on this small dataset, that fine-tuning is consistently beneficial, that more training data improves performance, and that few-shot prompting substantially improves over zero-shot, especially for answer classification. It also contains an internal inconsistency: the abstract states a best answer-classification Jaccard of **0.86**, whereas Table 3 reports **0.94** [2406.15966].

In **AraHealthQA 2025**, MentalQA became a shared-task benchmark with **three subtasks** over the same **500 question-answer pairs**, split into **300 training**, **50 development**, and **150 test** examples for all subtasks. Subtask 1 is **Question Type Classification** and Subtask 2 is **Answer Strategy Classification**, both evaluated by **Weighted F1** and **Jaccard score**. Subtask 3 is generative **Question Answering**, evaluated by **BERTScore**. Evaluation ran on **Codabench** with a **blind test set**, up to **five runs per subtask**, and only the highest-scoring submission per team counted. Track 1 registered **46 unique teams**, with **9 submissions from 9 teams** for Subtask 1, **7 submissions from 7 teams** for Subtask 2, **6 submissions from 6 teams** for Subtask 3, and **10 system description papers** accepted [2508.20047].

Official shared-task results show that question-type classification is the hardest of the three tasks: **mucAI** ranked first on Subtask 1 with **0.61 Weighted-F1** and **0.53 Jaccard**. Answer-strategy classification was stronger overall, with **Sindbad** and **MarsadLab** tied at **0.79 Weighted-F1**, while **Sindbad** and **Binary_Bunch** shared the highest **0.71 Jaccard**. For generative QA, **RetAug** ranked first with **0.679 BERTScore**, ahead of **MindLLM** at **0.670** and **Sindbad** at **0.668**. The shared-task paper emphasizes that retrieval augmentation, prompt engineering, instruction tuning, and hybrid pipelines were effective, but also that automatic semantic-overlap metrics do not fully capture cultural sensitivity, empathy, safety, or contextual appropriateness in mental-health QA [2508.20047].

## 6. Limitations, safety considerations, and place in the broader landscape

MentalQA’s limitations are explicit in the original dataset paper. The data come from a single consumer health platform, so the source population is not representative of everyone with mental-health concerns. Only **500 of 2,621** filtered QA pairs were annotated. The labels characterize communicative strategy rather than answer quality. The paper also leaves several reporting inconsistencies unresolved, including the **6-versus-7** question-category issue, the **0.96-versus-0.98** answer-strategy agreement issue, and the fact that only **200** posts are clearly described as triple-annotated even though **500** posts are described as annotated [2405.12619].

The shared-task paper extends the safety discussion. It states that the datasets were curated from **publicly available or anonymized sources**, warns that models may still generate **misleading, unsafe, or culturally inappropriate responses**, and argues that systems should be treated as **research artifacts rather than clinical advice**. It recommends future **human-in-the-loop evaluation** with clinicians and native speakers, along with stronger bias evaluation and safety guardrails [2508.20047]. A later auditing study on mental-health QA reinforces that caution by arguing that LLMs in mental-health settings can amplify stigma, suppress nuance, and differentially frame symptoms and treatment across demographic intersections; it treats multi-hop QA as a way to expose such “amplified” and “silenced” perspectives in mental-health discourse [2506.18116].

Within the broader landscape of mental-health QA resources, MentalQA occupies a distinct niche. Unlike **MHQA**, which is a multiple-choice benchmark built from PubMed abstracts across anxiety, depression, trauma, and obsessive/compulsive issues, MentalQA is grounded in real patient–doctor Arabic exchanges [2502.15418]. Unlike **MENTAT**, which is a clinician-authored dataset of psychiatry decision-making spanning treatment, diagnosis, documentation, monitoring, and triage, MentalQA focuses on support-oriented Arabic QA and communicative intent at the question and answer levels [2502.16051]. Its significance therefore lies less in exam-style knowledge testing than in providing an Arabic, conversational, professionally answered substrate for research on classification, retrieval, generation, and safety-sensitive benchmarking in mental-health communication [2405.12619][2508.20047].

Source: https://www.emergentmind.com/topics/mentalqa