Mental Health Disorder Detection Beyond Social Media: A Systematic Review of Available Datasets

Abstract: Detecting mental health disorders in a timely manner is an important societal challenge. NLP and ML methods used to assist with detection rely on data collected primarily from social media. However, such datasets often have sampling biases and inherent ethical and privacy issues. One avenue to overcome these limitations is non-social media data. We present the first comprehensive review of non-social media, free-text datasets for mental health research. We use the PRISMA methodology to conduct our survey and we review datasets available in multiple languages. We find that non-social media free-text based datasets are predominantly focused on English and on detecting depression. These datasets also vary in demographics, platforms, data types, annotation techniques, and methodologies. This systematic review also reveals key gaps and highlights opportunities to develop more diverse, reliable and clinically-relevant resources.

• The paper systematically reviews 45 non-social media datasets used for mental health disorder detection, revealing significant linguistic and annotation imbalances.
• It identifies a concentration in depression studies and English data, underscoring the need for more multilingual and disorder-diverse collections.
• It discusses the benefits and limitations of various annotation methods and computational models, urging standardized and transparent labeling practices.

Systematic Review of Non-Social Media Mental Health Disorder Datasets

Motivation and Scope

The paper "Mental Health Disorder Detection Beyond Social Media: A Systematic Review of Available Datasets" (2607.03540) addresses the critical limitations inherent to social media-derived datasets in mental health NLP—namely, demographic bias, privacy concerns, construct validity, and restricted API access. By systematically cataloguing and analyzing free-text datasets from clinical and other reliable sources (excluding social media), the authors illuminate a domain largely unexplored due to access and annotation complexities. Using the PRISMA methodology, the review encompasses 45 datasets spanning 2004–2025, multiple languages, and a spectrum of mental health conditions, emphasizing diversity in data source, form, annotation procedure, and population.

Figure 1: PRISMA flowchart illustrating the systematic literature search and screening process.

Dataset Distribution Analysis

Temporal and Linguistic Trends

Dataset publication rates remained steady until a marked increase post-2014, peaking in 2022 likely due to exacerbated mental health concerns following COVID-19. Dataset collection remains inherently difficult due to ethics reviews and resource-intensive annotation requirements.

Figure 2: Distribution of datasets by year.

Linguistically, English datasets dominate (62.2%), followed by Chinese (17.8%), with minimal representation in Korean, Polish, Greek, Japanese, Thai, Arabic, and code-mixed formats. The latter constitute approximately 6.4%, underscoring a substantial deficit in low-resource and non-English datasets.

Figure 3: Distribution of datasets into by language groups.

Disorder Focus and Platform Analysis

Depression-related disorders are most prevalent, with 33 datasets focusing on various forms (DP, MDD, MinDP, PD). Anxiety and suicidal ideation are also represented, but schizophrenia, PTSD, bipolar disorder, adjustment disorder, and dementia are notably underrepresented. The disorder-to-language mapping exhibits an overwhelming concentration of studies on depression in English.

Figure 4: Heatmap showing the distribution of datasets across language groups and disorders.

Datasets originate primarily from clinical settings (interviews, EHRs, discharge summaries), forums (user posts), and mixed/other platforms (apps, online chats, therapy sessions, suicide notes). Clinical datasets favor interviews and EHRs, while forums predominantly contain user-generated posts.

Figure 5: Distribution of data types across platforms.

Availability and Impact Metrics

Datasets requiring Data Use Agreements (DUA) or classified as Restricted (RSTR) exhibit significantly higher normalized citation rates than those publicly available (PUB) or with unknown status (UNK). English DUA/RSTR datasets are generally depression-focused and clinically sourced, enhancing impact and credibility. Chinese datasets, though more publicly accessible, attract lower citation rates, potentially reflecting linguistic biases.

Figure 6: Distribution of citation counts by dataset availability type.

Figure 7: Availability of datasets across language groups.

Demographic Representation

Demographic annotation focuses primarily on gender and age (>24 datasets), with moderate inclusion of education, race, and ethnicity. Attributes such as income, relationship status, and health indicators are rarely included. Some studies use demographic matching for negative controls, improving minority class performance—contrasting with the typically poor demographic balancing seen in social media datasets.

Figure 8: Frequency of demographic attributes across datasets.

Annotation Procedures and Clinical Labeling

The review reveals significant heterogeneity in annotation strategies:

• Clinical Diagnostic Instruments: DSM-IV, DSM-V, ICD-9/10, LIFE, CAPS, HAMD, HAMA, MADRS, often deployed by trained clinicians in structured settings.
• Screening Questionnaires/Self-Report Scales: PHQ-9/8, BDI-II, IDD-L, BAI, BSS, EPDS, QIDS-SR, GAD-7; efficient and scalable but susceptible to recall/social desirability bias.
• Manual/Contextual Labeling: Labels generated via trained annotators, inter-rater agreement metrics (Cohen’s/Fleiss’s Kappa), or NLP-based methods (Text Hunter, MALLET, LIWC, keyword search). Some datasets lack explicit documentation of labeling procedures, undermining reproducibility.
• Clinical Implications: Diagnostic instrument labeling ensures validity but is resource-intensive and time-constrained; self-report tools facilitate large-scale screening but should be considered supplementary due to bias susceptibility. Combining techniques yields comprehensive annotation.

Computational Modeling Trends

• Initial modeling applied traditional ML (SVM, LR, DT, NB, Adaboost, XGBoost), supported by feature engineering (LIWC, tf-idf, keyword extraction).
• Deep learning (CNN, LSTM, GRU) and hybrid systems (CRF, TextHunter+ConText) later gained prominence.
• Recent work leverages transformer architectures (Clinical-BigBird, MentalBERT, MentalRoBERTa) and LLMs (Qwen2-72B), adopting instruction-based prompt tuning and chain-of-thought/empathy frameworks to automate diagnostic annotation and enhance label consistency.

Identified Gaps and Future Directions

• Language and Disorder Imbalances: Overrepresentation of English/depression datasets; negligible coverage in low-resource languages or for disorders like schizophrenia, PTSD, or eating disorders. This restricts generalizability and exacerbates bias.
• Insufficient Clinical Dataset Release: Scarcity of public clinical datasets impedes reproducibility and collaborative research. Strict governance (DUA, RSTR) is prevalent, but public datasets are primarily non-English and seldom depression-focused.
• Opaque/Inconsistent Annotation Reporting: Many datasets lack explicit rationale for tool selection or labeling strategies, complicating cross-study comparison and validation.
• Opportunity for NLP Innovations: Standardized, semi-automated labeling procedures using LLMs and structured prompting (CoT, CoE) promise improved scalability, interpretability, and domain-specific accuracy, especially when clinical input is limited.

Conclusion

This systematic review rigorously maps the domain of non-social media, free-text datasets for mental health disorder detection and identifies significant linguistic, disorder, and annotation-related imbalances. The findings suggest a pressing need for multilingual dataset development, wider disorder inclusion, consistent and transparent labeling standards, and greater dataset accessibility governed by robust ethical frameworks. Advances in explainable NLP and LLM-guided annotation can address existing deficiencies, supporting scalable, trustworthy, and clinically relevant mental health modeling.

Implications

Strategic expansion in dataset collection—especially in low-resource languages and underrepresented disorders—will enhance the external validity of mental health NLP systems. The adoption of advanced, transparent annotation techniques and structured prompting within LLMs can mitigate labeling inconsistencies and facilitate real-world deployment. High-quality, accessible datasets will serve as benchmarks for robust, reproducible clinical NLP, with broader impact for mental health screening, diagnosis, and intervention.

Conclusion

The review establishes a foundation for future research at the intersection of clinical NLP and mental health, highlighting critical gaps in dataset diversity and annotation rigor. Comprehensive, multilingual, and diagnostically broad dataset development, partnered with standardized and explainable annotation, will be pivotal in advancing the field towards practical and equitable mental health AI.