---
title: 'REDDME: Reddit Mental Health Support Dataset'
url: https://www.emergentmind.com/topics/reddme
type: topic
---

# REDDME: Reddit Mental Health Support Dataset

Searching arXiv for REDDME and closely related Reddit mental-health NLP work.
REDDME is a Reddit-based dataset for modeling under-specified support-seeking language in online mental health communities (OMHCs). It is introduced as “a manually annotated corpus extended from the publicly available mental health subreddit corpus, BeCOPE,” and is designed to operationalize whether a post clearly expresses three support attributes: **event** (“what happened”), **effect** (“what did the user experience”), and **requirement** (“what support they need”) [2508.16788]. In the associated framework, REDDME is not only a static annotation resource but the training substrate for a pipeline that identifies missing or weakly expressed support cues and generates targeted follow-up questions intended to improve post articulateness and, plausibly, engagement within peer-support settings [2508.16788].

## 1. Dataset scope and source corpus

REDDME is built from Reddit OMHC posts drawn from the interactive subset of BeCOPE. The appendix specifies that the corpus uses interactive posts from **17 subreddits**, yielding exactly **4,760 posts** in the final dataset [2508.16788]. The subreddit inventory is: **r/Anxiety (469 posts), r/ptsd (494), r/addiction (487), r/ADHD (423), r/alcoholicsanonymous (498), r/Anger (464), r/BPD (519), r/depression (547), r/domesticviolence (425), r/getting_over_it (476), r/mentalillness (484), r/OpiatesRecovery (493), r/rapecounseling (481), r/sad (486), r/selfharm (467), r/selfhelp (419), and r/socialanxiety (461)** [2508.16788].

The paper states that REDDME “select[s] 4,760 posts and manually annotate[s] them,” but does not provide an additional sampling heuristic beyond restricting attention to those interactive posts from the listed subreddits [2508.16788]. The final split is **70:20:10**, corresponding to **3,331 train**, **953 validation**, and **476 test** posts [2508.16788].

The appendix describes the dataset fields as including **post title**, **post body**, **annotated post body**, **event-intensity**, **effect-intensity**, **requirement-intensity**, **event-question**, **effect-question**, and **requirement-question** [2508.16788]. This makes REDDME simultaneously a span-annotation corpus, an ordinal-intensity corpus, and a controlled question-generation corpus.

## 2. Annotation schema: support attributes, spans, and CueTaxo

The annotation scheme centers on three support attributes. **Event** is defined as “the specific situation, activity, or event that is the focal point of the support seeker’s concern.” **Effect** is “the impact or consequences of the identified event from the support seeker.” **Requirement** captures what support the user is seeking from peers, such as informational or instrumental help [2508.16788]. In the appendix example, the event span describes taking Adderall and coffee together, the effect span describes becoming “extremely jittery and anxious,” and the requirement span asks for “tips” about whether the cause is flu, Adderall, or caffeine [2508.16788].

Span annotation is performed directly in the post text using explicit rationale markers: **event** as `⟨es⟩ ... ⟨ee⟩`, **effect** as `⟨efs⟩ ... ⟨efe⟩`, and **requirement** as `⟨rs⟩ ... ⟨re⟩` [2508.16788]. Because Reddit writing is often informal, with poor grammar, inconsistent punctuation, and missing sentence boundaries, the paper states that annotation is done at the **token level**, while preferring complete sentences when possible [2508.16788]. If a long sentence expresses multiple attributes, annotators are instructed to split it using suitable punctuation and annotate the subparts separately; titles are included if they provide important context [2508.16788].

Each attribute also receives a 3-point intensity label: **absent (0)**, **moderately present / moderate (1)**, and **well-described / present (2)** [2508.16788]. The paper formalizes this as an intensity vector
$$
V = \{ v_\text{event}, v_\text{effect}, v_\text{req} \}, \quad v_i \in \{0,1,2\}.
$$
This framing treats support articulation as an ordinal property rather than a binary one: a post may mention an attribute yet still under-specify it [2508.16788].

A third layer links the annotations to intervention via **CueTaxo**, a five-level taxonomy over combinations of support-attribute completeness [2508.16788]. The nodes reported are **1A, 2A, 2B, 2C, 3A, 3B, 3C, 4A**, while **5A** is omitted because no question is generated when all attributes are present [2508.16788]. CueTaxo connects missing or moderate attributes to controlled prompt templates such as **“Can you tell me what happened? You can be as specific as you like.”**, **“Could you describe the specific effect the event has had on you?”**, and **“What kind of support or help you feel would be most beneficial?”** [2508.16788]. The placeholder \(X\) is instantiated from a user-mentioned entity extracted from the post or rationale, allowing context-sensitive follow-up even when the target attribute is absent [2508.16788].

## 3. Corpus statistics and empirical structure

The corpus-level statistics indicate that event information is relatively better represented than effect or requirement information, but that substantial under-specification remains across all three attributes [2508.16788]. Overall REDDME contains **4,760 posts**, **7,909 prompts/questions**, and an **average post length (APoL) of 179.62 words** [2508.16788].

The attribute distributions and average span lengths are as follows:

| Attribute | Intensity counts | Average span length |
|---|---:|---:|
| Event | 1,340 absent / 812 moderate / 2,608 present | 65.70 words |
| Effect | 2,190 absent / 591 moderate / 1,979 present | 26.48 words |
| Requirement | 2,461 absent / 515 moderate / 1,784 present | 19.31 words |

These counts imply **2,125** posts with event absent or moderate, **2,781** with effect absent or moderate, and **2,976** with requirement absent or moderate [2508.16788]. The appendix interprets this as showing that support seekers mainly narrate triggering incidents while giving substantially less detail about consequences and desired help [2508.16788]. The paper also notes that post-body length shows an increasing trend as cue intensity increases for each attribute [2508.16788].

Split-specific statistics preserve the same overall pattern. The train set has **5,533 prompts** for **3,331 posts**; the validation set has **1,583 prompts** for **953 posts**; the test set has **793 prompts** for **476 posts** [2508.16788]. Requirement remains the sparsest attribute across all three splits [2508.16788].

The annotation process used **two annotators aged 21–24 with expertise in linguistics and the relevant domain**, along with an expert reviewer or moderator who periodically evaluated quality [2508.16788]. The workflow included initial training, calibration sessions, and discussions to align interpretations. Cohen’s kappa was computed on a random sample of 50 instances, yielding **0.885** for event, **0.886** for effect, and **1.000** for requirement [2508.16788]. The paper does not provide separate agreement for span boundaries versus intensities, but the reported values indicate high consistency at the level it evaluates.

Figure 3 analyzes the distribution of the median number of comments by support-attribute level and against the sum of event, effect, and requirement levels [2508.16788]. Exact numeric values are not printed in the text, but this suggests that richer articulation of these support attributes is associated with greater engagement.

## 4. REDDME as the substrate for MH-COPILOT

REDDME is presented as the training substrate for **MH-COPILOT**, a framework with four modules: **contextual span identification**, **support attribute intensity classification**, **controlled question generation via CueTaxo**, and a **verifier for reward modeling** [2508.16788].

The first module, **CSpan**, treats attribute extraction as an entity-extraction problem over tokens
$$
\{t_1,t_2,\dots,t_n\}
$$
with corresponding labels
$$
\{a_1,a_2,\dots,a_n\},
$$
where each label is one of event, effect, or requirement [2508.16788]. Among the compared models, **RoBERTa-large** is the best span extractor, with **Accuracy 73.00**, **Precision 45.29**, **Recall 62.50**, and **F1 52.52** [2508.16788]. The paper compares it with T5-Large, SpanBERT, and RoBERTa-base [2508.16788].

The second module predicts the ordinal attribute-intensity vector \(V\). This is framed as **multi-class, multi-label classification with ordinal labels**, and the paper compares BERT, MentalBERT, MentalRoBERTa, and RoBERTa under both ordinal loss and mean squared error variants [2508.16788]. **RoBERTa with ordinal loss** is the strongest reported model, with **Accuracy 86.60** and **F1 77.40** [2508.16788].

The third module is controlled question generation. The paper fine-tunes **Llama-3-instruct (8B)**, **Phi-3-mini-4k-instruct (3.8B)**, **Mistral-instruct-v0.1 (7B)**, and **Gemma-2-instruct (2B)** on REDDME’s reference questions using **QLoRA** [2508.16788]. CueTaxo is embedded in the prompting structure, and the user prompt requires output in the schema
$$
\{event\_question: , effect\_question: , requirement\_question: \}.
$$
The model is instructed to “Generate 3 questions following the schema... Strictly follow the question format of schema. Give only the json output” [2508.16788].

The fourth module is the verifier and reward-modeling stage, which scores generated questions for **support attribute category correctness**, **contextual grounding**, **structural adherence**, and **empathy** [2508.16788]. The reward is defined as
$$
r = \sum_{i \in [1,2,3]} r_{Q_i}^{CC} \cdot r_{Q_i}^{CG} \cdot r_{Q_i}^{EA} \cdot r_{SA}
$$
where \(Q_1, Q_2, Q_3\) are the event, effect, and requirement questions, and the factors denote category correctness, contextual grounding, empathy assessment, and structural adherence [2508.16788]. Preference optimization then uses
$$
\mathcal{D}=\{x^{(i)},y_p^{(i)},y_{np}^{(i)}\}
$$
with DPO:
$$
F=\beta \log \frac{\pi_\theta(y_p|x)}{\pi_{ref}(y_p|x)}- \beta \frac{\pi_\theta(y_{np}|x)}{\pi_{ref}(y_{np}|x)}
$$
and
$$
\mathcal{L}_{DPO}(\pi_\theta;\pi_{ref})=-E_{(y_p,y_{np}) \sim D} [\log \sigma (F)].
$$
The paper says “Hinge-loss is used,” while the displayed objective is the standard log-sigmoid DPO form; this is a reporting inconsistency in the manuscript itself [2508.16788].

## 5. Empirical performance and evaluation

The generation experiments compare zero-shot prompting, supervised fine-tuning, and the full MH-COPILOT setup [2508.16788]. In zero-shot evaluation, Mistral and Gemma-2 are the stronger baselines on several metrics; in supervised fine-tuning, **Mistral** is strongest on several reported automatic scores, including **ROUGE-L 69.71**, **BLEU-4 76.21**, **BERTScore-F1 96.04**, and **METEOR 79.83** [2508.16788].

The full **MH-COPILOT** system, built on **Gemma-2 + SFT + CueTaxo + reward**, reaches **ROUGE-1 89.30**, **ROUGE-2 84.50**, **ROUGE-L 88.88**, **BLEU-1 93.84**, **BLEU-2 92.36**, **BLEU-3 91.12**, **BLEU-4 89.78**, **BERTScore P 98.81**, **R 98.68**, **F1 98.74**, and **METEOR 93.84** [2508.16788]. Relative to the best SFT baseline, the reported gains are **+23.35% R1**, **+35.74% R2**, **+27.49% RL**, **+14.02% B1**, **+15.45% B2**, **+16.64% B3**, **+17.80% B4**, **+2.34% BS-P**, **+3.24% BS-R**, **+2.81% BS-F1**, and **+17.54% METEOR** [2508.16788].

The ablation study clarifies the role of REDDME’s structured supervision. For the Gemma-based configuration, the **full model** reaches **ROUGE-L 88.88**, **BLEU-4 89.78**, **BERTScore-F1 98.74**, and **METEOR 93.84**; **without reward**, these drop modestly to **87.67**, **88.32**, **98.52**, and **93.33**; **without reward and without CueTaxo**, the scores fall much more sharply to **65.58**, **74.00**, **95.48**, and **76.98** [2508.16788]. The paper therefore attributes the largest quantitative jump to CueTaxo, while the verifier-based reward mainly improves generation quality in a more targeted way [2508.16788].

A representative pipeline example also reveals a failure mode. In a post about taking Adderall and coffee together, human annotation marks **Event = Moderate**, **Effect = Well-described**, and **Requirement = Well-described**, with the gold follow-up asking only **“Can you elaborate more on why you are taking adderall?”** [2508.16788]. MH-COPILOT instead predicts **Effect = Moderate** as well and generates an unnecessary effect question. The appendix explicitly notes that errors in early modules propagate into later generations [2508.16788].

Human evaluation on **40 random instances** compared MH-COPILOT with and without the verifier. The paper reports improvements on domain-centric metrics and linguistic metrics alike: without verifier, **D1 3.27**, **D2 1.82**, **D3 2.19**, **L1 3.46**, **L2 3.70**, **L3 3.82**, **L4 3.80**; with verifier, **D1 3.43**, **D2 2.27**, **D3 3.31**, **L1 3.62**, **L2 3.84**, **L3 4.02**, **L4 3.89** [2508.16788]. The appendix says D1–D3 were scored on a 1–3 scale, yet D1 exceeds 3 in the table, which indicates another reporting inconsistency [2508.16788].

## 6. Interpretation, limitations, and relation to adjacent Reddit resources

REDDME is best understood as a dataset for **support articulation**, not for generic emotion recognition or clinical diagnosis [2508.16788]. Its labels target the structure of self-disclosure—event, effect, and requirement—and the associated question layer is explicitly designed to elicit missing or weakly expressed information. This makes it distinct from adjacent Reddit datasets that annotate emotional states themselves. For example, **ReDDIT** formulates multiclass regret detection and domain identification over **Regret by Action**, **Regret by Inaction**, and **No Regret**, plus domains such as **Romance and Relationships**, **Education**, and **Health** [2212.07549]. REDDME, by contrast, is organized around support cues within OMHC posts rather than affect categories [2508.16788].

Several limitations are explicit. First, the corpus is derived from one source dataset and one platform, and specifically from the **interactive** subset of BeCOPE [2508.16788]. Second, the paper relies on public Reddit content but does not describe a substantive de-identification protocol, and privacy is not discussed in depth beyond public-subreddit sourcing and research release [2508.16788]. Third, annotator well-being is only indirectly addressed, despite the sensitive domains involved [2508.16788]. Fourth, the framework is vulnerable to error propagation from span extraction to intensity prediction to generation [2508.16788]. Fifth, the paper itself contains reporting inconsistencies, including the DPO “hinge-loss” description and the human-evaluation scale mismatch [2508.16788].

The broader methodological implication is that REDDME converts a diffuse engagement problem into a structured NLP problem. As a **measurement dataset**, it defines what “better articulation” means in OMHC posts; as a **generation dataset**, it supports controlled prompt construction for absent or underdeveloped support attributes [2508.16788]. This suggests a reusable template for future work on support-oriented moderation, domain-conditioned self-disclosure modeling, and intervention systems that improve post informativeness without attempting diagnosis or treatment.

Source: https://www.emergentmind.com/topics/reddme