REDDME: Reddit Mental Health Support Dataset

• The paper introduces REDDME, a dataset that systematically annotates Reddit posts using support attributes (event, effect, requirement) to guide intervention strategies.
• It comprises 4,760 interactive posts from 17 subreddits with token-level annotations and ordinal intensity ratings, capturing nuanced support articulation.
• REDDME underpins MH-COPILOT, a multi-module framework that extracts attributes, generates controlled follow-up questions, and benchmarks state-of-the-art models.

Searching arXiv for REDDME and closely related Reddit mental-health NLP work. REDDME is a Reddit-based dataset for modeling under-specified support-seeking language in online mental health communities (OMHCs). It is introduced as “a manually annotated corpus extended from the publicly available mental health subreddit corpus, BeCOPE,” and is designed to operationalize whether a post clearly expresses three support attributes: event (“what happened”), effect (“what did the user experience”), and requirement (“what support they need”) (Gaur et al., 22 Aug 2025). In the associated framework, REDDME is not only a static annotation resource but the training substrate for a pipeline that identifies missing or weakly expressed support cues and generates targeted follow-up questions intended to improve post articulateness and, plausibly, engagement within peer-support settings (Gaur et al., 22 Aug 2025).

1. Dataset scope and source corpus

REDDME is built from Reddit OMHC posts drawn from the interactive subset of BeCOPE. The appendix specifies that the corpus uses interactive posts from 17 subreddits, yielding exactly 4,760 posts in the final dataset (Gaur et al., 22 Aug 2025). The subreddit inventory is: r/Anxiety (469 posts), r/ptsd (494), r/addiction (487), r/ADHD (423), r/alcoholicsanonymous (498), r/Anger (464), r/BPD (519), r/depression (547), r/domesticviolence (425), r/getting_over_it (476), r/mentalillness (484), r/OpiatesRecovery (493), r/rapecounseling (481), r/sad (486), r/selfharm (467), r/selfhelp (419), and r/socialanxiety (461) (Gaur et al., 22 Aug 2025).

The paper states that REDDME “select[s] 4,760 posts and manually annotate[s] them,” but does not provide an additional sampling heuristic beyond restricting attention to those interactive posts from the listed subreddits (Gaur et al., 22 Aug 2025). The final split is 70:20:10, corresponding to 3,331 train, 953 validation, and 476 test posts (Gaur et al., 22 Aug 2025).

The appendix describes the dataset fields as including post title, post body, annotated post body, event-intensity, effect-intensity, requirement-intensity, event-question, effect-question, and requirement-question (Gaur et al., 22 Aug 2025). This makes REDDME simultaneously a span-annotation corpus, an ordinal-intensity corpus, and a controlled question-generation corpus.

2. Annotation schema: support attributes, spans, and CueTaxo

The annotation scheme centers on three support attributes. Event is defined as “the specific situation, activity, or event that is the focal point of the support seeker’s concern.” Effect is “the impact or consequences of the identified event from the support seeker.” Requirement captures what support the user is seeking from peers, such as informational or instrumental help (Gaur et al., 22 Aug 2025). In the appendix example, the event span describes taking Adderall and coffee together, the effect span describes becoming “extremely jittery and anxious,” and the requirement span asks for “tips” about whether the cause is flu, Adderall, or caffeine (Gaur et al., 22 Aug 2025).

Span annotation is performed directly in the post text using explicit rationale markers: event as ⟨es⟩ ... ⟨ee⟩, effect as ⟨efs⟩ ... ⟨efe⟩, and requirement as ⟨rs⟩ ... ⟨re⟩ (Gaur et al., 22 Aug 2025). Because Reddit writing is often informal, with poor grammar, inconsistent punctuation, and missing sentence boundaries, the paper states that annotation is done at the token level, while preferring complete sentences when possible (Gaur et al., 22 Aug 2025). If a long sentence expresses multiple attributes, annotators are instructed to split it using suitable punctuation and annotate the subparts separately; titles are included if they provide important context (Gaur et al., 22 Aug 2025).

Each attribute also receives a 3-point intensity label: absent (0), moderately present / moderate (1), and well-described / present (2) (Gaur et al., 22 Aug 2025). The paper formalizes this as an intensity vector

$$V = \{ v_\text{event}, v_\text{effect}, v_\text{req} \}, \quad v_i \in \{0,1,2\}.$$

This framing treats support articulation as an ordinal property rather than a binary one: a post may mention an attribute yet still under-specify it (Gaur et al., 22 Aug 2025).

A third layer links the annotations to intervention via CueTaxo, a five-level taxonomy over combinations of support-attribute completeness (Gaur et al., 22 Aug 2025). The nodes reported are 1A, 2A, 2B, 2C, 3A, 3B, 3C, 4A, while 5A is omitted because no question is generated when all attributes are present (Gaur et al., 22 Aug 2025). CueTaxo connects missing or moderate attributes to controlled prompt templates such as “Can you tell me what happened? You can be as specific as you like.”, “Could you describe the specific effect the event has had on you?”, and “What kind of support or help you feel would be most beneficial?” (Gaur et al., 22 Aug 2025). The placeholder $X$ is instantiated from a user-mentioned entity extracted from the post or rationale, allowing context-sensitive follow-up even when the target attribute is absent (Gaur et al., 22 Aug 2025).

3. Corpus statistics and empirical structure

The corpus-level statistics indicate that event information is relatively better represented than effect or requirement information, but that substantial under-specification remains across all three attributes (Gaur et al., 22 Aug 2025). Overall REDDME contains 4,760 posts, 7,909 prompts/questions, and an average post length (APoL) of 179.62 words (Gaur et al., 22 Aug 2025).

The attribute distributions and average span lengths are as follows:

Attribute	Intensity counts	Average span length
Event	1,340 absent / 812 moderate / 2,608 present	65.70 words
Effect	2,190 absent / 591 moderate / 1,979 present	26.48 words
Requirement	2,461 absent / 515 moderate / 1,784 present	19.31 words

These counts imply 2,125 posts with event absent or moderate, 2,781 with effect absent or moderate, and 2,976 with requirement absent or moderate (Gaur et al., 22 Aug 2025). The appendix interprets this as showing that support seekers mainly narrate triggering incidents while giving substantially less detail about consequences and desired help (Gaur et al., 22 Aug 2025). The paper also notes that post-body length shows an increasing trend as cue intensity increases for each attribute (Gaur et al., 22 Aug 2025).

Split-specific statistics preserve the same overall pattern. The train set has 5,533 prompts for 3,331 posts; the validation set has 1,583 prompts for 953 posts; the test set has 793 prompts for 476 posts (Gaur et al., 22 Aug 2025). Requirement remains the sparsest attribute across all three splits (Gaur et al., 22 Aug 2025).

The annotation process used two annotators aged 21–24 with expertise in linguistics and the relevant domain, along with an expert reviewer or moderator who periodically evaluated quality (Gaur et al., 22 Aug 2025). The workflow included initial training, calibration sessions, and discussions to align interpretations. Cohen’s kappa was computed on a random sample of 50 instances, yielding 0.885 for event, 0.886 for effect, and 1.000 for requirement (Gaur et al., 22 Aug 2025). The paper does not provide separate agreement for span boundaries versus intensities, but the reported values indicate high consistency at the level it evaluates.

Figure 1 analyzes the distribution of the median number of comments by support-attribute level and against the sum of event, effect, and requirement levels (Gaur et al., 22 Aug 2025). Exact numeric values are not printed in the text, but this suggests that richer articulation of these support attributes is associated with greater engagement.

4. REDDME as the substrate for MH-COPILOT

REDDME is presented as the training substrate for MH-COPILOT, a framework with four modules: contextual span identification, support attribute intensity classification, controlled question generation via CueTaxo, and a verifier for reward modeling (Gaur et al., 22 Aug 2025).

The first module, CSpan, treats attribute extraction as an entity-extraction problem over tokens

$\{t_1,t_2,\dots,t_n\}$

with corresponding labels

$\{a_1,a_2,\dots,a_n\},$

where each label is one of event, effect, or requirement (Gaur et al., 22 Aug 2025). Among the compared models, RoBERTa-large is the best span extractor, with Accuracy 73.00, Precision 45.29, Recall 62.50, and F1 52.52 (Gaur et al., 22 Aug 2025). The paper compares it with T5-Large, SpanBERT, and RoBERTa-base (Gaur et al., 22 Aug 2025).

The second module predicts the ordinal attribute-intensity vector $V$. This is framed as multi-class, multi-label classification with ordinal labels, and the paper compares BERT, MentalBERT, MentalRoBERTa, and RoBERTa under both ordinal loss and mean squared error variants (Gaur et al., 22 Aug 2025). RoBERTa with ordinal loss is the strongest reported model, with Accuracy 86.60 and F1 77.40 (Gaur et al., 22 Aug 2025).

The third module is controlled question generation. The paper fine-tunes Llama-3-instruct (8B), Phi-3-mini-4k-instruct (3.8B), Mistral-instruct-v0.1 (7B), and Gemma-2-instruct (2B) on REDDME’s reference questions using QLoRA (Gaur et al., 22 Aug 2025). CueTaxo is embedded in the prompting structure, and the user prompt requires output in the schema

$$\{event\_question: , effect\_question: , requirement\_question: \}.$$

The model is instructed to “Generate 3 questions following the schema... Strictly follow the question format of schema. Give only the json output” (Gaur et al., 22 Aug 2025).

The fourth module is the verifier and reward-modeling stage, which scores generated questions for support attribute category correctness, contextual grounding, structural adherence, and empathy (Gaur et al., 22 Aug 2025). The reward is defined as

$$r = \sum_{i \in [1,2,3]} r_{Q_i}^{CC} \cdot r_{Q_i}^{CG} \cdot r_{Q_i}^{EA} \cdot r_{SA}$$

where $Q_1, Q_2, Q_3$ are the event, effect, and requirement questions, and the factors denote category correctness, contextual grounding, empathy assessment, and structural adherence (Gaur et al., 22 Aug 2025). Preference optimization then uses

$\mathcal{D}=\{x^{(i)},y_p^{(i)},y_{np}^{(i)}\}$

with DPO:

$$F=\beta \log \frac{\pi_\theta(y_p|x)}{\pi_{ref}(y_p|x)}- \beta \frac{\pi_\theta(y_{np}|x)}{\pi_{ref}(y_{np}|x)}$$

and

$X$0

The paper says “Hinge-loss is used,” while the displayed objective is the standard log-sigmoid DPO form; this is a reporting inconsistency in the manuscript itself (Gaur et al., 22 Aug 2025).

5. Empirical performance and evaluation

The generation experiments compare zero-shot prompting, supervised fine-tuning, and the full MH-COPILOT setup (Gaur et al., 22 Aug 2025). In zero-shot evaluation, Mistral and Gemma-2 are the stronger baselines on several metrics; in supervised fine-tuning, Mistral is strongest on several reported automatic scores, including ROUGE-L 69.71, BLEU-4 76.21, BERTScore-F1 96.04, and METEOR 79.83 (Gaur et al., 22 Aug 2025).

The full MH-COPILOT system, built on Gemma-2 + SFT + CueTaxo + reward, reaches ROUGE-1 89.30, ROUGE-2 84.50, ROUGE-L 88.88, BLEU-1 93.84, BLEU-2 92.36, BLEU-3 91.12, BLEU-4 89.78, BERTScore P 98.81, R 98.68, F1 98.74, and METEOR 93.84 (Gaur et al., 22 Aug 2025). Relative to the best SFT baseline, the reported gains are +23.35% R1, +35.74% R2, +27.49% RL, +14.02% B1, +15.45% B2, +16.64% B3, +17.80% B4, +2.34% BS-P, +3.24% BS-R, +2.81% BS-F1, and +17.54% METEOR (Gaur et al., 22 Aug 2025).

The ablation study clarifies the role of REDDME’s structured supervision. For the Gemma-based configuration, the full model reaches ROUGE-L 88.88, BLEU-4 89.78, BERTScore-F1 98.74, and METEOR 93.84; without reward, these drop modestly to 87.67, 88.32, 98.52, and 93.33; without reward and without CueTaxo, the scores fall much more sharply to 65.58, 74.00, 95.48, and 76.98 (Gaur et al., 22 Aug 2025). The paper therefore attributes the largest quantitative jump to CueTaxo, while the verifier-based reward mainly improves generation quality in a more targeted way (Gaur et al., 22 Aug 2025).

A representative pipeline example also reveals a failure mode. In a post about taking Adderall and coffee together, human annotation marks Event = Moderate, Effect = Well-described, and Requirement = Well-described, with the gold follow-up asking only “Can you elaborate more on why you are taking adderall?” (Gaur et al., 22 Aug 2025). MH-COPILOT instead predicts Effect = Moderate as well and generates an unnecessary effect question. The appendix explicitly notes that errors in early modules propagate into later generations (Gaur et al., 22 Aug 2025).

Human evaluation on 40 random instances compared MH-COPILOT with and without the verifier. The paper reports improvements on domain-centric metrics and linguistic metrics alike: without verifier, D1 3.27, D2 1.82, D3 2.19, L1 3.46, L2 3.70, L3 3.82, L4 3.80; with verifier, D1 3.43, D2 2.27, D3 3.31, L1 3.62, L2 3.84, L3 4.02, L4 3.89 (Gaur et al., 22 Aug 2025). The appendix says D1–D3 were scored on a 1–3 scale, yet D1 exceeds 3 in the table, which indicates another reporting inconsistency (Gaur et al., 22 Aug 2025).

6. Interpretation, limitations, and relation to adjacent Reddit resources

REDDME is best understood as a dataset for support articulation, not for generic emotion recognition or clinical diagnosis (Gaur et al., 22 Aug 2025). Its labels target the structure of self-disclosure—event, effect, and requirement—and the associated question layer is explicitly designed to elicit missing or weakly expressed information. This makes it distinct from adjacent Reddit datasets that annotate emotional states themselves. For example, ReDDIT formulates multiclass regret detection and domain identification over Regret by Action, Regret by Inaction, and No Regret, plus domains such as Romance and Relationships, Education, and Health (Balouchzahi et al., 2022). REDDME, by contrast, is organized around support cues within OMHC posts rather than affect categories (Gaur et al., 22 Aug 2025).

Several limitations are explicit. First, the corpus is derived from one source dataset and one platform, and specifically from the interactive subset of BeCOPE (Gaur et al., 22 Aug 2025). Second, the paper relies on public Reddit content but does not describe a substantive de-identification protocol, and privacy is not discussed in depth beyond public-subreddit sourcing and research release (Gaur et al., 22 Aug 2025). Third, annotator well-being is only indirectly addressed, despite the sensitive domains involved (Gaur et al., 22 Aug 2025). Fourth, the framework is vulnerable to error propagation from span extraction to intensity prediction to generation (Gaur et al., 22 Aug 2025). Fifth, the paper itself contains reporting inconsistencies, including the DPO “hinge-loss” description and the human-evaluation scale mismatch (Gaur et al., 22 Aug 2025).

The broader methodological implication is that REDDME converts a diffuse engagement problem into a structured NLP problem. As a measurement dataset, it defines what “better articulation” means in OMHC posts; as a generation dataset, it supports controlled prompt construction for absent or underdeveloped support attributes (Gaur et al., 22 Aug 2025). This suggests a reusable template for future work on support-oriented moderation, domain-conditioned self-disclosure modeling, and intervention systems that improve post informativeness without attempting diagnosis or treatment.