EmoPsy Corpus Overview

• EmoPsy Corpus is a collection of rigorously constructed corpora for analyzing psychological emotion expression in clinical speech and written text.
• It employs multi-rater frameworks and integrates Ekman’s basic emotions with Valence–Arousal–Dominance models for precise, reproducible annotation.
• Applications include enhancing emotion recognition in neurodegenerative studies and refining computational models in affective computing.

The EmoPsy Corpus is an overarching term for recent, rigorously constructed corpora designed to support empirical research on psychological emotion expression and perception, with an emphasis on spontaneous and clinical speech as well as text. These corpora combine detailed multi-rater annotation frameworks, theoretically grounded emotion taxonomies (notably Ekman’s basic emotions and Valence–Arousal–Dominance, VAD), and empirically validated adjudication and mapping algorithms. Notable instantiations include DementiaBank-Emotion for clinical speech and EmoBank for written text, each tailored for reproducible, fine-grained emotion analysis using state-of-the-art psycholinguistic and affective computing protocols.

1. Corpus Structures and Composition

DementiaBank-Emotion exemplifies the clinical-speech branch of the EmoPsy Corpus. It comprises 1,492 spontaneous utterances extracted from the Cookie Theft picture-description task, featuring 108 speakers (54 Alzheimer’s Disease [AD] patients, 54 matched controls) balanced for age and gender, with clinical diagnoses for AD validated as per protocol. From the extracted utterances, 752 were from AD speakers (615 with valid final labels, 137 ambiguous), and 740 from controls (731 valid, 9 ambiguous). Emotion categories are: joy, sadness, fear, anger, surprise, disgust (Ekman), plus neutral.

EmoBank offers large-scale, sentence-level emotion annotation for English text, aggregating 10,062 filtered sentences sourced from well-curated corpora such as MASC and the SemEval-2007 Affective Text corpus. It covers seven genres, including news headlines, blogs, essays, fiction, and travel guides. EmoBank supports both continuous VAD ratings and categorical basic emotion labels, thereby facilitating robust mapping and cross-validation between emotion representation schemes (Buechel et al., 2022).

Corpus	Modality	Size	Labels	Perspectives
DementiaBank-Emotion	Clinical speech	1,492 utterances (108 speakers)	7-way (Ekman+Neutral)	Speaker (AD/Control)
EmoBank	Written text	10,062 sentences	VAD & Basic Emotions	Writer/reader (bi-perspective)

2. Annotation Frameworks and Protocols

DementiaBank-Emotion employs 11 interdisciplinary annotators deployed in multiple rounds (five clinical experts in Round 1, a mixed team for subsequent rounds). Calibration workshops involved a psychiatry professor and a memory-care director, culminating in a standardized “Guideline v2.0.” The protocol integrates explicit adjudication rules:

• Assign majority label if any reaches ⌈n/2⌉;
• For neutral/emotion ties, select emotion (non-neutral preference);
• Otherwise, use rater confidence for weighted voting or flag as ambiguous if unresolved.

Emotion labels are defined by the Paul Ekman Group’s prototypical patterns, supplemented with context-specific rules for laughter and pragmatic "surprise" (triggered by pitch excursions or lexical "noticing"). Prosodic cues (see Section 3) were referenced in annotation to operationalize emotive speech categories (Jeong et al., 4 Feb 2026).

EmoBank utilizes crowdsourced rating via Figure Eight, with five raters per VAD dimension and perspective. Writer-oriented tasks focus on “expressed” emotion; reader-oriented tasks assess “evoked” emotion. Quality is ensured by gold questions, geographical restrictions, and removal of all-(1,1,1) VAD responses (spurious). Categorical labels for 1,000 headlines are derived from prior SemEval-07 annotations and down-scaled to the [1,5] VAD format. Aggregation employs arithmetic mean; inter-annotator agreement (IAA) is evaluated with Pearson’s r and MAE (Buechel et al., 2022).

Corpus	Raters	Adjudication	IAA (κ/r)	Special Procedures
DementiaBank-Emotion	11	Hierarchical, majority & confidence	κ = 0.09–0.31	Clinical calibration workshops
EmoBank	5/dimension	Mean aggregation	r = 0.54–0.74	Gold questions, fraud filtering

3. Acoustic and Statistical Findings

DementiaBank-Emotion revealed that AD speakers express non-neutral emotions in 16.9% of utterances vs. 5.7% for controls (χ²(1)=38.45, p<.001). Exploratory acoustic analysis (speaker-normalized F0, measured in semitones: $F0_{st} = 12\cdot\log_2(F0_{Hz}/F0_{mean})$) showed a dissociation for sadness: controls employed significant F0 depression (Δ=–3.45 st), while AD speakers did not (Δ=+0.11 st; interaction β=–5.20, SE=2.28, p=0.023). Loudness differentiated emotions within AD speech (ANOVA F(4,610)=8.48, p<.001, η²=.053); for instance, joy exceeded neutral by ΔLoud=+0.59 sones (p<.001). No significant differences were detected for jitter, shimmer, or HNR post-normalization, aside from a group×sadness shimmer effect (β=–0.53, p=0.017) (Jeong et al., 4 Feb 2026).

In EmoBank, VAD distributions were centered near neutral (μ≈3, σ≈0.9), with reader ratings having slightly greater variance and intensity. Reader perspective increased IAA and emotional mean ratings (paired t-tests; r=0.738 vs 0.698 for valence, p<.05), and mapping between VAD and categories achieves human-like performance for joy, anger, sadness, and fear (Pearson’s r up to 0.78) (Buechel et al., 2022).

4. Theoretical Integration and Adjudication Algorithms

DementiaBank-Emotion’s annotation encodes both psychological (Ekman, theme-based) and acoustic criteria (prosodic cue chart from Sobin & Alpert 1999) for each basic emotion. For example, sadness demands consistently low F0 and slow rate; joy, high F0 and loudness; surprise is marked by sudden F0 excursions. Annotators were instructed to default to neutral for flat prosody, regardless of lexical affect.

The hierarchical adjudication algorithm follows explicit steps:

1. Assign any majority label;
2. For ties (neutral vs. emotion), select emotion;
3. If unresolved, weigh rater confidence; otherwise, label as ambiguous.

EmoBank’s protocols enable fine-grained, perspective-aware analysis by explicitly separating writer and reader affect, and by providing methodologies for converting VAD to categorical schemes (KNN regression on VAD vectors in $\mathbb{R}^3$), achieving near-human mapping accuracy (Buechel et al., 2022).

5. Public Release, Licensing, and Accompanying Materials

DementiaBank-Emotion is released via DementiaBank (https://dementia.talkbank.org/), with audio, CHAT transcripts, finalized gold emotion labels, confidence scores, and detailed speaker metadata. Licensing is governed by standard DementiaBank/CHILDES terms for academic research. Accompanying materials include full guidelines (“Labeling-guideline-ver2.0.pdf”), calibration workshop slides, and exemplar cases, as well as code for key preprocessing and adjudication operations (Jeong et al., 4 Feb 2026).

EmoBank is available at https://github.com/JULIELab/EmoBank, including complete annotation instructions, raw and filtered data, all VAD scores (writer and reader), and categorical emotion mappings (Buechel et al., 2022).

6. Research Impact and Applications

The EmoPsy Corpus portfolio enables:

• Training and evaluation of clinical and general-purpose emotion recognition systems, especially for underrepresented populations (e.g., AD and other neurodegenerative conditions).
• Comparative studies of emotion-prosody mappings and their degradation/preservation in clinical populations.
• Methodological contributions in annotation protocol design, including perspective-aware emotion annotation and cross-format label mapping.
• Development of bi-representational resources supporting both categorical and dimensional emotion models for computational linguistics, affective computing, and clinical psycholinguistics.
• Standardized benchmarks and reproducibility in emotion annotation for both text and speech.

A plausible implication is that corpus-driven, multi-perspective emotion resources—such as DementiaBank-Emotion and EmoBank—will continue to enhance reproducibility, enable formative acoustic and psychological investigations in clinical and affective domains, and refine automatic emotion recognition for nuanced, context-dependent settings (Jeong et al., 4 Feb 2026, Buechel et al., 2022).