MPDD: Age-Specific Multimodal Depression Detection

• The paper presents a benchmark that integrates multimodal behavioral signals with individual personality traits for age-specific depression detection.
• It employs distinct data cohorts and tailored fusion models, combining audio, video, and text-based personality embeddings to drive personalized analysis.
• Baseline and follow-up studies indicate that personality-enhanced fusion significantly improves prediction accuracy, addressing overfitting and class imbalance challenges.

Searching arXiv for the challenge paper and closely related follow-up work. Depression detection research has increasingly emphasized multimodal behavioral signals, but many benchmark resources have centered on young adults and have treated prediction as a direct mapping from audiovisual observations to depression labels. The Multimodal Personality-Aware Depression Detection Challenge (MPDD Challenge) was introduced to address this limitation by explicitly combining multimodal data with individual-difference factors, especially personality traits, and by separating evaluation into elderly and young cohorts (Fu et al., 15 May 2025). The challenge defines a benchmark for personalized depression detection under age-specific conditions, provides a baseline fusion model, and has subsequently served as a testbed for methods that refine multimodal fusion, personality-guided representation learning, and cross-context generalization (Fu et al., 15 May 2025).

1. Origin, scope, and problem formulation

The MPDD Challenge was presented as “The First MPDD Challenge: Multimodal Personality-aware Depression Detection” (Fu et al., 15 May 2025). Its stated motivation is that depression affects diverse age groups, while existing datasets and detection methods primarily focus on young adults and often neglect broader age variation and individual differences. The challenge therefore broadens automatic depression detection beyond young adults by introducing both elderly and young cohorts and by incorporating individual-difference factors—especially personality traits—to move beyond “direct” multimodal-to-depression mappings and toward personalized detection (Fu et al., 15 May 2025).

The benchmark is organized into two tracks. Track 1 uses the MPDD-Elderly dataset for detecting depression in older adults, and Track 2 uses the MPDD-Young dataset for detecting depression in younger participants (Fu et al., 15 May 2025). This age-specific partition is central to the challenge design: the elderly and young subsets differ not only in participant demographics but also in recording protocols, auxiliary annotations, and depression labeling schemes.

A defining feature of the challenge is its explicit integration of personality-aware information into multimodal prediction. In the challenge formulation, audio and video signals are not treated as sufficient on their own; instead, they are complemented by structured individual information such as Big Five personality traits, demographic factors, and, in the elderly track, contextual variables including disease category, financial stress, and number of co-habiting family members (Fu et al., 15 May 2025). This suggests a shift from purely behavior-driven recognition toward individualized inference, where the same observable cue may have different predictive significance depending on the subject profile.

2. Datasets, cohorts, and annotation protocols

The MPDD Challenge defines two age-specific datasets with different acquisition settings and annotation regimes (Fu et al., 15 May 2025).

For MPDD-Elderly, the challenge reports 94 elderly volunteers with mean age approximately 68, recruited from a hospital setting in China, many with chronic conditions (Fu et al., 15 May 2025). The data comprise 564 segmented samples generated with 1 s and 5 s sliding windows, split into 337 training and 227 test clips with no subject overlap (Fu et al., 15 May 2025). Annotations include depression labels from PHQ-9 in a five-class setting and HAMD-24 in binary and ternary settings, personality from BigFive-10 using one-hot top trait encoding, and additional labels for disease category, financial stress, and number of co-habiting family members (Fu et al., 15 May 2025). The recording protocol consists of semi-structured interviews guided by HAMD/BDI questions with synchronized video and audio capture (Fu et al., 15 May 2025).

For MPDD-Young, the challenge reports 88 young adults with mean age approximately 21, described as non-clinical and recruited from a university (Fu et al., 15 May 2025). This subset contains 528 samples, with 264 training and 264 test (Fu et al., 15 May 2025). Depression annotations are based on PHQ-9 in binary and ternary forms, personality is again represented using BigFive-10 one-hot encoding, and demographic variables include age, gender, and native place (Fu et al., 15 May 2025). The protocol follows a more controlled sequence: questionnaire, self-introduction, and two controlled text-reading tasks, with simultaneous video and audio capture (Fu et al., 15 May 2025).

A later challenge-participant study reports the training-set composition somewhat differently, describing MPDD-Elderly as 337 semi-structured interview recordings from 89 elderly participants with mean age $62.8 \pm 11.0$, and MPDD-Young as 264 recordings from 88 university-age speakers with mean age $20.0 \pm 2.2$ (Hong et al., 28 Aug 2025). That work also lists explicit class distributions for the training split, including severe imbalance in the elderly quinary task, where the “very severe” class accounts for 0.9% (Hong et al., 28 Aug 2025). The discrepancy in elderly participant count between the challenge overview and the participant study reflects differences in reported corpus granularity or subset accounting; the challenge description remains the primary definition of the official benchmark (Fu et al., 15 May 2025).

3. Modalities and personality-aware representations

The challenge provides multimodal features spanning audio, video, and text-derived personalized descriptions (Fu et al., 15 May 2025). Audio waveforms are represented through MFCC with 13 dimensions, OpenSMILE descriptors of approximately 6 k dimensions, and Wav2Vec embeddings with 768 dimensions (Fu et al., 15 May 2025). Video frames are represented through DenseNet-121 features with 1024 dimensions, OpenFace landmarks and AU intensities with $68+17$ dimensions, and ResNet-50 embeddings with 2048 dimensions (Fu et al., 15 May 2025). Textual prompts are generated by ChatGLM3 from each subject’s Big Five profile, stress information, and related attributes, then embedded by RoBERTa-large into a 1024-dimensional vector (Fu et al., 15 May 2025).

The individual-difference factors differ by track. Big Five personality traits are measured by BigFive-10, with scores for openness, conscientiousness, extraversion, agreeableness, and neuroticism, encoded as one-hot top trait(s) in the challenge release (Fu et al., 15 May 2025). Depression severity is labeled using PHQ-9 and, for the elderly track, HAMD-24 (Fu et al., 15 May 2025). Elderly-only auxiliary variables include disease category, financial stress, and family-member count, whereas the young track includes age, gender, and native place (Fu et al., 15 May 2025).

Subsequent work on the challenge retains the same overall principle but instantiates it differently. A participant system describes “personalized embedding” as raw trait descriptions tokenized and fed into pre-trained RoBERTa-large to produce a 1,024-dimensional personality embedding per participant (Hong et al., 28 Aug 2025). Another follow-up method for the elderly track, “Personality-Enhanced Multimodal Depression Detection in the Elderly,” also uses ChatGLM3-generated text with RoBERTa-large to obtain a trait embedding $p \in \mathbb{R}^d$, which is later fused with multimodal features via an explicit interaction module (Wang et al., 9 Oct 2025). A later MPDD-Young method, P$^3$HF, instead prompts GPT-4 with demographic attributes and Big Five scores to generate a 1–2 sentence contextual description, encodes that description with BERT, and obtains a 768-dimensional “personality embedding” $P_i$ (Fu et al., 16 Nov 2025).

These variants share the same conceptual premise: discrete personal attributes are first converted into a richer textual representation and then embedded with a pretrained LLM. A plausible implication is that the challenge helped normalize an LLM-mediated pathway from structured subject metadata to dense conditioning vectors for multimodal affective modeling.

4. Baseline architecture and official evaluation

The official challenge baseline fuses audio and video modalities with individual-difference information in a staged pipeline (Fu et al., 15 May 2025). The audio encoder is an LSTM over frame-level audio features, producing a final hidden vector $a$. The video encoder is an LSTM over frame-level visual features, producing a final hidden vector $v$. These are concatenated and passed to a small Transformer encoder to form a multimodal representation $m$. A personalized embedding $p \in \mathbb{R}^{1024}$, obtained from RoBERTa over the ChatGLM3-generated description, is concatenated with $20.0 \pm 2.2$0 to produce $20.0 \pm 2.2$1, and a two-layer MLP with softmax performs depression classification (Fu et al., 15 May 2025).

The challenge summary gives the multimodal fusion step as

$20.0 \pm 2.2$2

$20.0 \pm 2.2$3

$20.0 \pm 2.2$4

where $20.0 \pm 2.2$5 and $20.0 \pm 2.2$6 are learned weights and $20.0 \pm 2.2$7 is ReLU (Fu et al., 15 May 2025). An equivalent linear-fusion form is also stated: $20.0 \pm 2.2$8 For binary, ternary, and quinary tasks, the loss is cross-entropy: $20.0 \pm 2.2$9 The challenge defines Accuracy as $68+17$0 and $68+17$1, with weighted and unweighted versions used to account for class imbalance (Fu et al., 15 May 2025). The task-level score is the average of weighted and unweighted variants, and the track-level score is the mean of its task-level scores (Fu et al., 15 May 2025).

The baseline results reported for 1 s windows with personalized features (“PF”) are summarized below.

Track/task	Best reported baseline configuration	Reported result
MPDD-Elderly binary	MFCC + OpenFace + PF	$68+17$2, $68+17$3
MPDD-Elderly ternary	OpenSMILE + ResNet + PF	$68+17$4, $68+17$5
MPDD-Elderly quinary	OpenSMILE + DenseNet + PF	$68+17$6, $68+17$7
MPDD-Young binary	Wav2Vec + OpenFace + PF	$68+17$8, $68+17$9
MPDD-Young ternary	MFCC + DenseNet + PF	$p \in \mathbb{R}^d$0, $p \in \mathbb{R}^d$1

The challenge also reports ablations showing that PF improves elderly binary $p \in \mathbb{R}^d$2 by $p \in \mathbb{R}^d$3 points, from $p \in \mathbb{R}^d$4 to $p \in \mathbb{R}^d$5, and improves young binary $p \in \mathbb{R}^d$6 by $p \in \mathbb{R}^d$7 points, with consistent gains of 2–13 points across tasks (Fu et al., 15 May 2025). This is one of the clearest empirical claims attached to the benchmark: personality-aware embeddings materially improve prediction under the official baseline.

5. Follow-up methods on the benchmark

The MPDD Challenge rapidly became a target for method development, and the follow-up literature illustrates several distinct modeling strategies.

One challenge-participant study, “Exploring Machine Learning and LLMs for Multimodal Depression Detection,” compares XGBoost, a transformer-based architecture, and an LLM-based model on audio, video, and text features (Hong et al., 28 Aug 2025). Its XGBoost configuration uses PCA to project each modality to $p \in \mathbb{R}^d$8 principal components and fuses them as $p \in \mathbb{R}^d$9, with objective “multi:softprob” and class weighting for imbalance (Hong et al., 28 Aug 2025). The transformer projects modality features into a shared $^3$0 latent space, employs two modality-specific transformer encoders with 2 layers and 4 heads, performs time-pooling with learned attention weights, and concatenates $^3$1, $^3$2, and $^3$3 before two fully connected layers (Hong et al., 28 Aug 2025). The LLM model uses LLaMA-2-7B extended via Emotion-LLaMA, modality projections into a 4096-dimensional shared space, and a structured multiple-choice prompt, with a two-stage fine-tuning procedure involving frozen backbone training followed by LoRA adaptation (Hong et al., 28 Aug 2025). On the reported dev split, XGBoost reaches $^3$4 and $^3$5 for MPDD-Elderly 5 s binary, while the transformer achieves $^3$6 and $^3$7 on MPDD-Young 1 s binary; the LLM underperforms both (Hong et al., 28 Aug 2025). That study concludes that scale alone does not guarantee better depression detection and that transformer-based fusion is the most consistent top performer across tasks (Hong et al., 28 Aug 2025).

A second line of work focuses specifically on the elderly track. “Personality-Enhanced Multimodal Depression Detection in the Elderly” introduces a multi-feature fusion approach based on a co-attention mechanism for audio, combining LLDs, MFCCs, and Wav2Vec, while visual features combine OpenFace, ResNet, and DenseNet representations (Wang et al., 9 Oct 2025). Its multimodal stream produces per-utterance embeddings through LSTM encoders and Attentive Static Pooling, followed by a Transformer fusion block: $^3$8 The personality component is modeled by a Personality Traits & Multimodal Feature Interaction Module (PTMFIM), which uses binary correlation attention, triple interaction attention, and a gating regulator to combine a RoBERTa personality embedding with the fused multimodal representation (Wang et al., 9 Oct 2025). On 1 s windows, the reported results improve over a baseline from 85.01 to 89.18 in binary accuracy and from 82.42 to 88.53 in binary $^3$9, with corresponding gains for ternary and quinary tasks (Wang et al., 9 Oct 2025). The ablation with the largest degradation is removal of PTMFIM, where binary accuracy drops from 89.18 to 86.29 and $P_i$0 from 88.53 to 85.93 (Wang et al., 9 Oct 2025). This strengthens the challenge’s original claim that personality-aware fusion is not merely auxiliary but structurally important for performance.

A third line extends the benchmark toward graph-structured and domain-generalized modeling. “Personality-guided Public-Private Domain Disentangled Hypergraph-Former Network for Multimodal Depression Detection” proposes P$P_i$1HF for MPDD-Young (Fu et al., 16 Nov 2025). The method first derives personality embeddings from GPT-4-generated text and BERT, then uses a gating mechanism to modulate audio and video streams: $P_i$2

$P_i$3

It then builds a Hypergraph-Former over audio and video segments with positional encoding and multi-head self-attention, followed by public-private domain disentanglement with adversarial loss and HSIC regularization (Fu et al., 16 Nov 2025). On MPDD-Young, P$P_i$4HF reports 82.17% accuracy and 81.39% weighted $P_i$5 for binary classification, and 76.29% accuracy and 74.61% weighted $P_i$6 for ternary classification, with improvements over DepMamba of $P_i$7 in binary and $P_i$8 in ternary (Fu et al., 16 Nov 2025). Ablation shows that removing personality guidance reduces binary accuracy by approximately 5.5% (Fu et al., 16 Nov 2025).

Together, these studies indicate that the MPDD benchmark supports a broad range of model families—from gradient-boosted trees to transformer fusion modules and hypergraph-based architectures—while repeatedly reproducing the utility of personality-aware conditioning.

6. Technical themes, challenges, and research implications

Several technical themes recur across the challenge and follow-up systems.

First, multimodal fusion remains a central design variable. The official baseline uses simple concatenation with linear projection and a small Transformer encoder (Fu et al., 15 May 2025). Later work explores more structured alternatives: co-attention among heterogeneous audio streams, explicit multimodal-personality interaction modules, and hypergraph reasoning for high-order cross-modal temporal relations (Wang et al., 9 Oct 2025, Fu et al., 16 Nov 2025). This suggests that the benchmark is not only about adding personality metadata, but also about testing how personality should interact with temporally structured audiovisual evidence.

Second, personality encoding is consistently treated as a learned dense representation derived from textualization of structured traits. In the challenge baseline, ChatGLM3 converts subject attributes into textual prompts that RoBERTa-large embeds to 1024 dimensions (Fu et al., 15 May 2025). Later methods preserve the same general mechanism with different LLMs, including GPT-4 and BERT (Fu et al., 16 Nov 2025). A plausible implication is that natural-language reformulation acts as an intermediate semantic layer, making discrete subject descriptors more compatible with pretrained text encoders and attention-based fusion modules.

Third, the benchmark foregrounds clinically and methodologically relevant heterogeneity. The elderly and young tracks differ in acquisition setting—hospital versus university/laboratory-like controlled capture—and this variability is explicitly identified as a source of domain shift (Fu et al., 15 May 2025). The challenge summary also notes limited dataset size and class imbalance, especially in quinary classes (Fu et al., 15 May 2025). The participant comparison study reaches the same conclusion, emphasizing the risk of overfitting and the difficulty of skewed quinary classification, particularly where the “very severe” class is only 0.9% of the elderly training set (Hong et al., 28 Aug 2025).

Fourth, the benchmark makes room for event-structured reasoning in the young cohort. P$P_i$9HF exploits the fact that each young subject has three recorded “events”—self-introduction and two reading tasks—and frames generalization partly as a problem of disentangling event-specific from event-invariant representations (Fu et al., 16 Nov 2025). This is not part of the official baseline, but it demonstrates that the MPDD design can support research on behavioral-context disentanglement in addition to depression classification.

7. Limitations, open questions, and prospective directions

The challenge paper identifies three principal limitations: limited dataset size, class imbalance, and the possibility that hand-crafted fusion by concatenation underutilizes complex cross-modal and personality interactions (Fu et al., 15 May 2025). It also notes variability in recording conditions between hospital and lab settings as a source of domain shift (Fu et al., 15 May 2025). These points are echoed by follow-up work, which reports no formal statistical significance tests in one comparative study and highlights possible overfitting on small datasets (Hong et al., 28 Aug 2025).

Several future directions are explicitly proposed in the challenge description. These include attention-based cross-modal transformers or gated multimodal blocks for dynamic weighting of audio, visual, and personality signals; multi-task learning to jointly predict depression severity and auxiliary labels such as personality dimension scores; semi-supervised learning and data augmentation; interpretability methods to trace which modality or individual factor drives a decision; and expansion to text transcripts, physiological signals such as HRV and EEG, and contextual metadata (Fu et al., 15 May 2025). Follow-up papers refine these possibilities. The elderly-track PTMFIM study notes that end-to-end fine-tuning of audio and video backbones was not explored and that personality embedding derived from a generative LLM may warrant sensitivity and noise analysis (Wang et al., 9 Oct 2025). The comparative study proposes noise reduction, speaker diarization, cross-modal attention layers, domain-specific pre-training, and tighter integration of personality embeddings rather than simple concatenation (Hong et al., 28 Aug 2025).

A common misconception in multimodal mental-health modeling is that adding more modalities or a larger backbone will automatically improve detection. The MPDD-related evidence does not support that simplification. The participant comparison study reports that an LLM with 6.84 B parameters underperforms both XGBoost and a 1.06 M-parameter transformer on the examined settings (Hong et al., 28 Aug 2025). Conversely, the challenge baseline and later systems repeatedly show that carefully structured personality-aware fusion can yield substantial gains even with relatively simple fusion components (Fu et al., 15 May 2025, Wang et al., 9 Oct 2025).

The MPDD Challenge therefore occupies a specific place in depression detection research: it is a benchmark built around the hypothesis that multimodal behavioral signals should be interpreted in light of age-specific context and individual differences. Its continuing significance lies less in a single baseline score than in the methodological agenda it set—personalized multimodal inference, age-aware benchmarking, and explicit modeling of personality-conditioned depression manifestations (Fu et al., 15 May 2025).