- The paper introduces K-SENSE, a unified model that integrates external commonsense knowledge and self-augmentation to enhance mental health condition detection.
- It employs a triple-pass encoding and cross-space projection to align psychological inferences with robust semantic representations.
- Experiments on Dreaddit and Depression_Mixed datasets show statistically significant performance improvements using supervised contrastive learning.
Overview
K-SENSE addresses the persistent challenges of early mental health condition detection from social media, notably targeting stress and depression identification through textual signals. The model aims to bridge the gap between knowledge-driven psychological reasoning and robust neural representation learning by integrating external commonsense knowledge (via COMET) and internal self-augmentation within a unified encoding architecture.
Motivation and Theoretical Innovations
Detection of psychological conditions from noisy, figurative, and highly individualistic social media language requires advances beyond standard text classification architectures. Prior approaches fall into one of two camps: (1) knowledge-augmented models leveraging external reasoning resources such as ATOMIC/COMET to simulate theory of mind inference (e.g., KC-Net); or (2) representation-improving regimes based on self-augmentation and contrastive learning, enhancing generalization in low-resource and noisy contexts (e.g., SA-CL). However, these lines of work rarely interact, leading to models vulnerable to spurious features, external hallucination error propagation, or failure to mentalize in non-literal cases.
K-SENSE is developed to explicitly combine these regimes: it constructs a “semantic anchor” based on self-augmentation to serve as a stable query for knowledge selection, then reconciles cross-model geometric differences with a projection mechanism, and finally employs supervised contrastive learning to stabilize and sharpen representations.
Architecture
Triple-Pass Encoding
- Commonsense Knowledge Extraction: Sentence-wise COMET queries generate five psycho-social inference dimensions: xIntent, xReact, xNeed, oReact, and oEffect, all encoded with frozen MiniLM.
- Standard Semantic Encoding: MentalRoBERTa processes the full post, outputting the <s> embedding.
- Self-Augmented Semantic Anchor: A second pass with independent dropout is summed with the first to yield a low-variance semantic anchor.
Cross-Space Projection and Temporal Integration
A trainable linear projection aligns the MentalRoBERTa semantic anchor with the MiniLM knowledge space before attention-based selection. Temporal knowledge integration is performed via a GRU over sequential COMET outputs, preserving the emotional trajectory—a critical attribute in stress/depression narratives.
Knowledge-Attention and Filtering
Knowledge-aware attention is computed using the projected semantic anchor as a query, ensuring that attention weights are focused on COMET inferences consistent with the interior model’s understanding. This mechanism is shown to down-weight knowledge noise or hallucinated relations and is empirically supported by entropy reduction and ablation comparison.
Supervised Contrastive Learning
A SupCon loss is jointly optimized with cross-entropy (with α = 0.7), systematically separating class-conditional representation clusters and mitigating collapse in low-resource regimes. Batching is class-stratified, and loss balancing prevents degenerate optimization in highly imbalanced settings.
Numerical Results and Ablation Analysis
K-SENSE attains F1-scores of 86.1 ± 0.6% on Dreaddit and 94.3 ± 0.8% on Depression_Mixed, a relative improvement of 2.6 and 1.5 points over the best prior (KC-Net, re-implemented to match encoder/preprocessing choices) (2604.23493). These improvements are statistically significant (p < 0.05 for Dreaddit). Ablation studies robustly demonstrate:
- The necessity of the projection layer to prevent cross-space attention degradation.
- Freezing MiniLM knowledge encoder yields superior generalization than fine-tuning, corroborating concerns of representation drift and overfitting on small datasets.
- GRU-based temporal knowledge integration outperforms mean-pooling, especially on posts with complex emotional evolution.
- Self-augmentation and contrastive learning are synergistic, yielding improved cluster quality and narrowing performance variance.
- The semantic anchor effectively filters knowledge—entropy analysis reveals more selective and psychologically plausible attention distributions.
Implications and Limitations
Practical Implications
K-SENSE provides a framework suitable for semi-automated triage or prioritization in clinician-support systems, leveraging both automated reasoning over mental state proxies and robustness-enhanced text representations. Its knowledge filtering mechanism reduces risk of hallucinated reasoning adversely affecting model output—a critical feature for deployment in settings with possible distributional/linguistic drift.
Theoretical Implications
By demonstrating systematic gains from principled semantic-knowledge integration, the work suggests that bridging gaps between knowledge and representation spaces is essential for effective neuro-semantic evaluation, particularly in domains where literal and figurative language intermingle. The clear necessity of freezing knowledge encoders and selective knowledge querying informs future practice in knowledge fusion architectures.
Limitations
- Small dataset sizes yield test F1-scores vulnerable to sampling variance and limited generalizability.
- Results are not validated in clinical or clinician-labeled scenarios; current labels derive from crowdworkers.
- COMET and its underlying ATOMIC knowledge base retain Western, English-language, and domain-specific biases, limiting immediate cross-lingual or cross-cultural transferability.
- Soft filtering of knowledge noise is sensitive to anchor quality, and does not provide guarantees against systematic inference errors in underrepresented genres or idioms.
- The self-augmentation assumption of independent dropout-induced noise holds under specific hardware and software regimes.
Future Directions
Conclusion
K-SENSE establishes the value of unifying knowledge-augmented and self-augmenting neural architectures for the automatic, neuro-semantically informed identification of mental health indicators in social media text. The framework’s ablations highlight the importance of careful architectural integration, while quantitative cluster metrics substantiate the representation separation advantages conferred by contrastive objectives. While limited by current dataset size and label quality, the approach sets a clear direction for future neuro-symbolic models in computational mental health, advocating for scalable, robust, and knowledge-filtered encoding pipelines for sensitive detection tasks (2604.23493).
