PolyHope Dataset: Multilingual Hope Speech

• PolyHope is a curated dataset suite of Twitter messages annotated for hope speech, enabling both binary and multi-class classification of nuanced subtypes.
• It utilizes precise annotation protocols to distinguish generalized, realistic, and unrealistic hope, with recent expansions including a sarcasm category.
• The dataset supports state-of-the-art transformer models and active learning methods, advancing emotion detection and moderation across multiple languages.

PolyHope is a curated suite of datasets and benchmarks designed for fine-grained, multilingual hope speech detection in social media. It operationalizes the psychological construct of hope into annotation schemas suitable for NLP, enabling both binary and multi-class analysis. The dataset captures distinctions among generalized optimism, realistic and unrealistic expectations, and, in its latest version, sarcasm. PolyHope has emerged as a central resource in research at the intersection of sentiment analysis, emotion recognition, and social wellbeing, facilitating comparative evaluation and enabling deployment of transformer-based and multilingual deep learning models.

1. Definition and Scope

PolyHope refers primarily to a collection of tweets annotated for hope speech, as introduced in "PolyHope: Two-Level Hope Speech Detection from Tweets" (Balouchzahi et al., 2022). It is structured for two sequential analytic tasks:

• Level 1: Binary classification — each tweet labeled as “Hope” or “Not Hope.”
• Level 2: For tweets marked as “Hope,” a further multiclass annotation into:
  • Generalized Hope: Generic optimism, not tied to events or outcomes.
  • Realistic Hope: Hope for specific, reasonably probable outcomes.
  • Unrealistic Hope: Hope for highly improbable outcomes, sometimes valenced with anger or despair.

Subsequent expansions, notably PolyHope V2 (Butt et al., 24 Apr 2025), introduce multilingual and additional subcategories, including explicit “Sarcasm,” facilitating nuanced multi-class sentiment parsing.

2. Data Collection and Corpus Construction

The initial PolyHope corpus was sourced from Twitter from January–June 2022, restricted to English and constructed as follows (Balouchzahi et al., 2022):

• Collection: 2 × 50,000 tweets; one random sample, one using hope-related keywords (“hope,” “aspire,” “wish,” etc.), filtered for language and length.
• Cleaning: Incomplete, short (<10 words), duplicate, and retweeted messages were excluded.
• Sampling: ∼10,000 tweets annotated; final release is 8,256 tweets (4,175 Hope and 4,081 Not Hope).

PolyHope V2 was expanded for multilinguality (Butt et al., 24 Apr 2025):

• English: Built from a source corpus focused on rights and politics.
• Spanish: Compiled using translated triggers (e.g., “esperanza” for “hope”) and native validation.
• Additional Languages: Later versions introduce data in German and Urdu, with similar trigger-based extraction and rigorous pre-processing (Abiola et al., 30 Sep 2025).

3. Annotation Protocol and Subtype Taxonomy

Annotation is guided by operationalized definitions distinguishing factual, actionable, and affective aspects of hope (Balouchzahi et al., 2022):

• Annotator Selection: Rigorously screened for linguistic and domain expertise; only top-performing individuals after multi-stage evaluation are retained.
• Guidelines: Define hope as a future-oriented expectation or desire, with annotation workflow separating Not Hope from the three hope subtypes.
• Decision Process: Binary presence of hope, followed by multiclass labeling if present.
• Quality Control: Cohen’s Kappa for binary IAA achieved 85%; Fleiss’ Kappa for multiclass reached 82%.

PolyHope V2 adds a sarcasm label, leveraging external sarcasm datasets and synthetic generation verified by experts (Butt et al., 24 Apr 2025).

4. Model Benchmarks and Evaluation Metrics

PolyHope datasets provide systematic benchmarking for classical, deep learning, and transformer architectures (Balouchzahi et al., 2022, Butt et al., 24 Apr 2025, Abiola et al., 30 Sep 2025):

• Traditional ML: Logistic Regression (LR), SVMs, ensemble methods using unigram TF-IDF.
• Deep Learning: CNNs, BiLSTM using FastText and GloVe embeddings.
• Transformers: BERT, RoBERTa, DistilBERT, ALBERT, XLNet, ELECTRA; fine-tuned with max sequence length 100, LR=3e-5, 15 epochs, 5-fold CV.
• Multilingual Models: XLM-RoBERTa for cross-language generalization (Abiola et al., 30 Sep 2025).

Evaluation employs precision, recall, weighted-averaged and macro-averaged F1-scores:

• Binary task: BERT-base and RoBERTa achieve weighted F1 ≈ 0.85.
• Multiclass: BERT reaches weighted F1 = 0.77, macro F1 = 0.72; XLM-RoBERTa scores macro F1 ≈ 0.75 for English, outperforming classical and earlier transformer models.

PolyHope V2 includes LLMs (GPT-4, Llama 3) under zero/few-shot, consistently finding fine-tuned transformers outperform in nuanced multi-class scenarios (Butt et al., 24 Apr 2025).

5. Methodological Innovations: Active Learning and Multilinguality

Multiclass settings and low-resource languages present significant class imbalance and ambiguity; methodological advances include:

• Active Learning with Uncertainty Sampling: Iteratively select high-entropy samples (uncertainty $\mathcal{U}(x) = -\sum_k p_k \log p_k$) to augment labeled set (Abiola et al., 30 Sep 2025).
• Weighted Loss Functions: Class weights $\{w_k\}$ incorporated into binary cross-entropy to mitigate dominant class bias:

$$\mathcal{L}_{\text{weighted}} = -\sum_k w_k \left[ y_k \log p_k + (1-y_k)\log(1-p_k) \right]$$

• Multilingual Transformer Fine-tuning: Cross-lingual encoders (XLM-RoBERTa) demonstrate improved generalization, particularly in languages with fewer annotated instances (e.g., Urdu).
• Synthetic Data Augmentation: For sarcasm in Spanish, GPT-4 synthesized sarcastic examples, validated by expert annotators (Butt et al., 24 Apr 2025).

6. Technical and Conceptual Challenges

Several sources of difficulty are documented with direct analytic significance:

• Conceptual Ambiguity: Demarcation between realistic and unrealistic hope can be subtle; for instance, negative desires (e.g., “I hope they all get hurt”) challenge sentiment-oriented presumptions (Balouchzahi et al., 2022).
• Annotation Disagreement: Despite high IAA, fine-grained hope subtypes, especially in political or affect-laden contexts, exhibit boundary fuzziness; confusion matrices show large cross-label misclassification (Butt et al., 24 Apr 2025).
• Class Imbalance: Minority subtypes (Realistic, Unrealistic, Sarcasm) receive lower recall, even in advanced models; macro F1 scores reveal persistent performance gaps.
• Low-Resource Generalization: Syntactic, semantic complexity in underrepresented languages constrains detection models, necessitating active sampling and multilingual pretraining (Abiola et al., 30 Sep 2025).

7. Applications and Future Directions

PolyHope datasets support applications in social media research, education, mental health analytics, and online community moderation:

• Emotion analysis: Frameworks for distinguishing genuine optimism from wishful thinking or veiled despair can enrich sentiment tools.
• Positive Discourse Promotion: Automated moderation systems leveraging PolyHope-trained models can surface supportive content (Abiola et al., 30 Sep 2025).
• Cross-lingual deployment: Robust transformer models enable inclusion of diverse language communities.

This suggests that the continued evolution of PolyHope—incorporating richer features, pragmatic cues, and hybrid generative-discriminative modeling—will provide a foundation for advanced research in nuanced emotion detection and automated social well-being interventions.