SASS: Sentiment-Aware Sample Selection

• SASS is a framework that filters, selects, and augments samples based on sentiment characteristics, ensuring semantic relevance, context sensitivity, and fairness.
• It employs semi-supervised learning, domain adaptation, and statistical divergence metrics alongside sentiment-aware embeddings to improve training data quality.
• Empirical evaluations demonstrate that SASS enhances classification accuracy and bias detection in sentiment analysis across various domains and modalities.

Sentiment-Aware Sample Selection (SASS) encompasses a family of strategies and frameworks in natural language processing that aim to filter, select, or augment text (or multimodal) samples for training or evaluation in sentiment analysis according to their sentiment characteristics, semantic properties, and auxiliary criteria such as domain relevance, ambiguity, or fairness. The goal of SASS is to construct sample sets (for annotation, training, evaluation, bias probing, transfer, or data augmentation) that either optimize classifier performance, maximize robustness and coverage, or explicitly control for challenging or confounding factors in real-world sentiment modeling.

1. Conceptual Foundations of Sentiment-Aware Sample Selection

SASS builds upon classical data selection, semi-supervised learning, and domain adaptation research, introducing sentiment-driven perspectives in both mono- and multimodal contexts. Its foundations are:

• Semantic Relevance: Selection of samples that best reflect (or challenge) sentiment phenomena of interest, such as polarity, intensity, or sentiment ambiguity, rather than random or frequency-driven sampling.
• Context Sensitivity: Incorporation of domain, style, and context dependency in sentiment expression (e.g., slang, idioms, domain-specific sentiment shifts).
• Fairness and Bias Considerations: Systematic selection of counterfactual or demographically contrasted samples to reveal or mitigate social bias.
• Augmentation and Diversity: Generation or inclusion of samples to maximize coverage of linguistic, semantic, or emotional diversity relevant to sentiment analysis.

These objectives interact with existing challenges in NLP such as domain mismatch, label imbalance, and the need for robust, generalizable models.

2. Methodologies and Theoretical Principles

SASS methodologies range from feature-driven heuristics to full machine learning pipelines with feedback from model outputs or adversarial signals. Major categories include:

Semi-supervised and Feature-Informed Selection

Incorporating both labeled and unlabeled data via semi-supervised learning, models such as multinomial Naive Bayes with Expectation-Maximization (EM) estimate labels for unlabeled samples, which are then fed into feature selection procedures that consider both direct and imputed sentiment labels. The aim is to balance the bias–variance trade-off, as improper feature selection can amplify estimation bias or variance, especially when assumptions underlying generative models are violated. Mathematical formulations such as:

$$\log \ell(\theta) = \log \prod_{i=1}^{m_L} P(x^{(i)}, y^{(i)}; \theta) + \log \prod_{j=1}^{m_U} P(x^{(j)}; \theta)$$

demonstrate the integration of observed and hidden variables for parameter estimation (Ren et al., 2013).

Domain and Instance-Level Selection

Multi-domain adaptation introduces quantitative metrics for domain and instance similarity:

• Jensen–Shannon divergence, cosine similarity, and proxy 𝒜 distance (the latter approximated by domain-discriminative probabilities from a logistic regression classifier) are used for measuring relevance between source and target samples or domains.
• Selection operates at the domain, instance, or subset level; for instance, instance subset-level selection aggregates and compares similarity of random sample subsets to optimize both relevance and diversity (Ruder et al., 2017).

Such approaches are supported by representations based on word embeddings or autoencoders, offering alternatives to naive term counting.

Sentiment and Domain-Sensitive Embedding Approaches

SASS increasingly integrates representation learning. For example:

• Context-aware sentiment word embeddings propagate sentiment polarity through localized adjustments of vector space, enabling detection of words whose sentiment flips in domain-specific or idiomatic contexts (Yao et al., 2016).
• Domain-sensitive/sentiment-aware embeddings introduce latent variables governing whether token representations are shared across domains or specific, with the EM algorithm used for model inference (Shi et al., 2018). Sample selection is then informed by distinctions between domain-common (pooled) and domain-specific examples.

Statistical Divergence and Predictive Source Selection

Predictive models combine statistical divergence metrics (Earth Mover's Distance, Kullback-Leibler, Chi², Maximum Mean Discrepancy) in a linear predictor that forecasts cross-domain error, guiding the selection of sources most conducive to transfer with minimal sentiment drift. This is formalized by:

$$d(\mathbb{P}, \overline{\mathbb{P}}_x) = \beta_1 \,\mathrm{Chi}^2(\mathbb{P}_x, \overline{\mathbb{P}}_x) + \cdots + \beta_5 \,\xi(\mathbb{P}) + \beta_0$$

where the optimal β are trained to minimize absolute prediction error for out-of-domain classification (Schultz et al., 2018).

Bias and Fairness-Driven SASS

Recent work operationalizes SASS as a fairness probe. Tools such as BiasFinder and BTC-SAM automatically generate large, linguistically diverse test suites where only demographic identity terms vary. Pairwise output comparison reveals model biases. SASS here encompasses both efficient generation (via LLMs, metamorphic test design) and targeted sample selection for debiasing or auditing (Asyrofi et al., 2021, Kardkovács et al., 28 Sep 2025). Causal inference and backdoor adjustment formulas further underpin bias scoring and system rating (Lakkaraju et al., 2023).

Sampling under Class Imbalance

SASS adapts informed sampling techniques (e.g., SMOTE, ADASYN, one-sided selection, CNN/ENN cleaning) for sentiment analysis under severe class imbalance. Feature selection (e.g., information gain–based trigrambased reduction) ensures that only features most relevant to sentiment are included prior to sampling, enabling effective nearest neighbor–based synthetic sampling and noise deletion (Sayyed, 2021).

Augmentation and Mixup for Multimodal Sentiment

In multimodal settings, SASS prevents semantic confusion in mixup-based augmentation by selecting pairs whose latent emotional features match above a cosine similarity threshold after normalization. Only these pairs are mixed, with further adaptive mixing ratios and distributional alignment losses complementing the selection (Zhu et al., 13 Oct 2025).

3. Performance Outcomes and Empirical Evaluation

Performance gains attributed to SASS are domain and method dependent:

• Semi-supervised SASS strategies demonstrate that, when the feature space is tuned to optimize the bias–variance trade-off, the addition of unlabeled data can strictly improve sentiment classification accuracy—surpassing purely supervised approaches. However, if the feature set is mismatched to the available data or pseudo-labeling quality is low, performance can degrade dramatically (Ren et al., 2013).
• Domain and instance-level selection strategies consistently outperform random and uniform baselines (several percentage points in binary/ternary accuracy across review and tweet datasets); subset-level selection (aggregating over instance groups) often yields the most stable performance (Ruder et al., 2017).
• Bias test case generation techniques such as BTC-SAM and BiasFinder produce higher diversity and coverage for bias discovery, with substantial increases in the number of exposed bias cases relative to earlier, template-based approaches, e.g., >42,000 bias-revealing test cases for gender bias compared to 4,500 with MT-NLP (Kardkovács et al., 28 Sep 2025, Asyrofi et al., 2021).
• Multimodal SASS (MS-Mix) leads to statistically significant error reduction and accuracy improvement when benchmarked against conventional and mixup-based augmentation methods. Only sentiment-similar pairs are mixed, leading to more distinct decision boundaries and clearer feature distributions (Zhu et al., 13 Oct 2025).

4. Technical Implementation and Mathematical Formalism

SASS methods typically involve precise algorithmic and statistical machinery:

• Feature Selection with Unlabeled Data: Inclusion of pseudo-labels from EM or similar algorithms in supervised feature evaluators (e.g., information gain or mutual information).
• Similarity and Divergence Computation: Computation of pairwise or marginal similarity/distance metrics over features, embedding averages, or latent codes—with selection thresholding or global optimization over subsets (e.g., via argmin of cross-domain error predictors).
• Counterfactual Bias Testing: Automated pipeline instantiation with LLM-driven text generation, placeholder replacement, and candidate pruning. The key test statistic is the consistent differential in sentiment output for minimal identity switches.
• Statistical Testing and Causal Adjustment: Use of statistical hypothesis testing (e.g., Student’s t-test) and causal adjustment via Pearl’s do-calculus/backdoor formula:

$P(Y \mid do(X)) = \sum_Z P(Y \mid X, Z)P(Z)$

• Mixup Pipelines: Pre-selection based on cosine similarity of L2-normalized modality-specific features:

$$S = \frac{1}{3}\sum_{m \in \{t, v, a\}} {Z^{m}}^{norm} \cdot ({Z^{m}}^{norm})^T$$

Only pairs with $s_{ij} > \delta$ (δ = 0.2) are mixed (Zhu et al., 13 Oct 2025).

• Sentiment Alignment Losses: Addition of loss terms based on Kullback-Leibler divergence for regularizing prediction distribution, especially after mixup or augmentation.

5. Applications, Challenges, and Strategic Implications

SASS is applied in a variety of scenarios:

• Financial Sentiment Analysis: Addressing labeling cost by leveraging unlabeled MD&A report sentences while maintaining accuracy essential for downstream stock prediction (Ren et al., 2013, Muthivhi et al., 2022).
• Fairness in SA Systems: Automated bias discovery, balanced data augmentation, model auditing, and re-ranking by bias exposure in architectural or post-hoc model selection (Kardkovács et al., 28 Sep 2025, Asyrofi et al., 2021, Lakkaraju et al., 2023).
• Imbalanced Class Management: Improved f-score and recall (especially among rare sentiment classes) using synthetic minority oversampling and feature-driven filtering (Sayyed, 2021).
• Sentiment Transfer and Evaluation: Prioritizing or filtering samples based on sentiment transfer metrics, for example, by discounting sentence pairs with high sentiment divergence under the SAM paradigm for MT evaluation (Saadany et al., 2021).

Key challenges include:

• Computational Overhead: Many selection techniques (e.g., feature-based pairwise comparisons, statistical divergence calculations, and iterative augmentation) scale poorly as sample size increases.
• Domain and Context Dependence: SASS efficacy can be tightly linked to availability of high-quality sentiment lexicons, domain alignment in embeddings, or representative sample pools.
• Limitations in Bias Detection: For SASS that seeks to expose bias, coverage is contingent upon the diversity and representativeness of generated or selected test sets. Some subtle interaction effects or out-of-domain idioms remain hard to probe.
• Label Ambiguity in Augmentation: Even sentiment-aware mixup can struggle if latent sentiment estimation or similarity measures are unreliable across modalities or domains.

6. Future Directions

Emerging themes for SASS include:

• Integration with Self-/Unsupervised and Meta-Learning: Future SASS techniques are likely to incorporate more sophisticated dynamic selection via reinforcement or meta-learning; this may allow adaptive selection under distribution shift, for resource-constrained devices, or in continual learning scenarios (Zhu et al., 13 Oct 2025).
• Hybrid Measures and Multi-Objective Selection: Blending sentiment-awareness with adversarial robustness, fairness, or coverage measures, potentially in joint optimization frameworks.
• Extending Beyond Sentiment: Techniques developed in SASS are increasingly transferable to related categories, such as toxicity detection and hate speech classification, where nuanced sample selection is pivotal.
• Automated, Human-in-the-Loop Systems: As in BTC-SAM and BiasFinder, combining LLM-based sample generation and automated selection with expert review is poised to become standard in industrial NLP auditing pipelines.

7. Summary Table: SASS Methodological Dimensions

Approach	Key Mechanisms / Criteria	Example Paper
Semi-supervised Feature-Based	EM + pseudo-labeling, joint feature selection	(Ren et al., 2013)
Domain/Instance Selection	Similarity/divergence metrics, subset-level selection	(Ruder et al., 2017)
Representation-Based	Contextual/domain-sensitive embeddings	(Shi et al., 2018, Yao et al., 2016)
Statistical Divergence	CMEK predictor (EMD, KL, etc.)	(Schultz et al., 2018)
Fairness/Bias Test Generation	Counterfactual pairs, LLM-based augmentation	(Kardkovács et al., 28 Sep 2025, Asyrofi et al., 2021)
Mixup/Sampling for Imbalanced/Multimodal	Emotion similarity filtering, synthetic oversampling	(Zhu et al., 13 Oct 2025, Sayyed, 2021)

This table illustrates representative classes of SASS mechanisms and their associated research, reflecting the multidimensional nature of sample selection in state-of-the-art sentiment analysis research.