Ensemble Diversity Optimization for Subjective Supervision

Abstract: Subjective NLP tasks often exhibit systematic annotator disagreement, requiring models that represent uncertainty rather than collapse it. We introduce Ensemble Diversity Optimization (EDO), a prediction-space framework that jointly optimizes ensemble weights, effective cardinality, and calibration through a unified differentiable objective. EDO learns ensemble composition and size end-to-end via Gumbel-Softmax relaxation and incorporates a signed diversity regularizer, tuned on validation data, to steer optimization toward either preserving or suppressing disagreement. This regularization prevents ensemble collapse and enables controlled navigation of the utility-calibration trade-off. The framework integrates a soft F1 surrogate, class-weighted cross-entropy to address imbalance, and reliability-weighted diversity to regulate intra-ensemble variability. Experiments on four subjective text-classification benchmarks (ArMIS, ConvAbuse, HS-Brexit, MD-Agreement) show that EDO substantially improves probabilistic calibration, reducing cross-entropy (40-78% depending on baseline) and lowering Brier scores relative to Soft-CE, Soft-MD, Top-5 Voting, and WEL, while maintaining competitive F1 and better alignment with annotator distributions. These results demonstrate that jointly optimizing ensemble structure with a signed diversity regularizer provides an efficient, model-agnostic approach for modeling human subjectivity in supervised learning.

• The paper introduces a prediction-space framework that jointly learns reliability-aware ensemble weights and ensemble size while using a signed diversity regularizer to represent genuine annotator disagreement.
• EDO-Random achieved the lowest cross-entropy and Brier Score across all four LeWiDi 2023 benchmarks, reducing cross-entropy by up to 78% versus Soft-CE while maintaining competitive F1 performance.
• The results show that diversity works best as a complementary objective: preserving disagreement improves calibration under subjectivity, whereas suppressing it can help under severe class imbalance, highlighting the need for dataset-specific tuning.

Motivation and problem setting

Subjective NLP tasks—content moderation, hate speech detection, sentiment analysis—exhibit systematic annotator disagreement that reflects genuine interpretive multiplicity rather than annotation error. Prevailing practice aggregates annotations into a single target, discarding distributional information and overfitting to dominant interpretations. Soft-label supervision preserves the empirical annotator distribution but still optimizes a single predictive model, and partial-label learning (PLL) assumes a hidden ground-truth label, an assumption violated when multiple labels are simultaneously valid. The paper positions Ensemble Diversity Optimization (EDO) as an extension of PLL to settings of genuine multiplicity: rather than recovering a latent singleton, EDO represents the full distribution of human judgments.

The theoretical motivation draws on the unified theory of ensemble diversity [Wood et al., JMLR 2023], which decomposes cross-entropy-based ensemble error exactly into bias, variance, and diversity terms. Existing ensemble approaches treat diversity as emergent; EDO makes it an explicit, signed optimization objective.

Method

EDO operates entirely in prediction space over $K$ frozen pre-trained classifiers (BERT-base for English datasets, AraBERTv2 for Arabic), learning only reliability-aware ensemble weights $\mathbf{w}$ and effective cardinality $K$. The ensemble prediction is a convex combination $\hat{y}(x)=\sum_k w_k f_k(x)$, optimized through a joint multi-objective loss:

$$\min_{\mathbf{w},K}\ \lambda_{\text{F1}}\mathcal{L}_{\text{F1}} + \lambda_{\text{CE}}\mathcal{L}_{\text{CE}} + \lambda_{\text{Div}}\mathcal{L}_{\text{Div}}^{(s)} + \lambda_{\text{Reg}}\lVert\mathbf{w}\rVert_2^2$$

with four components:

• Soft micro-F1 surrogate on majority-vote hard labels, providing differentiable task utility.
• Class-weighted cross-entropy against soft labels (empirical annotator distributions), with inverse-frequency weights $\alpha_c$ to handle imbalance without distorting the target distribution.
• Signed, reliability-weighted diversity regularizer: a pairwise weighted $\ell_1$ disagreement term multiplied by a sign $s\in\{-1,+1\}$. Setting $s=-1$ encourages intra-ensemble dispersion to preserve epistemic uncertainty from systematic subjectivity; $s=+1$ suppresses dispersion when divergence is structurally induced (e.g., extreme class imbalance). Crucially, both the coefficient and sign are fixed, validation-tuned hyperparameters rather than learned end-to-end.
• $\mathbf{w}$0 regularization on weights to prevent collapse.

An appendix theorem establishes that the pairwise dispersion functional is bounded within a factor of two by the weighted deviation from the barycenter, so the signed objective directly controls predictive spread along the calibration–uncertainty Pareto frontier.

Two supervision strategies are supported: Random Select, which samples an annotator per instance as an unbiased stochastic estimator of expected loss under the annotator distribution (the default), and Per-Annotator, which pairs each ensemble member with a specific annotator and requires $\mathbf{w}$1 plus annotator identity metadata. Three cross-entropy aggregation variants (Random, Mean, All ensembles) are evaluated. Ensemble size is learned via Gumbel–Softmax relaxation over candidate sizes up to $\mathbf{w}$2, with annealed temperature and hard selection at inference ($\mathbf{w}$3). Model selection uses NSGA-II Pareto optimization over validation F1, CE, and MD, reporting the Pareto-optimal configuration with lowest CE.

Experimental results

Evaluation uses the four LeWiDi 2023 benchmarks (ArMIS, ConvAbuse, HS-Brexit, MD-Agreement), spanning moderate imbalance (ArMIS, 1.43:1), discourse-level subjectivity (ConvAbuse, 5.16:1), severe imbalance (HS-Brexit, 9.89:1), and sparse heterogeneous annotator coverage (MD-Agreement, 670 annotators). Metrics are micro-F1, cross-entropy against soft targets, Manhattan Distance, and Soft Brier Score.

Benchmark comparison

Against Soft-CE, Soft-MD, Top-5 Voting, and WEL, EDO-Random achieves the lowest CE on all four datasets, reducing CE by up to 78% relative to Soft-CE (ConvAbuse: 0.2149 vs. 0.9671) and 62% relative to WEL (0.2149 vs. 0.5577). It also attains the lowest Brier Score on all benchmarks (e.g., 0.0640 vs. WEL's 0.0699 on ConvAbuse; 0.1086 vs. 0.1606 on MD-Agreement), with BS and CE reductions strongly correlated ($\mathbf{w}$4). WEL retains the highest F1 scores, which the authors attribute to its utility-focused objective—an explicit utility–calibration trade-off rather than uniform dominance. EDO-PerAnn achieves the lowest MD on ConvAbuse and HS-Brexit but underperforms Random Select where annotator pools are small or fragmented (ArMIS's three annotators; MD-Agreement's sparse coverage). Five-seed standard deviations are low (mostly $\mathbf{w}$5), indicating stable calibration across seeds.

Ablations

Three findings stand out:

• Diversity is effective as a complementary regularizer, not standalone. Under single objectives, signed diversity yields modest changes; combined with the joint F1+CE objective, disagreement preservation ($\mathbf{w}$6) produces the largest calibration gain ($\mathbf{w}$7CE = −0.0993 dataset-averaged), and on severely imbalanced HS-Brexit it yields $\mathbf{w}$8CE = −0.4282.
• Reliability weighting consistently improves calibration. Learned versus uniform weights reduce CE in every setting, most dramatically on MD-Agreement ($\mathbf{w}$9CE = −0.2756 for EDO-Random, −0.5607 for EDO-PerAnn), with modest F1 effects.
• Aggregation variant interacts with diversity sign. Under moderate imbalance, Mean aggregation is most stable; under severe imbalance (HS-Brexit), suppression ($K$0) substantially reduces CE for Mean/All aggregation while preservation ($K$1) favors Random aggregation.

Sensitivity analysis via Spearman correlations shows that calibration emerges from synergistic gradient balancing rather than $K$2 alone, and that the learned ensemble size $K$3 correlates strongly and negatively with CE and BS on all datasets—the optimizer allocates members precisely when doing so improves calibration. Qualitative examples corroborate this: on highly disputed instances, EDO expands $K$4 (up to 7) and raises predictive entropy to match empirical annotator entropy, whereas baselines remain overconfident.

Limitations

The paper concedes several constraints. The effect of the signed diversity objective is inherently dataset-dependent, and EDO does not distinguish principled subjective variation from artifact-driven divergence caused by sparse coverage or inconsistent annotators—a limitation under distribution shift or mixed annotation quality. Hyperparameter sensitivity ($K$5, temperature settings) is dataset-specific, mitigated but not eliminated by Pareto-based selection. The prediction-space design with homogeneous frozen backbones limits representational flexibility, and Per-Annotator supervision degrades under sparse or heterogeneous annotator pools. The authors also note in the ethical statement that modeling disagreement does not mitigate biases embedded in annotations themselves. Open questions include dynamic objective reweighting during training, lightweight annotator representations within the ensemble, and mechanisms to infer disagreement structure (e.g., latent-variable or clustering approaches).

Conclusion

EDO contributes a unified, differentiable, prediction-space framework that jointly optimizes ensemble composition, cardinality, and calibration under subjective supervision, using a signed diversity regularizer as a directional control along the utility–calibration trade-off. Across four LeWiDi benchmarks it delivers large calibration gains—up to 78% CE reduction over single-model soft-label training and consistent Brier Score improvements—with competitive F1 and low seed variance, while remaining computationally lightweight and model-agnostic. The results support the paper's central claim that annotator disagreement carries structured signal, and that explicitly optimizing ensemble diversity is an efficient mechanism for representing it.