Per-Sample Score Aggregation Methods

• Per-sample score aggregation is a set of techniques that convert raw, noisy scores into robust statistical summaries, rankings, and calibrated predictions.
• It employs diverse methods such as exponential moving average filtering, quantile and order-statistic approaches, and submodular fusion to mitigate noise and redundancy.
• These techniques are crucial for applications in deep learning pruning, ensemble calibration, and NLP ranking, enhancing model reliability and prediction accuracy.

Per-sample score aggregation refers to the class of methods, theories, and computational mechanisms that transform sets or distributions of raw scores—assigned to individual samples, instances, or candidates—into statistical summaries, rankings, calibrated predictions, or collective decisions. These aggregation processes are central in scalable data-centric learning, robust inference, uncertainty quantification, and measurement science, particularly in situations involving redundancy, weak supervision, ensemble modeling, or combinatorial candidate sets.

1. Foundational Principles and Motivations

Per-sample score aggregation arises whenever a collection of possibly noisy, partial, redundant, or otherwise structurally diverse scores must be reduced to an actionable per-sample signal. Motivations are diverse:

• Reducing variance and bias: Noise in raw scores (from stochastic processes, batch effects, or randomness in modeling) is attenuated through aggregation (e.g., exponential moving average, robust statistics, or quantile-based summarization) (Zhou et al., 6 Apr 2026, Krebs, 2021).
• Redundancy and structure: In NLP, near-duplicates and paraphrases cause instability in raw scoring; aggregating across cluster structures regularizes sample-level rankings (Zheng et al., 1 May 2026).
• Weak/partial supervision: When only set-wise, top-k, or composite information is provided, aggregation allows models to learn meaningful per-sample statistics in the absence of fine-grained supervision (Legrand et al., 2016).
• Ensemble and model fusion: Multi-model systems require principled score-aggregation to produce collective uncertainty sets or optimal rankings (Alami et al., 7 Dec 2025, Qi et al., 2020).
• Calibration and guarantees: Aggregation in conformal prediction and risk control enables calibration of prediction sets or abstention with finite-sample reliability (Waldron, 8 Jan 2026, Gu et al., 13 May 2026).

Aggregation is not a single mathematical operation but a family of design choices, each with distinct tradeoffs in robustness, expressivity, and computational complexity.

2. Algorithmic and Statistical Methods

A rich taxonomy of per-sample aggregation mechanisms has emerged:

Exponential Moving Average (EMA) Filtering

Example: Batch Loss Score (BLS) in dynamic data pruning replaces explicit per-sample loss with an EMA of batch losses indexed per sample: $$S_i(t) = \begin{cases} \alpha S_i(t{-}1) + (1{-}\alpha) L(B_t, t), & \text{if } i \in B_t \ S_i(t{-}1), & \text{else} \end{cases}$$ Here, $L(B_t, t)$ is the batch mean loss; the EMA acts as a low-pass filter reducing high-frequency batch noise and provides a proxy for persistent sample-level loss (Zhou et al., 6 Apr 2026).

Quantile, Trimmed Mean, and Winsorized Aggregation

Classical robust estimators such as sample quantiles, $\alpha$-trimmed means, or Winsorized means combine per-sample scores to resist the influence of outliers: $$\bar{S}^{(\alpha)}_n = \frac{1}{M^0_n(1-2\alpha)}\sum_{i=a_n+1}^{b_n}S_{(i)}$$ Bahadur representations, functional central limit theorems, and law of the iterated logarithm results provide convergence rates and robustness guarantees (Krebs, 2021).

Order-Statistic and k-Min Aggregation

In conformal prediction for pooled one-shot predictors, the $k$-smallest nonconformity scores are averaged: $s(x,y) = \frac{1}{k}\sum_{j=1}^k s_{(j)}(x, y)$ Monotonicity under set expansion enables valid coverage guarantees with leave-one-out calibration, despite broken exchangeability (Waldron, 8 Jan 2026).

Symmetric and Permutation-Invariant Aggregation over Ensembles

Aggregating e-values or nonconformity scores from $M$ models by permutation-invariant functions (sum, mean, median, order-statistic) yields exchangeable aggregated scores: $F_i(y) = f(E_i^{(1)}(y),...,E_i^{(M)}(y))$ This supports sharper and often smaller uncertainty sets via split-conformal calibration (Alami et al., 7 Dec 2025).

Distributional and Information-Theoretic Aggregation

In anomaly detection via Isolation Forest ensembles, Rényi-divergence-motivated aggregation functions $f_\alpha$ interpolate between average and maximum over per-tree scores, tuning sensitivity to tails: $$f_\alpha(x) = n^{1/(α-1)} \left( \sum_{i=1}^n x_i^{1-α} \right)^{1/(1-α)}$$ This generalization captures the full distribution of individual anomaly votes (Dhouib et al., 2023).

Submodular and Lovász–Bregman Aggregation

Aggregating K sets of rank- or score-based permutations, as in distributed ASR or multi-view fusion, can leverage submodular set functions via the Lovász extension and LB divergence, supporting both linear and nested, non-linear, but submodular, score fusion (Qi et al., 2020): $L(B_t, t)$0 where $L(B_t, t)$1 is submodular and $L(B_t, t)$2 is induced by a permutation $L(B_t, t)$3.

Structure-Constrained and Multi-Seed Aggregation

To regularize sample rankings under data redundancy and stochasticity, aggregation may first smooth within clusters of redundant samples and then robustly fuse across runs (mean, median, Borda count), providing stability guarantees and reproducibility (Zheng et al., 1 May 2026).

Uniform and Adaptive Local Aggregation in ASD

For dense, bandwise data (e.g., time–frequency spectrograms), per-band nearest-neighbor distances are computed then aggregated via uniform mean or simple parameter-free fusion: $L(B_t, t)$4 Uniform integration outperforms min/max fusion, reducing normal-score variance in the presence of energy-coupling and local heterogeneity (Saengthong et al., 14 Mar 2026).

3. Applications Across Domains

The methodologies above instantiate in a range of major research areas:

Area	Application of Aggregation	Reference
Deep learning / data pruning	BLS for sample importance w/o per-sample loss	(Zhou et al., 6 Apr 2026)
Simulation-based inference	Score sum aggregation for composite posteriors	(Geffner et al., 2022)
Uncertainty quantification	Aggregation in CP for ensemble model sets	(Alami et al., 7 Dec 2025, Waldron, 8 Jan 2026, Gu et al., 13 May 2026)
Ranking & distributed modeling	LB submodular fusion across model/feature views	(Qi et al., 2020)
Acoustic anomaly detection (ASD)	Sub-band per-frame/local aggregation for robustness	(Saengthong et al., 14 Mar 2026)
Redundant NLP corpora	Multi-seed/cluster aggregation for ranking stability	(Zheng et al., 1 May 2026)
Quality evaluation (music, speech)	Per-sample MOS/MOS-like target via latent peak fit	(Kondo et al., 23 Jun 2025, Zhu et al., 24 Mar 2026)
Word alignment	Max/LSE aggregation over candidate alignment scores	(Legrand et al., 2016)

Each setting confronts distinct statistical, computational, or structural obstacles, but score aggregation mechanisms serve as the central technical solution.

4. Theoretical Properties and Guarantees

Several aggregation schemes admit nontrivial statistical and computational guarantees, with behaviors formalized through:

• Filter interpretation: EMAs as first-order low-pass filters, with quantifiable steady-state variance suppression and bias–variance tradeoff under frequency separation hypotheses (Zhou et al., 6 Apr 2026).
• Bahadur, CLT, and LIL for quantiles/trimmed means: Uniform-in-p representations deliver root-n consistency, robust limiting distributions, and almost sure fluctuation bounds (Krebs, 2021).
• Exchangeability and monotonicity: Leave-one-out k-min aggregation preserves type-I error guarantees via monotonicity—even when classical exchangeability is broken (Waldron, 8 Jan 2026).
• Permutation invariance: Aggregating e-values by symmetric functions preserves exchangeability of conformal sets and theoretical coverage (Alami et al., 7 Dec 2025).
• Submodularity: Submodular fusion ensures diminishing–returns, guarantees for greedy or coordinate descent optimization, and efficient approximate inference in high dimensions (Qi et al., 2020).
• Information-theoretic interpolations: Rényi-divergence aggregates tune the balance between sensitivity to anomalies and consensus robustness, via adjustably parameterized families (Dhouib et al., 2023).

These properties enable practitioners to select aggregation mechanisms with predictable risk, stability, and robustness tradeoffs for given problem settings.

5. Implementation and Practical Considerations

Implementation of per-sample score aggregation mechanisms varies significantly in complexity and system-level impact:

• Minimal code integration: BLS requires only three lines to wrap existing pruning frameworks, with negligible runtime or memory overhead (Zhou et al., 6 Apr 2026).
• Training and calibration: Latent peak fitting for speech/MOS or per-sample music quality scores integrates into established loss functions with minor changes (Kondo et al., 23 Jun 2025, Zhu et al., 24 Mar 2026).
• Clustering and seed budget: Robustness-oriented ranking schemes may demand prior clustering for redundancy detection, plus multiple training/inference seeds, with linear–adaptive overhead (Zheng et al., 1 May 2026).
• Permutation symmetry or submodular inference: Ensemble and multi-view settings admit both online and offline optimization; submodular and LB-divergence-based approaches require sorting, vector/matrix operations, and simplex or projection steps, but scale linearly with moderate dimensions (Alami et al., 7 Dec 2025, Qi et al., 2020).
• Pseudocode and runtime: Aggregation schemes are fully specified in algorithmic detail in cited works—for instance, band-wise ASD aggregation involves a few lines of feature extraction, pooling, and per-band matching (Saengthong et al., 14 Mar 2026).

Empirically, these methods achieve consistent improvements in sample selection, ranking stability, abstention error guarantees, and prediction set efficiency across diverse application domains.

6. Empirical Insights and Broader Impact

Empirical studies support the effectiveness of modern score aggregation:

• Pruning and model efficiency: BLS enables lossless pruning of 20–50% of training samples across 14 datasets and 18 models, matching or exceeding per-sample-loss baselines (Zhou et al., 6 Apr 2026).
• Simulation inference: Aggregation of single-observation scores enables robust posterior inference in complex, multimodal regimes with minimal simulator calls (Geffner et al., 2022).
• Uncertainty sets: Symmetric aggregation in SACP yields up to 10% tighter prediction intervals in regression/classification compared to best single-model conformal sets (Alami et al., 7 Dec 2025).
• ASD robustness: Sub-band uniform aggregation in BEAM/AdaBEAM reduces normal-score variance by 30–50%, yielding gains of 5–10 official-score points versus baselines under domain shift (Saengthong et al., 14 Mar 2026).
• Rank reproducibility: Multi-seed aggregation and cluster-based regularization in SCARV sharply improve the stability and reliability of data-centric decision-making in NLP pipelines (Zheng et al., 1 May 2026).
• Quality metrics: Quantized-distribution peak fitting for MOS targets increases utterance-level LCC/SRCC in speech quality models, and per-sample music quality metrics (MuQ-Eval) obtain utterance-level SRCC ≈0.84 and system-level SRCC ≈0.96 (Kondo et al., 23 Jun 2025, Zhu et al., 24 Mar 2026).
• Chain-of-thought aggregation: Weighted per-path scoring with conformal abstention achieves reliable confident-error control in complex LLM-based reasoning tasks (Gu et al., 13 May 2026).

These results demonstrate the versatility and practical potency of principled per-sample score aggregation in addressing statistical inefficiencies, instability, and intractability across modern machine learning, inference, and evaluation settings.