---
title: Label-Free Score Fusion
url: https://www.emergentmind.com/topics/label-free-score-fusion
type: topic
---

# Label-Free Score Fusion

Label-free score fusion is the combination of multiple score streams into a single decision statistic without using ground-truth labels to estimate the fusion rule itself. The surveyed literature indicates that the term is used in several distinct senses: strictly zero-label estimation, test-time label-agnostic deployment after supervised calibration, and training-free but validation-tuned interpolation. Across biometrics, retrieval, object detection, vision-language recognition, image quality assessment, and LLM verification, the recurring technical problem is score commensuration: heterogeneous systems produce outputs on different scales, with different dependence structures and failure modes, so fusion quality depends at least as much on calibration and reliability modeling as on the final algebraic combiner [2604.18547][2603.28886].

## 1. Terminological scope and taxonomy

The surveyed work supports a strict distinction between **label-free**, **training-free**, and **test-time label-agnostic** fusion. In the strictest sense, a method is label-free only if it estimates fusion behavior without any ground-truth correctness labels, same/different labels, or development-set relevance labels. FUSE is explicit on this point: it performs verifier ensembling with “zero ground truth labels” by estimating verifier qualities from unlabeled score matrices [2604.18547]. SUEL likewise operates on unlabeled continuous predictor scores and estimates both predictor ranking and fusion weights without ground truth [2408.07796]. The deep MAP IQA framework also fits its fusion model without MOS or other human labels, treating multiple IQA scores as noisy observations of a latent perceptual-quality variable [2605.30269].

A second category is label-free only in a narrower operational sense. PhaseGraph calibrates graph and vector scores to a common percentile-based scale without relevance labels, but the mixture weight and pool settings are chosen on tune splits [2603.28886]. The conversational-memory retrieval study is explicitly “training-free,” yet its interpolation weight is selected by leave-one-conversation-out validation, so it is not strictly label-free [2606.04194]. ACT in FusionAgent fuses scores for a test query without test labels, but it is embedded in an agent-selection framework trained with supervised rewards [2603.26908]. DBF similarly performs automatic test-time fusion from detector outputs, but its precision–recall confidence model is built from labeled validation data [2204.02890].

A third category comprises methods that are not label-free even if the deployed combiner is simple. Morrison’s logistic-regression calibration and fusion requires labeled same-origin and different-origin comparisons throughout parameter estimation [2104.08846]. The face–ECG hybrid identification method computes per-subject modality weights from labeled training folds, so it is “not truly label-free” despite its simple algebraic fusion rule at test time [2008.03353].

| Category | Representative papers | Label status |
|---|---|---|
| Strict zero-label estimation | FUSE [2604.18547], SUEL [2408.07796], deep MAP IQA [2605.30269] | No ground-truth labels used to estimate fusion |
| Label-free calibration with tuned fusion | PhaseGraph [2603.28886], lexical–dense fusion [2606.04194] | Calibration is score-distribution based; fusion weights are tune-set selected |
| Test-time label-agnostic fusion after supervised setup | DBF [2204.02890], ACT [2603.26908] | No test labels needed, but labeled training or validation is required |
| Supervised score fusion | Logistic-regression fusion [2104.08846], face–ECG hybrid [2008.03353] | Fusion parameters depend on labeled data |

This taxonomy suggests that “label-free score fusion” is not a single methodological family. It is better understood as a spectrum defined by where labels enter: nowhere, only in offline tuning, or directly in fusion-model estimation.

## 2. Problem formulations and fusion objects

The object being fused varies substantially across application domains. In LLM verification, FUSE assumes a generator produces \(N\) candidate responses \(r_1,\dots,r_N\) for a query \(q\), and \(m\) verifiers produce a score matrix
\[
V = (v_{i,j})_{i=1,\dots,N;\,j=1,\dots,m}, \qquad v_{i,j}=v_j(q,r_i),
\]
with unknown latent correctness labels \(y_i\in\{\pm 1\}\) [2604.18547]. Fusion is therefore candidate ranking under latent binary correctness.

In heterogeneous retrieval, the fusion objects are scalar retrieval scores. PhaseGraph combines vector-retrieval scores \(s_i^v\) and graph-retrieval scores \(s_i^g\) for the same document, treating graph–vector fusion as a score-calibration problem rather than a purely combinatorial one [2603.28886]. The conversational-memory study similarly fuses a session-level BM25 score with a dense late-interaction session score after within-candidate-set normalization [2606.04194].

In zero-shot multi-label recognition, SPARC fuses singleton prompt scores \(s_i^t\) with order statistics of compound-prompt scores \(\{\overline{s}_p^t : i\in C(p)\}\) to refine per-class rankings [2502.16911]. In multimodal biometrics, the face–ECG hybrid method does not fuse scalar matcher outputs but two heterogeneous **87-dimensional class-score vectors**, one for face and one for ECG, and then decides by
\[
\arg\max(F_{k,n}),
\]
so the fusion target is a multiclass confidence vector rather than a single verification score [2008.03353].

In unsupervised IQA score fusion, the fused quantity is a latent scalar quality \(z_i\), with observed model scores
\[
x_i = \{x_i^j \mid j=1,\dots,M\},
\]
modeled as noisy observations of that latent variable [2605.30269]. In SUEL, the fused output is again a scalar meta-score,
\[
g(x_t)=\sum_{i=1}^M w_i f_i(x_t),
\]
but the inputs are arbitrary continuous predictor scores rather than calibrated match or retrieval scores [2408.07796].

This diversity matters because label-free fusion cannot be reduced to one canonical formula. The same term covers at least four technically distinct settings: latent-class verification, score calibration for retrieval, class-wise ranking refinement, and latent-variable estimation from multiple regressors.

## 3. Principal methodological families

One family centers on **distributional commensuration**. PhaseGraph maps each score source to percentile rank,
\[
\hat{p}_i^k = \frac{|\{j : s_j^k \leq s_i^k\}|}{N_k},
\]
so vector and graph scores become comparable on a unit-free scale before fusion [2603.28886]. SPARC performs two-stage standardization: first within image across prompts, then across images for each prompt, specifically to remove image-level and prompt-level biases before adaptive fusion [2502.16911]. ACT uses per-query Z-score normalization,
\[
z_{m,q,g} = \frac{s_{m,q,g} - \mu_{m,q}}{\sigma_{m,q}},
\]
to turn each model’s score vector into a confidence-like signal before anchor-based fusion [2603.26908]. These methods differ in application, but all treat score comparability as the primary obstacle.

A second family uses **unlabeled latent-structure estimation**. FUSE searches for score transformations that reduce empirical triplet conditional-independence violations, then uses spectral moment estimation to recover verifier sensitivities and specificities without correctness labels [2604.18547]. SUEL replaces independence assumptions by a latent-group model: predictors are noisy perturbations of a smaller number of latent score sources, and fusion weights are estimated from the covariance structure of continuous unlabeled scores [2408.07796]. The IQA MAP method posits a latent quality variable and model-specific observation channels with heteroscedastic skew-normal noise, so fusion becomes latent-variable inference rather than rule-based averaging [2605.30269].

A third family uses **training-free or weakly tuned arithmetic fusion**. The conversational-memory study fuses BM25 and dense late-interaction scores by weighted interpolation after \(z\)-normalization within the candidate set, using a single leave-one-conversation-out scalar weight [2606.04194]. The biometric study on fingerprint and finger-vein compares min, max, simple sum, and user weighting after score normalization, reporting the best result for hyperbolic-tangent normalization plus simple sum [1805.10666]. These methods are operationally simple, but their status as label-free depends on whether normalization or weighting parameters were estimated with labels.

A fourth family is **quality- or consensus-conditioned fusion**. The BioSecure benchmark showed that top-performing systems under cross-device mismatch were those that exploited quality measures to condition fusion [2111.08703]. CCS is adjacent rather than identical to prediction fusion: it fuses agreement across test-time augmentations into a deployment-time reliability score,
\[
CCS = \frac{1}{M(M-1)} \sum_{i=1}^{M} \sum_{\substack{j=1\\ j \neq i}}^{M} \gamma_{ij},
\]
and is explicitly a label-free monitoring metric rather than a final detector-output combiner [2509.12871].

## 4. Representative formulations

PhaseGraph provides a concise example of label-free calibration followed by fusion. After percentile-rank normalization, scores are transformed to energies
\[
E_i^k = -\ln(\hat{p}_i^k + \epsilon),
\]
then to Boltzmann probabilities
\[
P_i^k = \frac{\exp(-E_i^k / T_k)}{Z_k},
\]
and fused as
\[
\text{score}(d_i) = \alpha \cdot P_i^v + (1-\alpha) \cdot P_i^g + \beta \cdot \mathbb{1}[d_i \in \mathcal{R}_v \cap \mathcal{R}_g].
\]
The paper’s main conclusion is that percentile-based commensuration is the robust design choice, while the exact post-calibration operator matters less [2603.28886].

SPARC is a different label-free construction. For class \(i\), it forms order statistics \(r_{i,k}^t\) of debiased compound-prompt scores and learns an unsupervised fusion direction by maximizing variance across images:
\[
w^{i*} := \argmax_{w^i} \textrm{Var}_t\!\left(w_0^i \bar{s}_i^t + \sum_{k} w_k^i r_{i,k}^t \right),
\]
\[
\tilde{\zeta}_i^t := w_0^{i*} \bar{s}_i^t + \sum_{k} w_k^{i*} r_{i,k}^t,
\]
followed by
\[
\zeta_i^t := s_i^t + \tilde{\zeta}_i^t.
\]
The paper’s ablations show that the first maximum compound score is often inferior to weaker order statistics because compound prompts behave mainly like an OR gate with an AND bonus [2502.16911].

FUSE represents the most explicit zero-label ensembling formulation. It binarizes or otherwise monotonically transforms verifier scores by choosing
\[
\tau^\star \in \arg\min_{\tau\in\mathcal T} \hat{\mathcal S}(g_\tau(V)),
\]
where \(\hat{\mathcal S}\) measures empirical triplet conditional-independence violation, then estimates triplet-based correctness posteriors and fits a final ensemble by
\[
\theta^\star := \arg\max_\theta \sum_{i=1}^N (2\hat p(r_i)-1)\,\hat f_\theta(V_{i\bullet}).
\]
Its distinctive feature is that dependency control is performed without labels, by optimizing score transformations rather than learning from correctness annotations [2604.18547].

SUEL gives the most direct unsupervised linear fusion rule for continuous scores:
\[
g(x_t)=\sum_{i=1}^M w_i f_i(x_t).
\]
The weights are inferred from a structured covariance model,
\[
R = G + \Theta,
\]
or equivalently
\[
R = B A B^T + \Theta,
\]
where latent groups explain predictor dependence [2408.07796].

The IQA MAP framework is the clearest latent-variable formulation. Each observed score is modeled as
\[
x_i^j = f^j(z_i) + n_i^j + n^j,
\]
with score-dependent uncertainty \(w_i^j=g^j(z_i)\) and final fused score approximated by an encoder
\[
z_i = e(x_i;\theta_e).
\]
The system is trained by maximizing the posterior of \(z\) under this observation model without MOS labels [2605.30269].

## 5. Empirical evidence across domains

Zero-label fusion can be competitive with supervised or semi-supervised baselines when the latent-structure assumptions are adequate. FUSE reports strong results on multiple best-of-\(N\) reasoning benchmarks. On Humanity’s Last Exam, for best-of-50 selection, Pass@1 is \(52.1\), Naive Ensemble is \(51.4\), Oracle Best Verifier is \(53.5\), Logistic Regression is \(53.4\), WEAVER is \(51.2\), and FUSE reaches \(54.3\) [2604.18547]. On 70B-generator GPQA Diamond, FUSE improves over Naive Ensemble from \(57.6\) to \(64.4\), essentially matching or slightly exceeding WEAVER at \(64.1\) [2604.18547].

In heterogeneous retrieval, label-free calibration also yields measurable gains. PhaseGraph improves held-out LastHop@5 from \(75.1\%\) to \(76.5\%\) on MuSiQue and from \(51.7\%\) to \(53.6\%\) on 2WikiMultiHopQA, both on independent held-out test splits [2603.28886]. The same paper’s ablation shows that percentile-based calibration is directionally more robust than min-max normalization, and that linear fusion after calibration performs comparably to Boltzmann weighting [2603.28886].

SPARC shows that zero-shot, training-free score fusion can materially improve multi-label ranking. Averaged over nine CLIP backbones, vanilla ZSCLIP rises from \(55.7\) to \(68.3\) on COCO, from \(80.4\) to \(89.2\) on VOC, and from \(39.3\) to \(47.2\) on NUSWIDE when SPARC’s debiasing and adaptive fusion are applied [2502.16911]. The same study reports that debiasing alone raises average performance from \(58.5\) to \(66.2\), which suggests that score commensuration is a major part of the gain [2502.16911].

The IQA MAP method reports average SRCC \(0.8869\) and PLCC \(0.9002\) for its score-fusion variant with model- and score-level uncertainty, improving over individual IQA models and over unsupervised rank-fusion baselines [2605.30269]. When two deliberately bad random-score models are added, its performance remains essentially stable, whereas RRF and RRFW degrade sharply, which the paper attributes to uncertainty-driven suppression of inconsistent models [2605.30269].

Training-free arithmetic fusion can also be effective, though not uniformly. In conversational-memory retrieval, fusing BM25 with dense late interaction raises LoCoMo Hit@1 for e5-large-v2 from \(0.664\) to \(0.752\), with NDCG@5 reaching \(0.829\) [2606.04194]. The same paper reports that on LongMemEval-S, a lexical-overlap-heavy regime, the net fusion gain over BM25 on R@1 is only \(+0.67\) percentage points, with 95% CI \([-2.67,+4.00]\) and \(p=0.43\), so the gain is small and not statistically significant [2606.04194]. This suggests that the value of label-free or weakly tuned fusion is regime-dependent rather than universal.

## 6. Limitations, misconceptions, and adjacent directions

A persistent misconception is that any simple algebraic combiner is label-free. The surveyed work shows otherwise. The face–ECG hybrid method computes a class-dependent weighted sum at test time, yet its weights come from labeled per-subject training behavior [2008.03353]. Morrison’s logistic-regression fusion is explicitly supervised, requiring same-origin and different-origin labels for both calibration and fusion [2104.08846]. DBF is label-free only at deployment; it still requires labeled validation data to convert detector scores into belief masses [2204.02890].

A second misconception is that fusion quality depends mainly on the final operator. Several papers point in the opposite direction. PhaseGraph argues that once vector and graph scores are brought to a common percentile scale, the exact post-calibration operator matters less [2603.28886]. SPARC’s ablations show that naive maxima are often suboptimal unless singleton and compound scores are first debiased [2502.16911]. ACT’s ablation reports that Z-score normalization materially outperforms no normalization or min-max normalization on LTCC, especially in FNIR [2603.26908]. These results suggest that the main design burden in label-free fusion is often commensuration rather than aggregation.

Dependence and redundancy are also central. FUSE was motivated by the failure of prior spectral methods under strong conditional correlations between verifiers [2604.18547]. SUEL likewise assumes that predictors are dependent, but explains this dependence through latent groups rather than ignoring it [2408.07796]. A plausible implication is that future label-free fusion systems will increasingly combine commensuration with explicit dependence control rather than relying on independence assumptions.

Operational robustness remains unresolved. The BioSecure benchmark showed that quality-aware systems are best under device mismatch, and that sequential methods are especially stable under missing scores [2111.08703]. This indicates that label-free fusion in deployment cannot be treated as a score-only problem; quality indicators, missingness patterns, and acquisition cost are integral to realistic fusion. CCS reinforces this point from a different angle: it uses unlabeled agreement across augmentations as a reliability proxy and is intended for deployment-time monitoring, not final prediction fusion [2509.12871]. This suggests an adjacent research direction in which consensus scores, quality measures, and label-free fusion are combined rather than studied separately.

The literature therefore supports a narrow but technically meaningful definition of label-free score fusion: unlabeled estimation of a fused score from heterogeneous sources, with explicit attention to commensuration, dependence, and reliability. The strongest current examples are zero-label verifier ensembling, covariance-structured unsupervised predictor fusion, latent MAP estimation over score channels, and score-distribution-based calibration. The main remaining gap is end-to-end elimination of tune-set labels without sacrificing robustness under domain shift, missing data, or heterogeneous score scales.

Source: https://www.emergentmind.com/topics/label-free-score-fusion