---
title: QE Re-ranking for MT and ASR
url: https://www.emergentmind.com/topics/quality-estimation-qe-re-ranking
type: topic
---

# QE Re-ranking for MT and ASR

Quality Estimation (QE) Re-ranking refers to a class of reference-free, model-driven selection strategies designed to improve the output of automatic sequence generation systems—most notably machine translation (MT) and automatic speech recognition (ASR)—by scoring and ranking candidate outputs according to predicted translation or recognition quality, rather than system-internal probabilities or external reference comparisons. QE re-ranking methods leverage learned metrics to choose the most reliable hypothesis from a candidate pool, balancing computational tractability with empirical correlation to human judgment.

## 1. Foundations and Motivation

The objective of QE re-ranking is to move beyond model-internal scoring, such as $p(y|x)$ in MT beam search or ASR decoder confidences, which frequently show only weak correlation with human assessments of quality. Instead, QE models are designed to predict a quality score for a candidate hypothesis $y$ given source input $x$, without access to references. Typical QE systems output a real-valued scalar $s(x, y)$, intended to approximate crowd-sourced human Direct Assessment (DA) scores in MT, or word error rate (WER) in ASR. This reference-free metric is then used to re-rank or combine candidates generated by one or multiple systems.

QE re-ranking is conceptually simple: generate a diverse pool of $N$ system outputs $\{y_i\}_{i=1}^N$, score each with a QE model, and select the hypothesis $y^*$ maximizing the estimated quality, i.e. $y^* = \arg \max_{i=1..N} s(x, y_i)$. This "best-of-$n$" approach typically yields better correlation with human preferences than choosing the highest-probability output under the generation model itself [2511.13884, 2502.08561, 2401.06688].

## 2. QE Model Architectures and Scoring Functions

Modern QE systems for MT predominantly use large pretrained architectures such as COMETKiwi, which builds on multilingual Transformer encoders, or task-specific variants that incorporate both source $x$ and candidate $y$ as joint inputs. The hidden representations $H(x, y)$ are pooled—often via mean or CLS token—and passed to a regression head that outputs a scalar $s(x, y)$, interpreted as the predicted quality [2511.13884]. Formally,

\[
s(x, y) = f_\theta\left(\text{Pool}\left(\text{Encoder}(x, y)\right)\right)
\]

Distinct approaches exist for applying QE metrics:

- **Full-sentence scoring:** Assign a score to each complete hypothesis.
- **Token-level or partial hypothesis QE:** Score prefixes during the decoding process, enabling integration of QE signals into search (see Section 4) [2502.08561].
- **Sliding-window or document-level QE:** For long inputs, windowed approaches such as SLIDE aggregate QE scores over local sentence batches to overcome input length limitations [2510.08870].
- **LLM-based QE metrics:** Direct assessment via prompting an instruction-tuned LLM, e.g., GEMBA-DA, scoring the quality of output across full documents [2510.08870].

In the ASR domain, QE models are often regression or pairwise ranking functions that predict word error rate (WER) based on signal, text, and hybrid features extracted from the hypothesis and source audio [1706.07238].

## 3. Algorithms and Workflows

The core QE re-ranking workflow can be formalized for MT as follows:

**Pseudocode for QE-Based Re-Ranking** [2511.13884]:
```python
def QE_ReRank(x, y_list, QE_model):
    best_score = -float('inf')
    best_hypothesis = None
    for y in y_list:
        score = QE_model.score(x, y)
        if score > best_score:
            best_score = score
            best_hypothesis = y
    return best_hypothesis
```

**Advanced variants include:**

- **Quality-Aware Decoding (QADec):** Integrates token-level QE with NMT model scores at each beam search expansion, merging model likelihood with reference-free prefix-wise quality scores [2502.08561]
- **QE-fusion:** Instead of only selecting among $N$ full candidates, QE-fusion combines complementary spans from multiple candidates by iterating over divergent segments, synthesizing new hypotheses and scoring each with QE. This approach enables construction of novel outputs not present in the original candidate set [2401.06688].
- **Document-level aggregation:** In SLIDE, source-output pairs are evaluated on overlapping windows (e.g., 7 sentences per window), with the overall document score as the mean of these local QE estimates [2510.08870].
- **ASR QE-guided fusion:** For each segment in an utterance, hypotheses are re-ranked by predicted WER or pairwise rank before being aligned and voted by ROVER, improving over confidence-based or random ordering [1706.07238].

## 4. Comparative Evaluation and Empirical Results

Evaluation of QE re-ranking methods relies on neural metrics (COMET, BLEURT), reference-based scores (BLEU, ChrF), and in ASR, WER reduction.

**Key findings:**
- **MT QE re-ranking with COMETKiwi:** Achieves consistent positive $\Delta$COMET (e.g., +0.0201) on segment-level error correction tasks, with per-language improvements up to +0.0365 [2511.13884].
- **Document-level translation:** Applying SLIDE or GEMBA-DA for re-ranking yields +2.00 to +5.09 BLEURT-20 improvements (with 2–32 candidates) at document-level, with maintained gains even for sequences of up to 1024 tokens [2510.08870].
- **Quality-Aware Decoding:** Integration of token-level QE during search outperforms N-best re-ranking, with XCOMET-XXL gains up to +1.4; benefits are greatest for long documents [2502.08561].
- **QE-fusion vs QE-reranking:** QE-fusion consistently surpasses standard QE-best-of-$n$ by +0.7–1.2 COMET and +0.8–1.4 BLEURT points for LLM-generated candidate pools, and produces novel outputs in over 50% of cases when $N$ is large [2401.06688].
- **ASR ROVER re-ordering:** Applying QE-based ranking prior to confusion network combination yields WER reductions of up to 7.3%, narrowing the gap to segmentation oracles without requiring decoder confidences [1706.07238].

| Method                | Typical Gains           | Scale Dep. | Reference       |
|-----------------------|------------------------|------------|-----------------|
| QE-reranking (MT)     | +0.02 $\Delta$COMET    | $N$ linear | [2511.13884]    |
| QE-fusion (MT)        | +0.7–1.2 COMET         | $N$ linear (5x) | [2401.06688] |
| SLIDE doc-level (MT)  | +2.00–5.09 BLEURT-20   | $N$ linear | [2510.08870]    |
| Token-level QADec     | +0.3–1.4 XCOMET-XXL    | Beam size  | [2502.08561]    |
| QE re-rank ROVER (ASR)| up to –7.3% abs. WER   | $N$ streams| [1706.07238]    |

## 5. Computational Characteristics and Practical Usage

QE re-ranking algorithms are computationally attractive due to the linear scaling of metric evaluations with candidate pool size $N$, in contrast to quadratic scaling in Minimum Bayes Risk (MBR) decoding, which requires $O(N^2)$ pairwise utility computations. QE-fusion incurs approximately 5 times the overhead of QE-reranking, remaining tractable for candidate pools of practical size ($N\leq25$) [2401.06688]. The overall runtime is usually dominated by the cost of generating hypotheses, not by metric evaluation, especially for large language models [2510.08870].

**Empirically validated usage guidelines include:**
- Small pools ($N=5$–10) realize most performance gains.
- Performance saturates beyond $N\approx25$.
- Higher candidate diversity (via temperature in sampling) increases the efficacy of fusion methods.
- For long documents, windowed or sliding evaluation circumvents input length limits [2510.08870].
- In ASR, QE-guided re-ranking is most effective when combining multiple diverse systems or channels and when decoder features are unavailable [1706.07238].

## 6. Advantages, Limitations, and Future Directions

**Advantages:**
- Reference-free and model-agnostic: compatible with any candidate generator.
- Linear computational scaling in candidate pool size.
- Robust empirical improvements across MT and ASR.
- QE-fusion supports output diversity by synthesizing novel combinations.

**Limitations:**
- Increased computational load as the number of candidates or span diversity rises.
- Reliance on automatic metrics (e.g., COMETKiwi) with no guarantee of alignment to subjective human quality, particularly in low-resource or domain-mismatched settings [2511.13884].
- For document-level tasks, performance declines as input length increases due to QE model context limitations; windowing partially mitigates this [2510.08870].
- Mask-and-fill and substring correction strategies guided by QE spans are sensitive to prompt engineering and may underperform reranking [2511.13884].
- Gains are reduced when candidate pool diversity is low, or when hypotheses are very homogeneous [1706.07238].

**Potential research directions include:**
- Human-in-the-loop assessments to fully validate reference-free QE metrics in target usage.
- Multi-stage or cascaded correction integrating lightweight masked LMs for edit proposal and larger LLMs for global ranking [2511.13884].
- Systematic selection of generator subsets to optimize cost-benefit trade-off in candidate generation [2511.13884].
- Specialization of QE models for token-level or streaming use, especially in document translation [2502.08561].
- Multitask and domain-adaptive QE to broaden applicability across tasks and datasets [1706.07238].

## 7. Broader Context and Cross-modal Application

While QE re-ranking is most mature within MT, it finds effective application in ASR system combination, notably within the ROVER framework. In settings where underlying decoder confidence scores are unavailable or uninformative, black-box QE—predicting WER from hybrid acoustic, textual, and contextual features—can order candidate streams segment-wise, improving downstream majority voting fusion and approaching oracle-level WER [1706.07238].

This methodological transferability suggests a general applicability of QE re-ranking principles to other structured generation tasks, wherever candidate diversity and reference-free evaluation are feasible and operationally advantageous.

Source: https://www.emergentmind.com/topics/quality-estimation-qe-re-ranking