---
title: Dual-Granularity WER-Gated Policy Optimization
url: https://www.emergentmind.com/topics/dual-granularity-wer-gated-policy-optimization
type: topic
---

# Dual-Granularity WER-Gated Policy Optimization

Searching arXiv for the cited paper and closely related optimization context.
Dual-Granularity WER-Gated Policy Optimization (DG-WGPO) is a reinforcement-learning objective introduced in Mega-ASR, a unified ASR-in-the-wild framework that combines scalable compound-data construction with progressive acoustic-to-semantic optimization [2605.19833]. DG-WGPO is designed for adverse-condition speech recognition settings in which conventional word error rate (WER) ceases to provide sufficiently discriminative supervision. It decomposes reward shaping into a token-level “refinement” reward and a sentence-level “reconstruction” reward, then applies a WER-threshold gate that shifts emphasis between them according to the severity of recognition failure. Within the broader Mega-ASR system, this optimization stage is trained from an Acoustic-to-Semantic Progressive Supervised Fine-Tuning checkpoint and forms part of a pipeline that reports gains on adverse-condition ASR benchmarks and complex compositional acoustic scenarios [2605.19833].

## 1. Position within the Mega-ASR framework

Mega-ASR is presented as a response to an “acoustic robustness bottleneck,” described as a setting in which ASR systems and large audio-language models often lose acoustic grounding and produce omissions or hallucinations under severe, compositional distortions [2605.19833]. The framework combines scalable compound-data construction with progressive acoustic-to-semantic optimization, and DG-WGPO is the policy-optimization component of that training stack.

The surrounding system context is important because DG-WGPO is not introduced as a standalone reward heuristic detached from data or pretraining. Mega-ASR also introduces Voices-in-the-Wild-2M, covering 7 classic acoustic phenomena and 54 physically plausible compound scenarios. The full system is trained with Acoustic-to-Semantic Progressive Supervised Fine-Tuning and DG-WGPO. Extensive experiments report 45.69% vs. 54.01% on VOiCES R4-B-F and 21.49% vs. 29.34% on NOIZEUS Sta-0, and on complex compositional acoustic scenarios the system delivers over 30% relative WER reduction against strong open- and closed-source baselines [2605.19833]. These figures characterize Mega-ASR as a whole rather than isolating DG-WGPO alone.

## 2. Motivation: local versus global failure regimes

The method is motivated by an explicit distinction between two acoustic-error regimes. In moderately noisy conditions, defined as approximately $\mathrm{WER} \lesssim 30\%$, recognition errors are described as mostly local word- or token-level mistakes. In harsher conditions, approximately $\mathrm{WER} \gtrsim 30\%$, the dominant failures become global: dropped phrases, hallucinations, or large-scale omissions [2605.19833].

A single WER reward is therefore treated as insufficient because it conflates these regimes and saturates when WER is high, offering little learning signal when it is most needed. DG-WGPO addresses this by splitting the reward into two components. The low-granularity component is a token-level “refinement” reward intended to sharpen local word corrections. The high-granularity component is a sentence-level “reconstruction” reward intended to preserve overall semantic structure. A WER-threshold gate then dynamically shifts emphasis from one granularity to the other.

A common simplification is to assume that conventional WER alone is an adequate RL reward for robust ASR. The reported motivation argues against that simplification: when outputs are severely corrupted, scalar WER can fail to discriminate among qualitatively different failure patterns. This suggests that DG-WGPO should be understood as regime-dependent reward shaping rather than as a replacement for WER per se.

## 3. Low- and high-granularity reward definitions

Let $H$ be the hypothesis sequence of tokens, $R$ the reference sequence, $|\cdot|$ the length in tokens, and $\mathrm{WER}(H,R)$ the conventional word error rate [2605.19833]. DG-WGPO defines two reward granularities.

For substituted token pairs $(h,r)$, the method first defines a character-level substitution similarity:
$$
\mathrm{sim}(h,r) = 1 - \frac{\mathrm{edit}(h,r)}{\max(|h|,|r|)} \in [0,1],
$$
where $\mathrm{edit}(\cdot)$ is the character-level edit distance.

The token-level refinement reward, denoted $R_{\mathrm{fine}}$, uses three counts: $n_C$, the number of correct tokens; $n_{\mathrm{hard}}$, the number of “hard” errors consisting of insertions, deletions, or substitutions with $\mathrm{sim}<0.5$; and $n_{\mathrm{soft}}$, the number of “soft” substitutions with $\mathrm{sim}\ge 0.5$. The reward is
$$
R_{\mathrm{fine}} =
\frac{n_C}{n_C + n_{\mathrm{hard}} + \alpha_s \cdot n_{\mathrm{soft}} + \epsilon},
$$
with $\alpha_s \in (0,1)$ down-weighting soft errors. The paper uses $\alpha_s = 0.4$ and $\epsilon = 1\mathrm{e}{-8}$.

The sentence-level reconstruction reward, denoted $R_{\mathrm{struc}}$, uses the longest common subsequence in tokens:
$$
R_{\mathrm{struc}} =
\frac{1}{2}\cdot \frac{\mathrm{LCS}(H,R)}{|R|}
+
\frac{1}{2}\cdot \max\left(0,\,1 - \frac{\big||H|-|R|\big|}{|R|}\right).
$$
This formulation combines a subsequence-preservation term with a length-consistency term. The first term favors retention of ordered reference structure; the second penalizes large hypothesis-reference length mismatch.

The distinction between the two rewards is operational rather than merely descriptive. $R_{\mathrm{fine}}$ sharpens local lexical correction, whereas $R_{\mathrm{struc}}$ measures whether the hypothesis still tracks the global sentence scaffold. A plausible implication is that the method treats severe acoustic corruption primarily as a structural reconstruction problem rather than solely as token replacement.

## 4. WER-gated fusion and the final reward objective

DG-WGPO introduces a gate based on a WER threshold $\tau$, set to $\tau = 0.3$ in the paper [2605.19833]. The dynamic reward is defined piecewise:
$$
\text{if } \mathrm{WER}(H,R) < \tau:
\quad
R_{\mathrm{dynamic}} = 0.75\cdot R_{\mathrm{fine}} + 0.25\cdot R_{\mathrm{struc}},
$$
$$
\text{else}:
\quad
R_{\mathrm{dynamic}} = 0.25\cdot R_{\mathrm{fine}} + 0.75\cdot R_{\mathrm{struc}}.
$$

This gate implements the paper’s central design principle. In lower-WER conditions, token refinement is emphasized. In higher-WER conditions, sentence reconstruction becomes dominant. The piecewise weights are fixed and asymmetric rather than continuously learned.

The method does not discard conventional WER. Instead, it keeps a static WER reward anchor:
$$
R_{\mathrm{wer}}(H,R) = 1 - \mathrm{WER}(H,R).
$$
It also defines an anti-repetition gate:
$$
R_{\mathrm{rep}}(H) =
\begin{cases}
0, & \text{if } H \text{ contains repeated } n\text{-grams beyond a small threshold},\\
1, & \text{otherwise}.
\end{cases}
$$
The static combined signal is then
$$
R_{\mathrm{simple}} = R_{\mathrm{rep}} \cdot R_{\mathrm{wer}}.
$$

The final reward interpolates between the static anchor and the gated dynamic signal:
$$
R(H,R) = (1-\alpha_{\mathrm{dyn}})\cdot R_{\mathrm{simple}} + \alpha_{\mathrm{dyn}}\cdot R_{\mathrm{dynamic}},
$$
with $\alpha_{\mathrm{dyn}} \in (0,1)$ and the paper using $\alpha_{\mathrm{dyn}} = 0.6$ [2605.19833].

This design makes WER an anchor rather than the sole optimization target. The anti-repetition gate additionally constrains degenerate rollouts. The resulting objective can be read as a two-layer reward structure: a static baseline that preserves conventional ASR fidelity and a dynamic component that reallocates credit assignment according to error regime.

## 5. Policy optimization procedure

The optimization is performed in DAPO style, where DAPO is “Direct Advantage-based Policy Optimization” [2605.19833]. The policy $\pi_\theta$ is a pretrained ASR policy initialized from an A2S-SFT checkpoint. Given an audio/text prompt, it generates a distribution over token sequences.

For each RL iteration, a minibatch $B \subset D$ is sampled from the RL training set $D = \{(\mathrm{audio}_i,\mathrm{ref}_i)\}$. For each $(\mathrm{audio},\mathrm{ref}) \in B$, the policy generates $K$ completions $\{h_k\}$ by sampling, for example with top-$p$ and temperature. For each sampled completion, the procedure computes $\mathrm{WER}_k$, the static reward anchor $R_{\mathrm{wer},k}=1-\mathrm{WER}_k$, the repetition gate $R_{\mathrm{rep},k}$, the combined static signal $R_{\mathrm{simple},k}$, the token-level reward $R_{\mathrm{fine},k}$, and the sentence-level reward $R_{\mathrm{struc},k}$. The gate then selects the appropriate mixture to obtain $R_{\mathrm{dynamic},k}$, and the final scalar reward is
$$
R_k = (1-\alpha_{\mathrm{dyn}})R_{\mathrm{simple},k} + \alpha_{\mathrm{dyn}}R_{\mathrm{dynamic},k}.
$$

The update uses group-relative normalization. For the $K$ rollouts associated with a prompt, the group-mean reward is
$$
\mu = \mathrm{mean}_k\, R_k,
$$
and the advantage is
$$
A_k = R_k - \mu.
$$
The method then accumulates log-probabilities $\log \pi_\theta(h_k \mid \mathrm{audio})$ and forms the policy gradient
$$
\nabla_\theta = \sum_{(\mathrm{batch})}\sum_{k=1}^{K} A_k \cdot \nabla_\theta \log \pi_\theta(h_k).
$$
Parameters are updated via gradient ascent, or an equivalent PPO/DAPO update with KL clipping [2605.19833].

The procedural emphasis on group-relative normalization is not incidental. The reproduction notes state that it is critical to expose intra-group signal. This suggests that DG-WGPO depends not only on the reward definition but also on the variance structure induced by multiple sampled completions per prompt.

## 6. Empirical comparison, ablations, and reproduction conditions

The paper contrasts DG-WGPO with single-granularity WER optimization [2605.19833]. In that comparison, “single-granularity WER reward” corresponds to $R_{\mathrm{simple}}$ alone. The reported conceptual limitation is that WER-only reward saturates in high-WER regimes, offering little discrimination when policies produce severely corrupted outputs. Empirically, Table 4 reports that replacing DG-WGPO by WER-only rewards yields higher WER on noisy benchmarks. On the Voices benchmark, Vanilla DAPO (WER reward only) gives WER $7.62\%$, whereas full DG-WGPO gives $7.35\%$. Ablations further show that removing $R_{\mathrm{fine}}$, removing $R_{\mathrm{struc}}$, or removing the gated fusion each degrades performance, which the paper interprets as evidence that both granularities and their dynamic weighting are necessary for maximal gains.

The reported reproduction settings are summarized below.

| Parameter | Value | Note |
|---|---:|---|
| WER gate $\tau$ | 0.3 | Threshold for dynamic weighting |
| Soft-error discount $\alpha_s$ | 0.4 | Down-weights soft substitutions |
| Dynamic weight $\alpha_{\mathrm{dyn}}$ | 0.6 | Weights dynamic signal |
| Rollouts per prompt $K$ | 12 or 16 | Depends on run |
| RL learning rate | $5\mathrm{e}{-5}$ | Cosine decay schedule, warmup 3% |
| KL-penalty coefficient $\beta$ | 0.04 | DAPO setting |
| DAPO clipping | 0.28 | Update constraint |
| Temperature | 0.5 | Generation parameter |
| Top-$p$ | 0.95 | Generation parameter |
| Repetition penalty | 1.08 | Generation parameter |
| Max length | 256 tokens | Generation parameter |
| Hardware scale | 3–8 GPUs | DAPO framework |
| Effective batch size | $\approx 192$ transcripts/update | Update scale |

Additional implementation notes are explicit. Static anti-repetition gate thresholds and overlong/output-filtering are enabled to discard degenerate rollouts. The reward computation pipeline should exactly match the piecewise weights above. Group-relative normalization is critical to expose intra-group signal. The method also recommends tuning $K$ versus generation noise so that sampled completions are neither too similar, which would weaken signal, nor too diverse, which would increase hallucination risk. Within those conditions, DG-WGPO is described as sharpening ASR robustness under both moderate and severe noise by dynamically focusing learning where it is most needed.

Source: https://www.emergentmind.com/topics/dual-granularity-wer-gated-policy-optimization