Voices-in-the-Wild-2M for Robust ASR

• Voices-in-the-Wild-2M is a large-scale simulation corpus that systematically exposes ASR models to realistic compound acoustic degradations.
• It leverages an acoustic ontology of seven atomic effects and 54 compound scenarios to mimic challenging in-the-wild environments.
• The dataset is integrated into the Mega-ASR framework, serving as a curriculum resource for both supervised fine-tuning and reinforcement learning.

Voices-in-the-Wild-2M is a speech dataset introduced as the central data resource in the Mega-ASR framework for robust automatic speech recognition under adverse and compositional acoustic conditions. It is designed for “in-the-wild$^2$” training: not only isolated degradations, but realistic compositions of multiple acoustic effects that push systems into regimes where acoustic grounding weakens and omissions or hallucinations become common. The paper reports a generation scale of 2.4M synthesized clips, while Table 1 lists “Voices-in-the-wild-2M” with scale 2M; the dataset covers 7 classic acoustic phenomena and 54 physically plausible compound scenarios, and is used for both supervised fine-tuning and reinforcement-learning stages in Mega-ASR (Xie et al., 19 May 2026).

1. Position within robust ASR

Voices-in-the-Wild-2M is motivated by what the Mega-ASR paper terms the “acoustic robustness bottleneck”: conventional ASR models perform well on clean or mildly degraded audio, but fail sharply in adverse real-world settings, especially when distortions compound and the WER regime exceeds 30%. In that regime, failures shift from local substitution errors toward sentence-level omissions and hallucinations. The dataset is intended to expose models to precisely these failure modes, at scale, and under a scenario taxonomy that emphasizes realistic multi-effect co-occurrence rather than isolated corruption (Xie et al., 19 May 2026).

The paper defines “in-the-wild$^2$” as a setting that goes beyond conventional “in-the-wild” robustness. The emphasis is on environments that are simultaneously difficult along multiple acoustic axes and that require semantic reconstruction when acoustic reliability is low. Within Mega-ASR, Voices-in-the-Wild-2M functions as the backbone training corpus for Acoustic-to-Semantic Progressive Supervised Fine-Tuning (A2S-SFT) and Dual-Granularity WER-Gated Policy Optimization (DG-WGPO). This suggests that the dataset is not merely a source of noisy waveforms, but a curriculum-bearing resource whose difficulty distribution is aligned to staged optimization.

2. Scale, source corpora, and data organization

The paper reports two closely related scale statistics. Section 3.1 gives a generation scale of 2.4M synthesized clips, while Table 1 lists “Voices-in-the-wild-2M” with scale 2M. The same table reports an average WER of 18.42% for strong open baselines, and the details note that Qwen3-ASR averages approximately 35% on this dataset prior to training (Xie et al., 19 May 2026).

Clean speech is drawn from LibriSpeech (English), Common Voice (multilingual), WenetSpeech (Mandarin), and AISHELL-1 (Mandarin). Noise comes from MUSAN, DNS Challenge, ESC‑50, and UrbanSound8K, described as a pool of approximately 42K clips totaling 129 hours. Demographic diversity is inherited from the source corpora; the paper specifically notes speaker diversity from Common Voice and WenetSpeech/AISHELL. The associated evaluation set, Voices-in-the-Wild-Bench, contains 5,000 clips, of which 3,500 are synthetic and 1,500 are real recordings, in English and Mandarin (Xie et al., 19 May 2026).

The paper states that routers and downstream models are trained at 16 kHz, and simulation operates at spectrogram level. Exact bit depth and release file formats are not specified. Training uses Voices-in-the-Wild-2M for both SFT and RL stages, while evaluation is carried out on the separate Voices-in-the-Wild-Bench. Synthetic audio preserves the clean source transcripts, and the data are organized by atomic effect and compound scenario. In RL training, the per-sample schema includes solution and base_wer.

3. Acoustic ontology: seven atomic effects and fifty-four compound scenarios

The dataset’s defining structure is its explicit acoustic ontology. The paper introduces seven classic atomic acoustic effects and then composes them into 54 physically plausible compound scenarios through an anchor–modifier taxonomy. Anchor effects define acoustic geometry, whereas modifiers can co-occur with an anchor or with other modifiers. Anchors are not combined with each other; modifiers can be layered broadly. This design is intended to avoid implausible mixtures while preserving realistic compositionality (Xie et al., 19 May 2026).

Atomic effect	Definition used in the paper
Noise	Low SNR via additive environmental/device noise plus output loudness normalization
Far-field	Distance-induced reverberation and attenuation, implemented by reverberation → low-pass filtering → loudness reduction
Obstructed	Occlusion-induced spectral loss, via repeated low-pass filtering + reverberation + attenuation
Echo–reverb	Strong reflections and delayed echoes, combining room reverberation + high-pass filtering + echo delay + loudness control
Recording	Narrow-band, noisy re-recordings via resampling + noise + high-pass + low-pass filters + loudness normalization
Electronic distortion	Nonlinear device overload (drive/clipping) + low-pass filtering + loudness adjustment
Transmission dropout	Local temporal discontinuity via frame-level stutter/repetition/silence insertion + loudness normalization

The 54 compound scenarios are enumerated as 7 single-effect scenarios, 18 two-effect scenarios, 13 three-effect scenarios, and 16 higher-order scenarios. The decomposition is given explicitly: 12 anchor–modifier pairs plus 6 modifier–modifier pairs for two-effect cases; 9 anchor plus two modifiers plus 4 modifier-only triples for three-effect cases; and 12 anchor plus three modifiers, 1 four-modifier combination, and 3 anchor plus all four modifiers for higher-order cases. Example classes include “Church interior” (far-field + echo–reverb + noise), “Phone playback recorded by phone” (recording coloration + noise), “Vehicle comms” (far-field + transmission dropout + electronic distortion), and “Behind-the-door speech” (obstructed + noise + reverb). A plausible implication is that the dataset is structured to stress both acoustic invariance and long-range semantic reconstruction.

4. Simulation pipeline, severity calibration, and learnability filtering

Voices-in-the-Wild-2M is generated through a spectrogram-level, code-based simulation pipeline built from filtering, convolution, resampling, distortion, loudness normalization, and frame-level stutter. The paper states that the simulation was iteratively calibrated to real recordings and validated by supervised fine-tuning on Qwen3-ASR. Each atomic effect is represented as an ordered chain of primitive operators; compound scenarios are formed by merging atomic chains while preserving within-scene order and removing cross-scene duplicates except additive noise. This ordering constraint is presented as part of the physical plausibility criterion (Xie et al., 19 May 2026).

Difficulty is controlled by a single global severity variable $m \in [0,1]$ shared across all parameters in a sample. The paper probes four mappings from $x \sim U(0,1)$ and adopts the Linear distribution:

$m_{\text{linear}}(x)=x$

$m_{\text{sqrt-fwd}}(x)=\sqrt{x}$

$m_{\text{sqrt-bwd}}(x)=x^2$

$$m_{\text{gaussian-mid}}(x)=\mathrm{clip}\!\left(\Phi^{-1}(0.05+0.9x;\mu=0.5,\sigma),\,0,\,1\right)$$

For a parameter with range $[a,b]$, instantiation under severity $m$ is defined as

$^2$0

The paper also gives parameter ranges for each atomic effect. Examples include noise_db ∈ [−5, 10] dB for Noise; room_size ∈ [0.4, 0.6], damping ∈ [0.6, 0.8], and low-pass cutoff ∈ [3500, 4500] Hz for Far-field; drive_db ∈ [20, 60] for Electronic distortion; and stutter_prob ∈ [0.05, 0.3], frame_ms = 20 for Transmission dropout. To stabilize optimization, samples with WER above 70% are discarded. This learnability filtering is a core part of the dataset’s operational definition inside Mega-ASR.

5. Use in Mega-ASR training and optimization

Voices-in-the-Wild-2M is not treated as a passive repository; it is tightly coupled to Mega-ASR’s staged training strategy. In A2S-SFT, the dataset supports a three-phase curriculum: first, a WER-graded acoustic curriculum on encoder and aligner, progressively expanding from WER $^2$1 to $^2$2 to $^2$3; second, LLM-side fine-tuning on the full WER $^2$4 subset to activate semantic recovery under unreliable acoustics; and third, joint fine-tuning of encoder, aligner, and LLM for end-to-end alignment. The learning rates reported in Section 6.1 are $^2$5 for the audio encoder/adapter, $^2$6 for the LLM, and $^2$7 for the joint stage (Xie et al., 19 May 2026).

The same dataset also supplies medium- and high-WER, compound-condition inputs for DG-WGPO. The static reward is

$^2$8

combined with an anti-repetition gate

$^2$9

and

$m \in [0,1]$0

The dynamic component introduces token-level and sentence-level terms:

$m \in [0,1]$1

$m \in [0,1]$2

$m \in [0,1]$3

with WER-gated mirrored fusion at $m \in [0,1]$4:

$m \in [0,1]$5

The final objective is

$m \in [0,1]$6

with $m \in [0,1]$7, $m \in [0,1]$8, and $m \in [0,1]$9. In this formulation, Voices-in-the-Wild-2M supplies the regime diversity required for both progressive acoustic exposure and reward shaping matched to distinct error modes.

6. Empirical performance, release status, and scope limits

The dataset is tied to the empirical claims of Mega-ASR on both standard adverse-condition benchmarks and compositional scenario evaluation. On VOiCES R4-B-F, Mega-ASR achieves 45.69% versus 54.01% for Qwen3-ASR. On NOIZEUS Station 0 dB, the abstract reports 21.49% versus 29.34%, while the per-dataset table reports 21.10% versus 29.34%. Table 2 gives overall averages of 6.70% versus 7.93% across CHiME‑4, VOiCES, and NOIZEUS. On Voices-in-the-Wild-Bench mixed degradations, Mega-ASR records 2.73/4.57 versus Whisper-Large-v3 at 8.91/14.79, which the paper states as 65.8%/69.1% relative reduction. Across atomic scenarios, the paper reports consistent gains over open-source baselines, including Noise at 6.33/8.26 versus 7.51/9.52 and Echo at 8.62/12.59 versus 10.40/14.61 (Xie et al., 19 May 2026).

Voices-in-the-Wild-2M is publicly linked from the title page through a project page, a Hugging Face dataset page, and a GitHub benchmark repository:

• https://xzf-thu.github.io/Mega-ASR/
• https://huggingface.co/datasets/zhifeixie/Voices-in-the-Wild-2M
• https://github.com/xzf-thu/Voices-in-the-Wild-Bench

The license is not specified in the paper; the dataset card is identified as the appropriate place to check license and usage restrictions. The paper also notes several limitations: the core data are simulated rather than captured from full real environments at scale; language coverage is centered on English and Mandarin; parameter realism is physically motivated but abstracted rather than tied to explicit RT60, device IR libraries, or source–mic distance distributions; and samples with WER above 70% are filtered out, leaving ultra-hard regimes for future work. These constraints define the intended scope of the resource rather than invalidating its use: it is a large-scale, scenario-structured simulation corpus for training robustness to compound acoustic degradation.

A separate nomenclatural point is relevant for literature searches. The VoxCPM2 Technical Report discusses an unrelated “over 2 million hours of multilingual speech” corpus assembled for multilingual and controllable speech generation, but explicitly states that the report does not use the name “Voices-in-the-Wild-2M,” “ViW-2M,” or any variant. That corpus is therefore not the same named resource, even though both works involve multi-million-scale speech data (Zhou et al., 5 Jun 2026).