Speech Restoration Challenge

Updated 12 July 2026

Speech Restoration Challenge is a benchmark that tests models to recover high-quality speech from multiple, interacting distortions including acoustic, codec, and enhancement artifacts.
It employs a composite degradation model with three stages—acoustic corruption, codec-induced artifacts, and secondary enhancement distortions—to simulate real-world speech pipelines.
The challenge integrates objective metrics like PESQ, DNSMOS, and word accuracy with model complexity assessments, promoting robust and deployable restoration methods.

Searching arXiv for the challenge paper and closely related speech restoration work. The Speech Restoration Challenge denotes a class of benchmark tasks in which a model must recover high-quality, intelligible speech from signals corrupted by multiple interacting degradations rather than a single impairment in isolation. In the formulation introduced in “The CCF AATC 2025: Speech Restoration Challenge” (Zhang et al., 16 Sep 2025), the task explicitly targets composite distortions arising from acoustic conditions, codec processing, and secondary artifacts introduced by prior enhancement systems. This framing places the challenge within a broader shift from single-task speech enhancement toward universal or general speech restoration, as seen in systems such as VoiceFixer (Liu et al., 2021), SelfRemaster (Saeki et al., 2022), Miipher-2 (Karita et al., 7 May 2025), and Sidon (Nakata et al., 21 Sep 2025), all of which address restoration under mixed or open-world degradations with different architectural assumptions.

1. Definition and problem formulation

In the CCF AATC 2025 formulation, the degraded input $\hat{x}$ is produced by a three-stage degradation operator

$\hat{x} = D_3 \circ D_2 \circ D_1(x),$

where $x$ is clean speech and $D_1$ , $D_2$ , and $D_3$ correspond to acoustic, codec, and secondary processing degradations, respectively (Zhang et al., 16 Sep 2025). The stated motivation is that real-world speech pipelines often incur multiple, interacting distortions: a single utterance may be recorded in reverberant, noisy environments, then passed through low-bitrate codecs, and finally further “enhanced” by imperfect onboard algorithms, compounding artifacts (Zhang et al., 16 Sep 2025).

This definition departs from traditional enhancement settings that isolate denoising, dereverberation, declipping, or codec artifact removal. The challenge description explicitly contrasts this composite task with prior single-degradation challenges such as DNS Challenge for denoising and REVERB Challenge or ICASSP dereverberation tasks, arguing that real signals rarely suffer only one degradation type (Zhang et al., 16 Sep 2025). A plausible implication is that the task is intended to measure robustness under non-linear cascades of distortions rather than performance on any one canonical corruption.

The broader literature uses related terminology. VoiceFixer defines “general speech restoration” as learning a single restoration function for combinations of additive noise, reverberation, low-resolution distortion, and clipping (Liu et al., 2021). SelfRemaster frames restoration as inversion of an explicit recording process without paired clean–distorted corpora (Saeki et al., 2022). More recent work such as Miipher-2 emphasizes conditioning-free multilingual restoration at million-hour scale (Karita et al., 7 May 2025), while Sidon treats restoration as multilingual dataset cleansing with a two-stage parametric resynthesis pipeline (Nakata et al., 21 Sep 2025). These systems suggest that the Speech Restoration Challenge sits at the intersection of robustness, generative modeling, and data-centric speech pipeline design.

2. Composite degradation model

The challenge partitions degradation into three categories. The first, complex acoustic degradations $D_1$ , includes Room Impulse Responses drawn from a library to simulate reverberation, microphone saturation simulated via hard clipping, low-pass filtering to mimic limited system bandwidth, and stationary and non-stationary noise such as babble, traffic, and white noise added at multiple SNR levels (Zhang et al., 16 Sep 2025). The second, signal-chain artifacts $D_2$ , consists of MP3 compression and decompression using FFmpeg at varying bitrates such as 16 kbps–128 kbps, introducing spectral “pre-echo” and “burbly” noise typical of low-bitrate lossy codecs (Zhang et al., 16 Sep 2025). The third, secondary artifacts $D_3$ , passes already degraded signals through ten off-the-shelf enhancement models to simulate residual processing artifacts (Zhang et al., 16 Sep 2025).

The ten secondary processors are divided into discriminative baselines and generative baselines. The discriminative baselines are Demucs, FRCRN, VoiceFixer, NSNet2, and TF-GridNet. The generative baselines are Sgmse+, Storm, AnyEnhance, MaskSR, and LLASE-G1 (Zhang et al., 16 Sep 2025). Because these models are used as degradation generators rather than only as comparison points, the challenge treats enhancement artifacts themselves as part of the corruption process. This is a notable design choice: many restoration benchmarks assume a clean target and a physically degraded input, whereas here model-induced artifacts are explicitly embedded in the data-generation chain.

Several related works illuminate why this matters. MaskSR2 shows that generative full-band restoration can attain high quality but still benefit substantially from explicit semantic guidance to improve intelligibility, with WER reduced from 4.74% to 3.18% when scaling from MaskSR-L to MaskSR2-L (Avg-feature) on the simulated VCTK full-band test (Liu et al., 2024). RestSE argues that lightweight SE modules can over-suppress speech, motivating a progressive SE-plus-codec design (Chiang et al., 2024). ReverbMiipher shows that conventional SR may completely remove reverberation, even though reverberation encodes spatial information regarding the acoustic source environment (Nakata et al., 8 May 2025). Together, these results suggest that “restoration” is not a single objective; preserving, removing, or regenerating signal attributes depends on task definition.

3. Dataset construction and multilingual scope

The clean speech sources for the challenge are VCTK, AISHELL-3, and EARS (Zhang et al., 16 Sep 2025). VCTK provides multi-speaker English speech with various accents, AISHELL-3 provides multi-speaker Mandarin Chinese, and EARS provides anechoic, fullband English recordings (Zhang et al., 16 Sep 2025). Each clean utterance is processed through $D_1 \rightarrow D_2 \rightarrow D_3$ to produce a degraded counterpart (Zhang et al., 16 Sep 2025).

The dataset creation pipeline specifies randomized perturbations at each stage. Reverberation is applied by convolving the clean signal with a randomly selected RIR. Clipping thresholds are varied to simulate different microphone headrooms. The low-pass filter cutoff is randomly varied in a predetermined band such as 3–7 kHz. Noise SNR levels are sampled uniformly between 0 dB and 20 dB using multiple noise types. MP3 bitrates are uniformly sampled, for example 16, 32, 64, and 128 kbps. Each utterance is then processed by one of the ten enhancement systems used for secondary processing (Zhang et al., 16 Sep 2025).

While the paper does not enumerate exact counts, it states that the training set comprises tens of thousands of utterances drawn from all three corpora, ensuring diversity in speakers, languages, recording conditions, and degradation permutations; a dev set of several thousand utterances is held out for leaderboard scoring; and a blind test set of comparable size and composition is reserved for final evaluation (Zhang et al., 16 Sep 2025). This bilingual English–Mandarin construction is narrower than large multilingual restoration systems, but it still extends beyond monolingual benchmark design.

A comparison with recent restoration datasets clarifies the design space. Miipher-2 is trained on 3,195 h of studio-quality speech spanning 44 languages and 1,642 speakers, with synthetic degradation formed by reverberation, additive noise, and optional codec artifacts (Karita et al., 7 May 2025). Sidon trains on approximately 2,219 h covering 104 languages, with a noising pipeline that applies reverberation, background noise, bandlimiting, clipping, MP3 codec, and packet loss in random order, yielding approximately 9,000 h of paired data (Nakata et al., 21 Sep 2025). By contrast, the Speech Restoration Challenge emphasizes a more tightly specified but operationally realistic cascade, especially through its use of secondary enhancement artifacts (Zhang et al., 16 Sep 2025).

4. Evaluation protocol and ranking procedure

The evaluation protocol is divided into a preliminary round and a final round (Zhang et al., 16 Sep 2025). In the preliminary round, objective metrics account for 40 points total and are equally weighted: Word Accuracy for intelligibility, measured via an automatic speech recognizer; DNSMOS for non-intrusive perceptual quality estimation; and PESQ (ITU-T P.862.2) for waveform fidelity (Zhang et al., 16 Sep 2025). Model complexity contributes 20 additional points via parameter-count tiers: fewer than 10 M parameters receives 20 points, 10–20 M receives 16 points, 20–50 M receives 12 points, 50–100 M receives 10 points, and more than 100 M receives 8 points (Zhang et al., 16 Sep 2025). Teams are ranked on the composite objective score, with a maximum of 60 points, and assigned to four performance tiers (Zhang et al., 16 Sep 2025).

In the final round, the top six teams from preliminaries advance (Zhang et al., 16 Sep 2025). The objective score carries over, and a 40-point subjective score is added. This subjective component includes Solution Innovation worth 20 points, judged by novelty and algorithmic contributions, and a Formal Listening Test worth 20 points based on MOS tests with four performance tiers (Zhang et al., 16 Sep 2025). A technical presentation and defense before an expert panel finalizes the rankings (Zhang et al., 16 Sep 2025).

The challenge therefore combines signal quality, intelligibility, efficiency, and expert assessment rather than reducing ranking to a single metric. This differs from challenges that rely only on ASR performance or only on subjective listening. For example, the Helsinki Speech Challenge 2024 uses Character Error Rate from Mozilla DeepSpeech v0.9.3 as its official quantitative metric, although PESQ and STOI are recommended as supplementary metrics (Ludvigsen et al., 2024). KS-Net, which won the ICASSP 2024 SSI Challenge, reports ITU-T P.804 MOS and Word Accuracy Rate as headline evaluation measures (Yu et al., 2024). Gesper, which ranked first in both tracks of the ICASSP 2023 SSI Challenge, was evaluated with ITU-T P.835, P.804, DNSMOS, and NISQA (Liu et al., 2023). The Speech Restoration Challenge inherits this multi-axis evaluation logic but adds an explicit model-complexity score (Zhang et al., 16 Sep 2025).

5. Baselines and system paradigms

The official baseline is AnyEnhance V1, described as a non-causal masked generative model with two stages: semantic enhancement, which restores high-level content, and acoustic enhancement, which refines waveform details (Zhang et al., 16 Sep 2025). It is pre-trained on the full training pipeline, and its code and models are released through the cited repository (Zhang et al., 16 Sep 2025). On the validation set, the paper reports illustrative baseline figures of PESQ around 2.6, DNSMOS around 3.4, WAcc around 80%, and a parameter count of 15 M, corresponding to a complexity score of 16/20 (Zhang et al., 16 Sep 2025). The listed strengths are joint handling of complex distortions and competitive PESQ and DNSMOS, while weaknesses are that the non-causal design limits real-time deployment and that word accuracy can still lag on severe degradations (Zhang et al., 16 Sep 2025).

The challenge’s degradation generators and the wider literature expose several distinct architectural paradigms:

Paradigm	Representative systems	Defining property
Analysis–synthesis / parametric resynthesis	VoiceFixer, Miipher-2, Sidon, ReverbMiipher	Predict clean intermediate features, then vocode
Generative token or codec restoration	MaskSR2, RestSE	Restore via masked acoustic modeling or generative codec
Direct complex-domain restoration	Gesper, KS-Net	Restore complex STFT directly
Self-supervised or self-supervised-adapted restoration	SelfRemaster, Sidon	Use real degraded data or frozen SSL backbones

VoiceFixer separates an analysis stage predicting mel-spectrogram features from an overview stage implemented by a neural vocoder (Liu et al., 2021). Miipher-2 uses a frozen Universal Speech Model as a conditioning-free feature extractor, parallel adapters for feature cleaning, and a modified WaveFit vocoder, achieving a real-time factor of 0.0078 on a single TPU v4i (Karita et al., 7 May 2025). Sidon similarly adopts a two-stage “parametric resynthesis” flow with a w2v-BERT 2.0 feature predictor and a HiFi-GAN vocoder, and reports inference approximately 3,390 times faster than real time on a single NVIDIA H200 (Nakata et al., 21 Sep 2025). ReverbMiipher extends this paradigm by adding a dedicated ReverbEncoder and reverb-feature control (Nakata et al., 8 May 2025).

Alternative designs remain competitive. MaskSR2 keeps the same inference-time compute and model size as MaskSR while using semantic knowledge distillation during training to improve intelligibility (Liu et al., 2024). Gesper performs restoration then enhancement in the complex STFT domain and achieves 3.27 P.804 overall MOS and 3.35 P.835 overall MOS on the blind test set of the ICASSP 2023 SSI Challenge (Liu et al., 2023). KS-Net uses a complex-domain GAN followed by fine-grained multi-band fusion and achieves an overall P.804 MOS of 3.49 and a WAcc of 0.78 for the real-time track of the ICASSP 2024 SSI Challenge (Yu et al., 2024). This suggests that the Speech Restoration Challenge is not architecturally prescriptive; the task can be approached through vocoder-based resynthesis, token generation, complex spectral mapping, or hybrid cascades.

6. Research significance, limitations, and emerging directions

The challenge is presented as a step toward universal, “all-in-one” restoration systems that can generalize across noise, reverberation, codec artifacts, and residual processing distortions (Zhang et al., 16 Sep 2025). That emphasis aligns with a broader research trajectory. VoiceFixer argued early for general speech restoration rather than separate SSR systems (Liu et al., 2021). SelfRemaster addressed the domain-mismatch problem of synthetic paired training by using an analysis-by-synthesis framework trained with real degraded speech data in a self-supervised manner (Saeki et al., 2022). Miipher-2 and Sidon then reframed restoration as a scalable infrastructure problem for cleaning training corpora for ASR, TTS, and LLM-adjacent data pipelines (Karita et al., 7 May 2025, Nakata et al., 21 Sep 2025).

A recurring tension in this literature concerns fidelity versus intelligibility, and suppression versus regeneration. MaskSR2 explicitly demonstrates that generative restoration quality does not guarantee optimal intelligibility without semantic guidance (Liu et al., 2024). RestSE argues that in challenging environments traditional SE can create over-suppressed speech and harm downstream tasks, motivating progressive restoration through SE and codec modules (Chiang et al., 2024). ReverbMiipher introduces a further complication: removing reverberation can improve classical enhancement scores while destroying spatial characteristics that may be desirable to preserve (Nakata et al., 8 May 2025). This suggests that “better restoration” depends on whether the target is anechoic speech, perceptually improved speech, studio-quality speech, or speech that preserves some environmental attributes.

Another emerging direction is multilingual and cross-domain generalization. Sidon covers dozens of languages and is designed for large-scale dataset cleansing (Nakata et al., 21 Sep 2025). Miipher-2 supports over 300 languages through a frozen USM backbone (Karita et al., 7 May 2025). DisSR, a later cross-domain diffusion-based approach, introduces speaker-invariant degradation priors and hierarchical MMD alignment to improve adaptability on unseen speakers and languages (Liang et al., 13 Feb 2026). Although DisSR is not part of the challenge design itself, it suggests one route for systems attempting to generalize beyond the bilingual English–Mandarin data of the competition.

The official organization details underscore the challenge’s role as a community benchmark. The timeline covers dataset release and baseline code publication from February to May 2024, preliminary submission in June 2024, finalist announcement in July 2024, and final submissions, listening tests, and technical presentations in August 2024 (Zhang et al., 16 Sep 2025). Submissions consist of restored waveforms and model metadata such as parameter count and run-time specifications; automated scoring scripts and submission instructions are provided in the open-source toolkit; and leaderboard updates occur weekly during preliminaries (Zhang et al., 16 Sep 2025). In this sense, the Speech Restoration Challenge is both a benchmark specification and an organizational mechanism for consolidating work on composite, multi-stage speech degradation.

A plausible implication is that the challenge formalizes a transition already underway in speech restoration research: from narrowly defined enhancement tasks toward models evaluated simultaneously for intelligibility, perceptual quality, robustness to cascaded artifacts, and deployability. Whether future systems converge on universal parametric resynthesis, direct complex-domain mapping, diffusion-based conditional generation, or hybrid progressive pipelines remains unresolved, but the challenge makes that comparison experimentally tractable (Zhang et al., 16 Sep 2025).