WaveGrad Diffusion-Based Waveform Generation
- The paper introduces WaveGrad as a conditional denoising diffusion model that iteratively refines Gaussian noise into high-quality speech, with applications in neural vocoding.
- WaveGrad employs a fully convolutional neural network with FiLM-based up/downsampling to balance the speed–quality trade-off, achieving high MOS with few iterations.
- Extensions like WaveGrad 2 and VoiceGrad demonstrate its adaptability for end-to-end TTS and non-parallel voice conversion, broadening its diffusion-based audio generation paradigm.
Searching arXiv for WaveGrad and closely related follow-up work to ground the article in the cited literature. WaveGrad is a conditional model for waveform generation that estimates gradients of the data density and is built on prior work on score matching and diffusion probabilistic models (Chen et al., 2020). In its original formulation, the model starts from a Gaussian white noise signal and iteratively refines the signal via a gradient-based sampler conditioned on the mel-spectrogram, thereby functioning as a neural vocoder for speech synthesis (Chen et al., 2020). Subsequent work situates WaveGrad within a broader family of diffusion-based audio generators, examines its computational and perceptual trade-offs in standardized vocoder benchmarking, and extends or adapts its principles to end-to-end text-to-speech, voice conversion, inference-time correction, accelerated sampling, and biosignal generation (AlBadawy et al., 2021, Chen et al., 2021, Kameoka et al., 2020, Liu et al., 2024, Lam et al., 2022).
1. Origins and formal definition
WaveGrad was introduced by Chen et al. as a waveform generation method inspired by score-based generative models and diffusion probabilistic models (Chen et al., 2020). Rather than directly parameterizing a waveform density, it models the gradient of the log-density, i.e. the score function,
and uses denoising-based training together with iterative sampling to transform noise into speech (Chen et al., 2020).
The diffusion formulation used in the paper defines a forward Markov chain that progressively corrupts a clean waveform with Gaussian noise:
with the equivalent closed form
where (Chen et al., 2020). The learned reverse process removes noise step by step, conditioned on acoustic features.
In the original paper, WaveGrad is explicitly a conditional model for waveform generation given mel-spectrogram input , that is, it models (Chen et al., 2020). This conditionality distinguishes it from unconditional score-based samplers and defines its role as a vocoder in a text-to-speech pipeline.
A recurrent theme in later work is that WaveGrad occupies an intermediate position between autoregressive and non-autoregressive paradigms. The original paper reports that it bridges the gap between non-autoregressive and autoregressive models in terms of audio quality, while offering a tunable inference process whose number of refinement steps can be adjusted at test time (Chen et al., 2020). This speed–quality trade-off became one of the model’s defining characteristics in subsequent analyses and extensions.
2. Core methodology and architecture
WaveGrad trains a neural network to predict the noise component in a noisy waveform sample given the current noise level and the conditioning features (Chen et al., 2020). The training objective is presented in the diffusion-noise-prediction form and, in practice, uses an loss for stability:
(Chen et al., 2020). The same paper also situates this objective within denoising score matching and Langevin dynamics, with the latter expressed as
to emphasize its conceptual relation to score-based sampling (Chen et al., 2020).
Sampling starts from Gaussian white noise 0 and iteratively produces cleaner waveform estimates. The update equation given in the original paper is
1
followed, when 2, by the addition of Gaussian noise 3 (Chen et al., 2020). This formulation makes explicit that generation is an iterative reverse diffusion process rather than a single-pass waveform decoder.
Architecturally, WaveGrad employs a fully convolutional neural network with upsampling and downsampling blocks and uses feature-wise linear modulation for conditioning on both mel-spectrogram features and noise level (Chen et al., 2020). In later biosignal-oriented adaptations, the architecture is described in terms of UBlock and DBlock modules with FiLM-based conditioning, and this representation is consistent with the original architectural characterization (Marocchi et al., 15 Sep 2025, Abbott et al., 2024). Those later descriptions additionally note that local conditioning can be supplied by a mel-spectrogram and, in customized variants, global conditioning can be injected through labels such as disease class or channel identity (Marocchi et al., 15 Sep 2025, Abbott et al., 2024).
A separate but related formulation appears in work that interprets WaveGrad primarily through weighted denoising score matching over multiple noise levels. VoiceGrad, which is explicitly “inspired by WaveGrad,” describes the score approximator as a fully convolutional network with a U-Net structure and frames sampling in terms of annealed Langevin dynamics over a sequence of noise scales (Kameoka et al., 2020). This suggests that the WaveGrad family can be read from two compatible viewpoints: as a DDPM-style noise-prediction model and as a score-based model with iterative gradient-guided sampling.
3. Inference behavior and the speed–quality trade-off
A central property of WaveGrad is that inference quality can be traded against speed by varying the number of refinement steps at test time (Chen et al., 2020). The original paper states that it can generate high fidelity audio samples using as few as six iterations (Chen et al., 2020). In the same study, the reported MOS is approximately 4 for 6 steps and approximately 5 for 1000 steps, indicating little degradation in perceptual quality in the low-step regime under the tested setup (Chen et al., 2020).
This property is important because diffusion-based vocoders are otherwise associated with slow sequential sampling. Later work repeatedly treats WaveGrad as a strong but computationally expensive baseline. In “InferGrad: Improving Diffusion Models for Vocoder by Considering Inference in Training” (Chen et al., 2022), WaveGrad is described as using a U-Net architecture with 15M parameters, trained with a fixed diffusion schedule 6 and 7 linearly from 8 to 9, while its fast inference behavior depends on inference-schedule optimization performed after training. That paper emphasizes a training–inference mismatch: WaveGrad does not consider inference schedules during training, and schedule optimization for a small number of reverse steps occurs only at test time (Chen et al., 2022).
The same study quantifies the limitations of this setup at very low step counts. The reported MOS values for WaveGrad are 0 at 1, 2 at 3, and 4 at 5, whereas InferGrad exceeds these values at the same step counts and matches or exceeds WaveGrad at 6 steps using only 2 steps (Chen et al., 2022). The paper’s conclusion is precise: “InferGrad achieves better voice quality than the baseline WaveGrad under same conditions while maintaining the same voice quality as the baseline but with 6 speedup (7 iterations for InferGrad vs 8 iterations for WaveGrad)” (Chen et al., 2022).
Acceleration work beyond InferGrad reinforces the same point. BDDM reports WaveGrad at 1000 steps with MOS 9 and Real-Time Factor 0, and states that BDDMs can produce comparable or higher quality samples with only seven sampling steps, “143x faster than WaveGrad and 28.6x faster than DiffWave” (Lam et al., 2022). In this literature, WaveGrad functions as the reference model illustrating the standard quality–latency trade-off of early diffusion vocoders.
4. Benchmarking as a neural vocoder
A standardized view of WaveGrad as a vocoder is provided by VocBench, a framework that benchmarks state-of-the-art neural vocoders in a shared environment using the same setup for datasets, training pipeline, and evaluation metrics (AlBadawy et al., 2021). In that benchmark, WaveGrad is implemented following its original architecture, with a noise scheduler transforming white Gaussian noise into speech waveforms conditioned on 80-dimensional mel-spectrograms extracted with a 40ms Hanning window, 12.5ms frame shift, 1024-point FFT, and 0–12kHz range (AlBadawy et al., 2021).
VocBench trains and evaluates WaveGrad on LJ Speech, LibriTTS, and VCTK, using hyperparameter search over each dataset, training on a Tesla V100 GPU, and evaluation on both GPU and CPU (AlBadawy et al., 2021). Inference uses 50 denoising steps, “in line with the original WaveGrad release” (AlBadawy et al., 2021).
The benchmark reports both objective and subjective metrics: SSIM, LS-MSE, PSNR, FAD, and MOS (AlBadawy et al., 2021). The recorded WaveGrad scores are as follows.
| Dataset | SSIM | LS-MSE | PSNR | FAD | MOS |
|---|---|---|---|---|---|
| LJ Speech | 0.76 | 0.006 | 22.57 | 3.12 | 3.85±0.07 |
| LibriTTS | 0.71 | 0.005 | 22.94 | 3.10 | 3.48±0.08 |
| VCTK | 0.59 | 0.007 | 21.54 | 4.10 | 3.77±0.07 |
VocBench interprets these results conservatively. It states that WaveGrad is “competitive but not best-in-class,” that it is outperformed by GANs such as MelGAN and Parallel WaveGAN on SSIM, LS-MSE, and PSNR, and that FAD is higher, hence worse, than for GANs and WaveNet on all tested datasets (AlBadawy et al., 2021). On subjective MOS, the benchmark records 1 on LJ Speech, 2 on LibriTTS, and 3 on VCTK (AlBadawy et al., 2021).
The same benchmark also highlights the computational profile of WaveGrad. It reports 15.81M parameters, 33.75 GFLOPS per step, GPU RTF 4, and CPU RTF 5, with the explicit note that “WaveGrad inference uses 50 denoising steps (so total computational effort is much higher)” (AlBadawy et al., 2021). VocBench therefore presents WaveGrad as a model with robust but not state-of-the-art quality under the tested configuration and with substantially higher computational cost than GAN-based vocoders.
5. Extensions for speech synthesis and voice conversion
WaveGrad 2
WaveGrad 2 generalizes the original vocoder into a non-autoregressive end-to-end text-to-speech model trained to estimate the gradient of the log conditional density of the waveform given a phoneme sequence (Chen et al., 2021). The principal distinction is architectural and representational: WaveGrad 2 removes the intermediate mel-spectrogram representation and directly synthesizes waveforms from phoneme sequences (Chen et al., 2021).
The model comprises an encoder for phoneme sequences, a resampling layer that matches encoder outputs to waveform time resolution using duration information, and a WaveGrad decoder that iteratively refines Gaussian noise into speech (Chen et al., 2021). Its loss retains the diffusion-noise-prediction form,
6
and the iterative update similarly follows a denoising reverse process (Chen et al., 2021).
WaveGrad 2 explicitly preserves the original model’s speed–quality trade-off. The paper reports MOS 7 with 1000 steps and 8 with 50 steps, and states that its best configuration reaches MOS 9, close to Tacotron2 + WaveRNN at 0 and Tacotron2 + WaveGrad(L) at 1 (Chen et al., 2021). Thus WaveGrad 2 is not merely a vocoder replacement; it reinterprets WaveGrad’s diffusion refinement process as the decoder of a full text-to-waveform system.
VoiceGrad
VoiceGrad applies WaveGrad’s score-based and Langevin-dynamics foundations to non-parallel any-to-many voice conversion (Kameoka et al., 2020). It trains a fully convolutional U-Net score approximator to predict the gradient of the log density of speech feature sequences of multiple speakers and performs conversion using annealed Langevin dynamics (Kameoka et al., 2020). Rather than starting from random noise, it starts from a source speaker’s mel-spectrogram and iteratively updates it toward the target speaker distribution (Kameoka et al., 2020).
The paper presents this as a direct methodological adaptation of WaveGrad’s principles—weighted denoising score matching, multi-noise-level conditioning, U-Net score modeling, and iterative score-guided sampling—to the feature conversion setting (Kameoka et al., 2020). The VoiceGrad training objective is written as
2
with conditioning on a target speaker index 3 (Kameoka et al., 2020). This suggests that WaveGrad’s core machinery is portable beyond waveform vocoding into conditional sequence transformation problems.
6. Successors, corrections, and acceleration strategies
A substantial body of later work identifies two recurring limitations of WaveGrad: computationally intensive sampling and imperfect adherence to mel-spectrogram conditioning under distribution shift.
InferGrad and BDDM
InferGrad addresses the first limitation by incorporating the inference process into training (Chen et al., 2022). The model adds an infer loss to the usual DDPM objective,
4
where 5 is a multi-resolution STFT loss combining magnitude and phase terms (Chen et al., 2022). The paper’s claim is specific: on LJSpeech, InferGrad achieves better voice quality than WaveGrad under the same conditions and maintains the same voice quality as the baseline with 6 speedup, namely 2 iterations for InferGrad versus 6 iterations for WaveGrad (Chen et al., 2022).
BDDM proposes a bilateral denoising diffusion model with a score network and a schedule network, allowing the score network to be inherited from any pretrained DPMs and learning a data-adaptive short noise schedule (Lam et al., 2022). In that comparison, WaveGrad is characterized as a classic DDPM-style vocoder with a single score network and fixed or manually optimized schedules (Lam et al., 2022). BDDM reports that with seven sampling steps it attains MOS 7, RTF 8, and is “143x faster than WaveGrad” while producing samples statistically indistinguishable from human speech (Lam et al., 2022). These claims are comparative rather than intrinsic to WaveGrad, but they have shaped how the model is positioned in the literature: as a high-quality baseline whose sampling procedure motivated more efficient descendants.
GLA-Grad and GLA-Grad++
GLA-Grad and GLA-Grad++ address the second limitation by introducing Griffin-Lim-based corrections into the reverse process (Liu et al., 2024, Baoueb et al., 27 Nov 2025). GLA-Grad proposes applying Griffin-Lim at each step of the regular diffusion process to minimize conditioning error and improve speech generation for previously unseen target speakers (Liu et al., 2024). It is explicitly a drop-in inference-time correction that can be applied to an already trained waveform generation model without additional training or fine-tuning (Liu et al., 2024).
On single-speaker LJ Speech, GLA-Grad does not surpass WaveGrad; the reported scores are PESQ 9, STOI 0, and WARP-Q 1, compared with WaveGrad at PESQ 2, STOI 3, and WARP-Q 4 (Liu et al., 2024). By contrast, under unseen-speaker conditions, GLA-Grad improves substantially over WaveGrad. For LJ5VCTK generalization, GLA-Grad achieves PESQ 6, STOI 7, WARP-Q 8, whereas WaveGrad obtains PESQ 9, STOI 0, WARP-Q 1 (Liu et al., 2024). For VCTK adaptation to new speakers, GLA-Grad reports PESQ 2, STOI 3, WARP-Q 4, versus WaveGrad at PESQ 5, STOI 6, WARP-Q 7 (Liu et al., 2024).
GLA-Grad++ modifies the correction strategy by computing the correction term only once, with a single application of GLA, and applying it only to the predicted clean-waveform term in early reverse steps (Baoueb et al., 27 Nov 2025). The modified update is written as
8
during the initial stage, after which sampling returns to standard DDPM/WaveGrad updates (Baoueb et al., 27 Nov 2025). On LJSpeech and VCTK, GLA-Grad++ reports higher PESQ and STOI than WaveGrad while keeping WARP-Q similar; for example, on LJSpeech it records PESQ 9, STOI 0, WARP-Q 1, versus WaveGrad at PESQ 2, STOI 3, WARP-Q 4 (Baoueb et al., 27 Nov 2025). Its inference speed, expressed as times real-time, is only slightly slower than vanilla WaveGrad and much faster than GLA-Grad (Baoueb et al., 27 Nov 2025).
A common interpretation across these papers is that WaveGrad’s mel conditioning can be imperfect under out-of-domain inputs or unseen-speaker conditions, and phase-aware correction offers one remedy (Liu et al., 2024, Baoueb et al., 27 Nov 2025). That interpretation is directly stated in the corresponding works.
7. Beyond speech vocoding: biosignal synthesis and robustness-oriented augmentation
WaveGrad has also been adapted for synthetic biosignal generation, particularly phonocardiogram augmentation. Two papers describe the use of WaveGrad and DiffWave to generate synthetic heart sounds conditioned on ECG-derived mel-spectrograms for downstream classification (Marocchi et al., 15 Sep 2025, Abbott et al., 2024).
In the 2025 multimodal and multichannel heart sound classification study, WaveGrad is described as a conditional denoising diffusion model for audio synthesis that employs upsampling and downsampling modules conditioned with mel-spectrograms of the reference ECG and disease labels, with FiLM modules adapting feature maps based on conditioning information (Marocchi et al., 15 Sep 2025). The forward process gradually adds noise to heart sound signals and the reverse process predicts and removes that noise to synthesize clean PCG signals. The paper gives the standard diffusion expressions
5
6
and
7
(Marocchi et al., 15 Sep 2025).
That study trains WaveGrad and DiffWave for 24 hours on an RTX 3090, generates 3,200 synthetic subjects with a 3:1 ratio of normal to abnormal labels, applies cardiac cycle rearrangement 25% of the time, and conditions on ECG mel-spectrograms from the Icentia database together with disease labels (Marocchi et al., 15 Sep 2025). The synthetic signals are then used to fine-tune a Wav2Vec 2.0 BASE classifier. Reported results with augmented and synthetic data reach 92.48% accuracy and 0.8283 MCC on single-channel CinC 2016 PCG, 93.14% accuracy and 0.8380 MCC on synchronized PCG+ECG training-a data, and 77.13% accuracy and 0.5082 MCC on a wearable vest multichannel PCG dataset (Marocchi et al., 15 Sep 2025).
A related 2024 study focuses explicitly on robustness rather than only performance and again uses conditional WaveGrad and DiffWave models to synthesize PCG signals from ECG (Abbott et al., 2024). In that implementation, WaveGrad uses multiple UBlock and DBlock modules with FiLM, 1D convolutions, an 8 noise-prediction loss, a linear noise schedule with 9 over 0, and a custom global conditioner to control specific abnormalities (Abbott et al., 2024). The paper reports improvements in both in-distribution and out-of-distribution classification metrics when training with augmented datasets, including an increase from 40.9% to 84.0% accuracy and from 0.192 to 0.489 MCC on one out-of-distribution PCG evaluation set (Abbott et al., 2024).
These biosignal applications do not alter the core identity of WaveGrad as a conditional denoising diffusion model. Rather, they demonstrate that its conditioning and iterative refinement framework can be transferred from speech waveforms to physiologically plausible synthetic audio generation when suitable paired conditioning signals exist (Marocchi et al., 15 Sep 2025, Abbott et al., 2024). A plausible implication is that the WaveGrad design is valued not only for speech naturalness but also for controllable conditional synthesis in low-resource scientific domains.
8. Position in the diffusion-audio literature
WaveGrad occupies a historically important position in diffusion-based audio generation. It is one of the early models to show that a score-based, diffusion-style vocoder can achieve high-fidelity speech synthesis while exposing an adjustable inference budget (Chen et al., 2020). In the original comparison, it outperforms adversarial non-autoregressive baselines and matches a strong likelihood-based autoregressive baseline using fewer sequential operations (Chen et al., 2020).
At the same time, later benchmark and systems papers consistently portray its limitations with equal clarity. VocBench finds that WaveGrad is competitive but not best-in-class, with quality below leading GANs and DiffWave on several benchmark metrics under a common evaluation protocol (AlBadawy et al., 2021). InferGrad and BDDM treat WaveGrad as a baseline whose core reverse diffusion is insufficiently optimized for few-step inference (Chen et al., 2022, Lam et al., 2022). GLA-Grad and GLA-Grad++ treat it as a baseline whose mel-spectrogram conditioning can become inconsistent under speaker mismatch or other out-of-domain conditions (Liu et al., 2024, Baoueb et al., 27 Nov 2025). WaveGrad 2, finally, transforms the same iterative refinement principle into an end-to-end TTS decoder conditioned directly on phonemes (Chen et al., 2021).
Several misconceptions are therefore clarified by the later literature. WaveGrad is not synonymous with all diffusion vocoders; it is a specific conditional waveform generator with a particular denoising architecture and inference procedure (Chen et al., 2020). Nor is it uniformly slow in the sense of requiring thousands of steps for acceptable speech; the original paper reports high fidelity with as few as six iterations, but later work shows that, relative to improved schedulers or training objectives, its low-step regime can still be substantially improved (Chen et al., 2020, Chen et al., 2022, Lam et al., 2022). Likewise, although the model is often associated with mel-spectrogram-conditioned vocoding, the same basic method has been adapted to phoneme-conditioned TTS, non-parallel voice conversion, and biomedical signal generation (Chen et al., 2021, Kameoka et al., 2020, Marocchi et al., 15 Sep 2025).
Taken together, the literature presents WaveGrad as both a specific vocoder and a methodological template. Its combination of score estimation, conditional denoising diffusion, FiLM-based modulation, and iterative refinement established a widely reused design pattern whose descendants and modifications address speed, robustness, and broader conditioning modalities while retaining the same generative core (Chen et al., 2020, Chen et al., 2021, Kameoka et al., 2020, Liu et al., 2024, Lam et al., 2022).