Vocos: Fourier Neural Vocoder

• Vocos is a Fourier-based neural vocoder that directly predicts complex STFT coefficients from acoustic features, avoiding traditional upsampling stages.
• It employs a ConvNeXt-inspired backbone with fixed frame resolution and shared feature topology to achieve robust joint magnitude–phase estimation and rapid inference.
• Its design has inspired derivatives in spiking vocoding, bandwidth extension, and low-bitrate speech codecs, emphasizing both high quality and efficiency.

Vocos is a neural vocoder in the time–frequency domain that directly generates Fourier spectral coefficients from acoustic features and reconstructs the waveform with inverse short-time Fourier transform (iSTFT), rather than predicting sample-domain waveforms through learned upsampling stacks (Siuzdak, 2023). In the original formulation, it is a generative adversarial model conditioned on mel-spectrograms and built from ConvNeXt-style blocks operating at fixed frame resolution; this design was introduced to narrow the quality gap between time-domain GAN vocoders and Fourier-based vocoders while substantially improving inference efficiency (Siuzdak, 2023). Subsequent work has treated Vocos both as a baseline and as a reusable architectural principle for low-energy spiking vocoding, bandwidth extension, low-bitrate codecs, and larger spectrogram-conditioned singing voice systems (Chen et al., 16 Sep 2025, Sharma, 7 Mar 2026, Yang et al., 19 Jan 2026, Yang et al., 3 Aug 2025).

1. Foundational architectural premise

Vocos was introduced as a Fourier-based neural vocoder that predicts complex short-time Fourier transform (STFT) coefficients directly and then reconstructs audio via iSTFT (Siuzdak, 2023). Its conditioning signal in the original speech-vocoding setting is a mel-spectrogram extracted at 24 kHz with $n\_fft = 1024$, hop $H = 256$, and 100 mel bins, and the model predicts the single-sideband complex spectrum for each frame (Siuzdak, 2023). Because the generator stays at the conditioning frame rate throughout the network, temporal resolution remains constant across depth and no transposed-convolution upsampling is required (Siuzdak, 2023).

This architectural choice distinguishes Vocos from several major vocoder families. Compared with autoregressive models such as WaveNet, WaveRNN, and LPCNet, it is non-autoregressive and therefore avoids sequential synthesis latency; compared with time-domain GAN vocoders such as MelGAN, HiFi-GAN, and BigVGAN, it avoids the large upsampling stacks that dominate compute and memory bandwidth; compared with diffusion or flow-matching vocoders, it requires a single forward pass rather than tens or hundreds of iterative denoising steps (Chen et al., 16 Sep 2025). The original motivation was that time-domain GAN systems inherit large upsampling factors, potential aliasing from learned upsamplers, and heavier final-stage computation, whereas Fourier-domain synthesis can exploit FFT/iFFT algorithms and the inductive bias of harmonic organization in time–frequency representations (Siuzdak, 2023).

The generator backbone adapts ConvNeXt blocks to 1D sequences. The blocks use depthwise convolution for temporal context, pointwise convolutions for expansion and projection, GELU activations, Layer Normalization, and residual connections, all at fixed hop-rate resolution (Siuzdak, 2023). This fixed-resolution design is central to later Vocos-derived systems as well: Spiking Vocos explicitly preserves the same macro-architecture, input/output parameterization, and block topology while replacing the ANN backbone with a spiking variant (Chen et al., 16 Sep 2025), and VoCodec adopts an “improved Vocos” encoder/decoder architecture for causal low-bitrate coding in the time–frequency domain (Yang et al., 19 Jan 2026).

A recurrent misconception in neural vocoding has been that Fourier-domain models are structurally faster but intrinsically inferior in perceptual quality because of phase recovery difficulties. Vocos was proposed specifically to challenge that assumption by eliminating a separate phase-recovery stage and learning complex coefficients directly (Siuzdak, 2023).

2. Spectral parameterization and waveform reconstruction

The defining technical mechanism of Vocos is its parameterization of complex STFT coefficients. If $h$ denotes the hidden representation projected to the output head, it is split into magnitude and phase parameters, $m$ and $p$, per frequency bin. Magnitude is produced as

$M = \exp(m),$

while phase is mapped onto the unit circle through

$x = \cos(p), \qquad y = \sin(p).$

The predicted complex coefficient is then

$\hat{X}(k,n) = M(k,n)\cdot (x(k,n) + j y(k,n)),$

which implies a wrapped phase $\phi = \operatorname{atan2}(y,x)\in(-\pi,\pi]$ by construction (Siuzdak, 2023).

This parameterization is significant because it avoids direct regression of absolute wrapped phase with bounded activations, a strategy that the original paper reports as degraded relative to the unit-circle formulation (Siuzdak, 2023). In the broader literature around Vocos, this point became central to subsequent analyses of joint magnitude–phase estimation. A later comparative study categorized Vocos as a “single-stream” model in which a shared backbone processes features for both magnitude and phase and only the final output heads are separated; in that study, this topology was contrasted with “dual-stream” APNet2-style models, and Vocos’s single-stream formulation was associated with robust magnitude–phase coupling on LibriTTS (Dai et al., 23 Sep 2025).

Reconstruction is performed by iSTFT with overlap-add. In the original formulation, if $w[\cdot]$ is the analysis window, $H = 256$0 the FFT size, and $H = 256$1 the hop size, Vocos generates $H = 256$2 for single-sideband bins and reconstructs the waveform through the usual overlap-add iSTFT pipeline (Siuzdak, 2023). Because waveform-rate expansion is delegated to iSTFT rather than to learned transposed-convolution layers, the “upsampling” step is computationally cheap and lossless in the signal-processing sense used by the model design (Chen et al., 16 Sep 2025).

This STFT-centric view of generation has enabled multiple reinterpretations of Vocos. In bandwidth extension, a Vocos-style network maps mel features to complex STFT coefficients at a fixed 48 kHz grid and then merges generated high-frequency content with the original low band through a Linkwitz–Riley-inspired frequency-domain refiner (Sharma, 7 Mar 2026). In low-bitrate speech coding, VoCodec computes STFT, forms

$H = 256$3

encodes the frame sequence, quantizes it with residual vector quantization, and decodes complex spectra that are again inverted through iSTFT (Yang et al., 19 Jan 2026). These systems preserve the core Vocos principle that spectral prediction plus iSTFT can replace explicit in-network waveform upsampling.

3. Training objectives, discriminators, and learned representations

The original Vocos is trained adversarially with Multi-Period Discriminator (MPD) and Multi-Resolution Discriminator (MRD) components, using hinge GAN losses, an $H = 256$4 mel reconstruction term, and discriminator feature matching (Siuzdak, 2023). The mel loss is

$H = 256$5

and the feature-matching loss averages $H = 256$6 distances between discriminator feature maps on real and generated audio (Siuzdak, 2023). The paper emphasizes that, unlike some other Fourier-domain systems, Vocos does not rely on multi-resolution STFT losses as an explicit reconstruction loss; MRD appears in the discriminator stack rather than as a separate spectral loss (Siuzdak, 2023).

Training in the original speech-vocoding setting used LibriTTS at 24 kHz, AdamW with learning rate $H = 256$7 and betas $H = 256$8, cosine decay, random crops of 16384 samples, batch size 16, and up to 2M iterations, split as 1M generator and 1M discriminator iterations (Siuzdak, 2023). Random gain augmentation was applied so that peaks fell between $H = 256$9 and $h$0 dBFS (Siuzdak, 2023).

Later systems preserved the Vocos backbone while altering objectives for new tasks. Spiking Vocos distills from an ANN Vocos teacher and supervises both intermediate features and final magnitude/phase spectra. Its distillation losses include a layer-wise feature alignment term,

$h$1

a log-magnitude loss,

$h$2

and phase-aware anti-wrapping losses for instantaneous phase, group delay, and phase time-difference, all combined as

$h$3

(Chen et al., 16 Sep 2025). VoCodec, by contrast, adds codebook and commitment losses for residual vector quantization and uses least-squares GAN losses rather than the hinge setup reported for original Vocos (Yang et al., 19 Jan 2026). Vocos-based bandwidth extension combines MRSTFT, mel $h$4, adversarial training with an MRD discriminator, and feature matching, though the paper explicitly notes that some MRSTFT term details are standard or inferred rather than fully spelled out in the source (Sharma, 7 Mar 2026).

The representation learned by Vocos has also been studied structurally. In the joint-estimation analysis of Vocos and APNet2, Vocos was treated as the canonical single-stream architecture with shared ConvNeXt v1 blocks and direct phase prediction, and the study argued that shared representations provide implicit information exchange between magnitude and phase throughout the network (Dai et al., 23 Sep 2025). This interpretation is notable because it reframes Vocos not merely as a fast Fourier-domain vocoder, but also as a topology for stable joint estimation in complex spectral spaces.

4. Empirical quality and efficiency

In the original LibriTTS 24 kHz evaluation, Vocos achieved UTMOS 3.734, ViSQOL 4.66, PESQ 3.700, V/UV F1 0.9582, and periodicity 0.101 (Siuzdak, 2023). Among the reported baselines, it achieved the best ViSQOL, PESQ, V/UV F1, and lowest periodicity, while BigVGAN slightly led on UTMOS with 3.749 (Siuzdak, 2023). Subjective listening yielded MOS 3.62 ± 0.15 and SMOS 4.55 ± 0.15, matching BigVGAN without a statistically significant difference according to a Wilcoxon test with $h$5 (Siuzdak, 2023).

The efficiency results are a central part of Vocos’s identity. On an NVIDIA A100 GPU and an AMD EPYC 7542 CPU, measured with batches of 16 one-second samples and no hardware-specific optimizations, Vocos reached 6696.52× real-time on GPU and 169.63× real-time on CPU, with 13.5M parameters (Siuzdak, 2023). In the same setup, HiFi-GAN achieved 495.54× on GPU and 5.84× on CPU, BigVGAN 98.61× on GPU and 0.40× on CPU, and iSTFTNet 1045.94× on GPU and 14.44× on CPU (Siuzdak, 2023). These numbers are the empirical basis for the claim that Vocos delivers an order-of-magnitude increase in speed over prevailing time-domain neural vocoding approaches (Siuzdak, 2023).

A compact comparison is useful because later papers repeatedly position their own methods relative to this efficiency–quality profile.

System	Role of Vocos	Selected reported result
Original Vocos (Siuzdak, 2023)	Fourier-based GAN vocoder	GPU 6696.52× real-time; CPU 169.63× real-time
Spiking Vocos (Chen et al., 16 Sep 2025)	SNN reimplementation of Vocos backbone	Final 4-step model uses 14.7% of ANN energy
BWE via Vocos (Sharma, 7 Mar 2026)	Backbone generator for bandwidth extension	RTF 0.0001 on A100, batch 32
VoCodec (Yang et al., 19 Jan 2026)	Vocos-based speech codec backbone	349.29M MACs/s and 30 ms latency

Later evaluations reinforced both the quality and the topology arguments around Vocos. In the 2025 study on joint magnitude–phase estimation, baseline Vocos on LibriTTS achieved PESQ 3.487, UTMOS 3.356, VISQOL 4.861, MCD 3.260, M-STFT 0.921, V/UV F1 0.945, periodicity RMSE 0.124, pitch RMSE 32.846, and 13.53M parameters (Dai et al., 23 Sep 2025). In the same study, a baseline APNet2* dual-stream system exhibited severe collapse on LibriTTS, with PESQ 2.556 and UTMOS 1.886, leading the authors to argue that Vocos’s single-stream design is comparatively robust on large, diverse datasets (Dai et al., 23 Sep 2025).

5. Major derivatives and application domains

Vocos has become a reusable design pattern rather than remaining only a standalone vocoder. Several later systems preserve its Fourier-domain synthesis pathway while changing the task definition or internal computation.

Spiking Vocos recasts the Vocos backbone as a spiking neural network for low-energy inference on computationally restricted edge devices (Chen et al., 16 Sep 2025). It preserves the mel-input, constant-resolution, iSTFT-based macro-architecture of Vocos, but replaces the artificial-neuron ConvNeXt backbone with a spiking variant using Parametric Leaky Integrate-and-Fire neurons inserted before pointwise convolutions (Chen et al., 16 Sep 2025). It adds three targeted mechanisms: a Spiking ConvNeXt design that moves dominant pointwise channel-mixing into sparse spike-driven accumulations, an amplitude shortcut path

$h$6

to mitigate the binary-spike information bottleneck, and self-architectural distillation from an ANN Vocos teacher (Chen et al., 16 Sep 2025). On LibriTTS test-clean, the final 4-step model reported UTMOS 3.74, PESQ 3.45, ViSQOL 4.65, V/UV F1 0.9558, periodicity 0.116, and an estimated energy of $h$7 pJ versus $h$8 pJ for ANN Vocos, corresponding to 14.7% of the ANN energy budget (Chen et al., 16 Sep 2025).

In audio bandwidth extension, a Vocos-style vocoder serves as the backbone generator that maps an 80-bin mel-spectrogram computed on a 48 kHz grid to complex STFT coefficients, followed by iSTFT and a frequency-domain refiner that blends the generated high band with the original low band around the input Nyquist (Sharma, 7 Mar 2026). The model uses 8 residual ConvNeXt-style blocks with channel width 512, depthwise separable temporal convolution with kernel $h$9, and feed-forward expansion $m$0 (Sharma, 7 Mar 2026). On VCTK, the proposed system reported LSD 0.85/0.80/0.74 for 8/12/16→48 kHz and ViSQOL 3.51/3.53/3.69, with RTF 0.0053 on an 8-core CPU and 0.0001 on an A100 at batch 32 (Sharma, 7 Mar 2026).

In speech coding, VoCodec uses Vocos as the vocoder backbone in both encoder and decoder, again operating entirely in the time–frequency domain (Yang et al., 19 Jan 2026). It computes STFT with a 30 ms window, 10 ms hop, and FFT size 720, concatenates log-magnitude and phase, projects the result to 192 channels, encodes the frame sequence with ConvNeXt blocks and a lightweight attention module, applies 6-layer residual vector quantization, and decodes complex spectra through a Vocos-style decoder and iSTFT (Yang et al., 19 Jan 2026). The reported system has 3.47M parameters, 349.29M MACs/s, receiver-side compute 144.82M MACs/s, 30 ms latency, and ranked fourth on Track 1 of the 2025 LRAC Challenge while achieving the highest subjective MUSHRA score on the clean speech test set (Yang et al., 19 Jan 2026).

Vocos has also been adapted to larger spectrogram representations. In singing voice synthesis, a redesigned “Vocos2D” follows a diffusion-based linear spectrogram estimation stage and replaces the original 1D mel-conditioned generator with a 2D ConvNeXt-style vocoder specialized for large linear spectrograms such as $m$1 bins, using per-frequency learned embeddings, 2D convolutions, and persistent shortcut injection of the input condition into every block (Yang et al., 3 Aug 2025). This was motivated by the observation that the original Vocos generator “sees” the conditioning spectrogram once and must propagate it through hidden states, which becomes brittle when the conditioning spectrogram has many more frequency bins (Yang et al., 3 Aug 2025).

6. Comparative analyses, modifications, and open questions

Subsequent literature has treated Vocos as both a reference model and a source of modifiable components. One line of work studied whether magnitude and phase should be estimated in a shared stream or in separate streams. In that comparison, Vocos represented the shared or “single-stream” paradigm, while APNet2 represented the dual-stream paradigm (Dai et al., 23 Sep 2025). On LibriTTS, the paper reported that APNet2’s dual-stream baseline collapsed severely, and then proposed three stabilization strategies—topological sharing or shuffling between branches, a pseudo-inverse mel-spectrogram prior in the source space, and MI-RI output coupling in the output space—to narrow the gap with Vocos (Dai et al., 23 Sep 2025). A plausible implication is that Vocos’s robustness is not only a matter of Fourier-domain synthesis, but also of shared feature topology.

A separate line of work addressed model size in frequency-domain GAN vocoders. FreeV compared itself against Vocos and APNet2 and introduced a pseudo-inverse mel filterbank prior for amplitude estimation,

$m$2

turning amplitude prediction into a residual refinement problem (Lv et al., 2024). In that study’s reported setup, Vocos had 13.5M parameters and GPU RTF 0.0006, APNet2 had 31.4M and 0.0011, and FreeV had 18.2M and 0.0006, while the authors argued that the pseudo-inverse trick also improved early-stage convergence when inserted into Vocos-like generators (Lv et al., 2024). This does not alter the core Vocos design directly, but it shows that Vocos is compatible with source priors that partially offload spectral estimation from the neural backbone.

The broader ConvNeXt-vocoder literature has also positioned Vocos as a precursor to unified frameworks. WaveNeXt 2 describes Vocos as a ConvNeXt-based vocoder that predicts high-resolution STFT spectra from mel conditioning and reconstructs via iSTFT, then argues that such ConvNeXt generators had primarily been used in GAN frameworks and showed limited performance in multi-speaker settings unless carefully engineered (Zhou et al., 25 May 2026). WaveNeXt 2 responds by moving from direct spectrum prediction to residual denoising and sub-modeling across GAN and diffusion variants (Zhou et al., 25 May 2026). This suggests an ongoing diversification of the Vocos design ethos: the original efficiency-driven ConvNeXt-plus-spectral-reconstruction template remains influential, but later work increasingly reinterprets what the generator should predict.

Several limitations recur across the ecosystem. Original Vocos depends on consistent STFT configuration between training and inference; mismatches in sample rate, FFT size, hop, or window can degrade quality (Siuzdak, 2023). Spiking Vocos notes that binary spiking can introduce small degradations in signal-level metrics such as PESQ because of quantization-like effects, requiring amplitude shortcuts, temporal shift modules, and distillation to compensate (Chen et al., 16 Sep 2025). Bandwidth-extension work notes that the model hallucinates missing high-band content and can produce overly bright textures or faint harmonic residues on some inputs, although the smooth crossover refiner mitigates abrupt spectral seams (Sharma, 7 Mar 2026). Singing-voice work points to the compute overhead of diffusion-based spectrogram estimation and the increased memory cost of 2D processing over large frequency grids (Yang et al., 3 Aug 2025).

Taken together, these developments frame Vocos as more than a single 2023 vocoder architecture. It is a family of design commitments: frame-rate processing, explicit complex spectral prediction, iSTFT-based synthesis, ConvNeXt-derived backbones, and a bias toward replacing learned waveform upsampling with signal-processing structure (Siuzdak, 2023). Later research has preserved those commitments while stretching them toward energy-efficient spiking computation, causal speech coding, arbitrary-rate bandwidth extension, and large-bandwidth singing synthesis (Chen et al., 16 Sep 2025, Yang et al., 19 Jan 2026, Sharma, 7 Mar 2026, Yang et al., 3 Aug 2025).