AudioGAN: Adversarial Audio Generation

• AudioGAN is a framework where adversarial training models realistic audio using spectrogram and learned features rather than end-to-end waveform synthesis.
• It leverages diverse signal representations (e.g., log-magnitude STFT, mel-spectrograms) and conditioning regimes to address tasks like speech enhancement and text-to-audio generation.
• It employs auxiliary losses and specialized architectural motifs (e.g., CasNet, Style Blocks, SDT Attention) to enhance stability, controllability, and efficiency.

AudioGAN denotes both a broad class of generative adversarial network methods for audio and, more specifically, a 2025 text-to-audio framework titled "AudioGAN" (Chung, 17 Dec 2025). Across the literature, the term covers adversarial audio enhancement, unconditional speech synthesis, emotional text-to-speech, controllable musical synthesis, and unsupervised machine-audio anomaly detection (Abdulatif et al., 2019, Baas et al., 2022, Hortal et al., 2021, Nistal et al., 2021, Jiang et al., 2023). The unifying principle is that adversarial training is used to model the distribution of realistic audio, spectrograms, or audio features more effectively than pointwise regression alone, often with additional conditioning, reconstruction, or contrastive objectives.

1. Scope of the term

In current usage, AudioGAN is not restricted to free-form waveform generation. It includes methods that synthesize audio features and then rely on a vocoder, methods that denoise corrupted speech, and methods that use adversarial learning to support downstream analysis such as anomaly detection (Abdulatif et al., 2019, Jiang et al., 2023, Chung, 17 Dec 2025). This suggests that AudioGAN is best understood as a task family defined by adversarial learning over audio representations rather than by a single canonical architecture.

System	Task	Primary representation
AeGAN	speech denoising	log-magnitude STFT TF magnitude
ASGAN	unconditional speech synthesis	log mel-spectrograms or HuBERT features
GANtron	emotional speech synthesis	text-to-Mel spectrogram
DarkGAN	musical audio synthesis	STFT magnitude and instantaneous frequency
AEGAN-AD	machine-audio anomaly detection	mel-spectrogram segments
AudioGAN	text-to-audio generation	text-conditioned mel-spectrogram

The breadth of this scope is visible in the contrast between AeGAN, which frames denoising as a source-separation problem under harsh SNR conditions and speech-like interference (Abdulatif et al., 2019), and AudioGAN, which frames text-to-audio generation as a latency-constrained alternative to diffusion-based TTA (Chung, 17 Dec 2025). ASGAN and GANtron occupy intermediate positions: the former is unconditional speech synthesis from a latent prior, while the latter is a GAN-augmented Tacotron 2 system with emotional control (Baas et al., 2022, Hortal et al., 2021).

2. Signal representations and conditioning regimes

A central axis of variation in AudioGAN systems is the choice of signal representation. AeGAN operates in the 2D time-frequency magnitude domain using the log-magnitude STFT representation of noisy speech, reconstructing waveform with the noisy phase and LS-ISTFT (Abdulatif et al., 2019). Its STFT setup uses 16 kHz, a Hamming window, window length $S = 512$, 256 frequency bins, and a fixed TF image size of $256 \times 256$. To embed variable-duration utterances into that fixed-size representation, it introduces dynamic time resolution with

$$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$

with $N_T = 256$, allowing the longest directly handled track to be about 6.1 s (Abdulatif et al., 2019).

Other systems likewise prefer structured feature domains over direct waveform synthesis. ASGAN generates either log mel-spectrograms or HuBERT features, followed by a pretrained HiFi-GAN vocoder (Baas et al., 2022). GANtron is explicitly a text-to-mel-spectrogram model, with a modified Tacotron 2 generator and adversarial Mel-spectrogram discrimination (Hortal et al., 2021). AudioGAN also generates a text-conditioned mel-spectrogram and then uses a HiFi-GAN vocoder for waveform reconstruction (Chung, 17 Dec 2025). DarkGAN uses the magnitude and instantaneous frequency of the STFT, with FFT size 2048 and 75% overlap, for one-second musical notes sampled at 16 kHz (Nistal et al., 2021). AEGAN-AD uses log transform, affine scaling to $[-1,1]$, and overlapping $128 \times 128$ mel-spectrogram segments for machine-sound anomaly detection (Jiang et al., 2023).

Conditioning regimes are equally heterogeneous. ASGAN is unconditional speech synthesis, using $$\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$$ and a mapping network $\mathbf{w} = W(\mathbf{z})$ without text, speaker labels, or phonetic targets (Baas et al., 2022). GANtron conditions on style tokens containing random noise, emotion labels, speaker ID, or noise plus emotion labels (Hortal et al., 2021). DarkGAN conditions on pitch and teacher-derived soft semantic labels $\alpha \in [0,1]^{128}$ from a PANN CNN-14 trained on AudioSet (Nistal et al., 2021). AudioGAN conditions on word-level and sentence-level features from CLAP (Chung, 17 Dec 2025). AeGAN and AEGAN-AD use corrupted or nominal input audio as conditioning: noisy TF magnitude for denoising in the former, and normal machine spectrograms for reconstruction-based anomaly scoring in the latter (Abdulatif et al., 2019, Jiang et al., 2023).

A common misconception is that AudioGAN methods are primarily end-to-end waveform GANs. The literature surveyed here points in the opposite direction: many high-performing systems synthesize in spectrogram or learned-feature space and depend on explicit reconstruction pipelines or pretrained vocoders (Baas et al., 2022, Hortal et al., 2021, Chung, 17 Dec 2025).

3. Adversarial objectives, auxiliary losses, and stabilization

Although all AudioGAN systems use adversarial learning, the effective objective is almost never adversarial loss alone. AeGAN combines a standard conditional GAN objective with pixel-wise $L_1$ loss and a discriminator-based feature/perceptual loss. The generator maps noisy TF magnitude $256 \times 256$0 to enhanced magnitude $256 \times 256$1, and the feature-based term compares discriminator activations $256 \times 256$2 and $256 \times 256$3 to preserve higher-level spectral structure and phonetic detail (Abdulatif et al., 2019).

ASGAN uses the non-saturating logistic loss with $256 \times 256$4 regularization, equalized learning rate, EMA of generator weights, gradient clipping at norm 10, and leaky ReLU with $256 \times 256$5 (Baas et al., 2022). Its most distinctive stabilization mechanism is probabilistic skipping of discriminator updates. The skip probability is initialized at $256 \times 256$6, adjusted every 16 generator steps or whenever the discriminator is updated, and increased or decreased according to whether the running average $256 \times 256$7 of real-sample correctness is above or below 0.6 (Baas et al., 2022). The same probability controls augmentations including Gaussian noise with $256 \times 256$8, random scaling by $256 \times 256$9, and random replacement of a subsequence of frames with frames from a real example (Baas et al., 2022).

GANtron employs Wasserstein GAN (WGAN) training for stability and defines

$$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$0

Its guided attention warm-up applies the attention penalty only during the first $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$1 steps to encourage near-diagonal alignment and then disables it so the model can refine alignments freely (Hortal et al., 2021). The authors report that WGAN was more stable than WGAN-GP in their setup and that the best generator/discriminator update ratio was to train the generator twice for every discriminator update (Hortal et al., 2021).

DarkGAN combines a Wasserstein GAN objective with auxiliary supervision for pitch and distilled semantic attributes. The distillation target is a binary cross-entropy over soft AudioSet probabilities, with temperature $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$2 used to expose more dark knowledge (Nistal et al., 2021). AEGAN-AD trains its discriminator with WGAN-GP and its generator with feature matching plus mean square error, explicitly arguing that pure reconstruction behaves like a denoiser and therefore deprecates anomaly sensitivity (Jiang et al., 2023).

The 2025 AudioGAN uses the hinge GAN loss and augments it with four contrastive losses: Global-to-Sentence (G2S), Local-to-Word (L2W), Fake-to-Real (F2R), and Oneself-Conditioned Contrastive (OCC) (Chung, 17 Dec 2025). The intended division of labor is explicit: G2S provides coarse semantic alignment, L2W aligns local audio structure with words, F2R guides generated audio toward real audio in feature space, and OCC preserves diversity (Chung, 17 Dec 2025). The ablations report that removing G2S hurts training stability the most, while removing OCC reduces diversity as reflected in lower IS (Chung, 17 Dec 2025).

4. Architectural motifs

Several architectural patterns recur across AudioGAN systems. AeGAN uses a CasNet generator inspired by MedGAN, stacking three U-blocks with encoder-decoder structure and skip connections, together with a patch discriminator whose $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$3 patch size is reported to work better than the standard pix2pix $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$4 patch for speech TF structure (Abdulatif et al., 2019). The design goal is progressive TF refinement with preservation of local and global spectral organization.

ASGAN transfers the StyleGAN family into speech synthesis. A sampled 512-dimensional Gaussian latent vector is mapped by a multilayer perceptron into a more linearly disentangled $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$5-space, passed through a Gaussian Fourier feature layer, and refined by stacked Style Blocks with modulated 1D convolutions and anti-aliasing (Baas et al., 2022). The generator organizes these blocks in groups of 5, 4, 3, and 2, with the last block in each group upsampling by $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$6 instead of $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$7 (Baas et al., 2022). A major contribution is the adaptation of StyleGAN3-style anti-aliasing: around nonlinearities the model applies upsampling, low-pass filtering, non-linearity, and downsampling, using windowed sinc filters with a width-9 Kaiser window and a generator cutoff schedule from $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$8 to $$O = S - \left\lceil \frac{L}{N_T} \right\rceil, \qquad L = N_T(S - O) + O,$$9 cycles/sample (Baas et al., 2022).

GANtron retains a seq2seq TTS backbone. Its generator is a modified Tacotron 2 with encoder, attention, decoder, and Post-Net, while the discriminator distinguishes real and generated Mel spectrograms under WGAN training (Hortal et al., 2021). Two discriminator families are evaluated: a convolutional discriminator built from 1D convolutional layers and a linear discriminator built from dense layers (Hortal et al., 2021). The style token is concatenated either to the input of the encoder or to the output of the encoder, and the paper reports that this placement matters substantially (Hortal et al., 2021).

DarkGAN follows the DrumGAN architecture conceptually. The generator takes Gaussian noise $N_T = 256$0, pitch one-hot vector $N_T = 256$1, and soft labels $N_T = 256$2, inserts the conditioning as a “column” in a 4D tensor, and passes it through convolutional and box upsampling blocks (Nistal et al., 2021). The discriminator mirrors this structure and outputs a Wasserstein real/fake score, pitch prediction, and AudioSet attribute predictions (Nistal et al., 2021).

AEGAN-AD uses an autoencoder as generator and a convolutional discriminator whose last layer is replaced by a depth-wise convolution so that discriminator embeddings become semantically useful during detection (Jiang et al., 2023). A notable design choice is Layer Normalization rather than BatchNorm to reduce the risk that batch statistics from one domain mask those of another under strong source-target shift (Jiang et al., 2023).

The 2025 AudioGAN is organized around a CLAP text encoder, a GAN generator/discriminator pair, and a HiFi-GAN vocoder (Chung, 17 Dec 2025). Its distinctive architectural elements are Single-Double-Triple (SDT) Attention, which combines sentence attention, word attention, and self-attention in different orders in generator and discriminator, and Time-Frequency Cross-Attention (TF-CA), which explicitly models interactions along both time and frequency axes instead of flattening audio into a single sequence (Chung, 17 Dec 2025).

5. Representative task domains and empirical behavior

In speech enhancement, AeGAN is evaluated on TIMIT clean speech and QUT-NOISE-TIMIT noise, with training noise from cafe, food court, and home kitchen mixed at 0, 5, and 10 dB SNR, producing 36,000 noisy-clean training examples from 462 speakers and 6 dialects (Abdulatif et al., 2019). Evaluation uses PESQ, CSIG, CBAK, COVL, STOI, and ASR WER from a pretrained Deep Speech model. On matched speech-like noise at 10 dB SNR, the reported WER is 29.7% for AeGAN, compared with 35.8% for SEGAN and 44.7% for FSEGAN (Abdulatif et al., 2019). On unseen city-street noise, the paper reports about a 20.1% relative WER improvement over SEGAN at 0 dB SNR (Abdulatif et al., 2019).

In unconditional speech synthesis, ASGAN is benchmarked on SC09, the Google Speech Commands digits subset, using a ResNeXT classifier with 98.1% classification accuracy for evaluation (Baas et al., 2022). The best-performing variant, ASGAN (HuBERT), reports IS 7.67, mIS 226.7, FID 0.14, AM 0.26, eMOS 1.99, and MOS 3.68 (Baas et al., 2022). Its reported generation speed is 816.27 ksamples/sec, compared with 0.83 ksamples/sec for DiffWave, 0.14 ksamples/sec for SaShiMi, and 0.47 ksamples/sec for SaShiMi + DiffWave (Baas et al., 2022). The paper explicitly positions this as evidence that GANs can be substantially faster than diffusion and autoregressive baselines while remaining competitive in sample quality.

GANtron evaluates four configurations ranging from a Vanilla GANtron trained on LJ Speech with noise-only conditioning to a Complete model trained on LJ Speech + VESUS with noise + emotion labels (Hortal et al., 2021). In the Labelled model, classification accuracy reaches 96% with the linear discriminator and 90% with the convolutional discriminator, but the convolutional discriminator is discarded because about 18.03% of generated files are erroneous due to decoding stopping at max steps (Hortal et al., 2021). The paper presents the strongest result as evidence that generated samples lie in the same distribution as the original training data when augmenting VESUS with 6,000 GANtron-generated samples (Hortal et al., 2021).

DarkGAN is trained on a subset of NSynth containing about 90k sounds, roughly balanced across pitch classes, using only the first second of each sample and a 90/10 train/validation split (Nistal et al., 2021). Evaluation uses Pitch Inception Score, Instrument Inception Score, Kernel Inception Distance, and Fréchet Audio Distance, along with attribute-correlation analyses. The paper reports that $N_T = 256$3 and $N_T = 256$4 provide the best overall balance, with $N_T = 256$5 achieving the best FAD among the reported models and the strongest overall attribute correlations (Nistal et al., 2021).

AEGAN-AD is evaluated on DCASE 2022 Challenge Task 2 with five machine types: bearing, fan, gearbox, slider, and ToyCar (Jiang et al., 2023). Using AUC and pAUC-related evaluation, it reports 76.03 for bearing, 65.83 for fan, 75.27 for gearbox, 74.06 for slider, 78.46 for ToyCar, and a harmonic mean of 73.66 (Jiang et al., 2023). The largest improvement is on bearing at +10.91%, while ToyCar is the only case where one baseline autoencoder slightly outperforms AEGAN-AD (Jiang et al., 2023).

The specific model titled AudioGAN is evaluated on AudioCaps with about 49,000 training clips and 964 test clips, using 10-second clips at 16 kHz, 64 mel channels, a 1024-sample window, and 160-sample hop (Chung, 17 Dec 2025). It reports FAD = 1.38, FD = 12.66, IS = 9.63, KL = 1.15, CLAP = 0.49, Inference GPU memory = 1.34 GB, and Parameters = 28M (Chung, 17 Dec 2025). The central claim is state-of-the-art performance on key metrics with roughly 90% fewer parameters, 20× faster inference, and audio generation in under one second (Chung, 17 Dec 2025).

6. Controllability, disentanglement, misconceptions, and limitations

A recurring ambition in AudioGAN research is not only realism but also controllability. In ASGAN, disentanglement is encouraged by the mapping $N_T = 256$6 and by layerwise linear modulation. The latent space is then used for voice conversion, speech editing, speech enhancement, speaker verification, and keyword classification, including zero-shot voice conversion and digit/content editing on unseen SC09 speakers (Baas et al., 2023). The inversion procedure initializes $N_T = 256$7 at the mean latent $N_T = 256$8, optimizes an $N_T = 256$9 loss, and uses style mixing so that earlier layers control coarse content while later layers control fine speaker traits (Baas et al., 2022). This suggests that in successful AudioGAN systems, controllability is often an emergent property of architecture and training geometry rather than of explicit task supervision.

GANtron approaches control directly through emotion labels, speaker ID, and stochastic style tokens (Hortal et al., 2021). DarkGAN seeks human-interpretable labels without manual annotation by distilling a teacher’s soft labels into the conditioning pathway (Nistal et al., 2021). AudioGAN addresses control at the text-audio interface by jointly modeling global semantics, local word-level correspondences, and time-frequency structure (Chung, 17 Dec 2025). AEGAN-AD is not a generator in the conventional creative sense, yet it extends adversarial audio modeling into anomaly localization, where bright pixels in a localization map indicate regions that are more anomalous in time-frequency space (Jiang et al., 2023).

The literature also corrects several persistent misconceptions. One is that GANs are no longer competitive for audio because diffusion models dominate the strongest recent benchmarks. ASGAN explicitly argues that earlier GANs were weak, not that GANs are inherently unsuitable for speech synthesis, and reports state-of-the-art results on SC09 together with major inference-speed advantages (Baas et al., 2022). AudioGAN makes the same argument for text-to-audio on AudioCaps, emphasizing single-pass generation rather than iterative denoising (Chung, 17 Dec 2025). Another misconception is that reconstruction error alone is a reliable anomaly signal. AEGAN-AD explicitly rejects this assumption for machine audio, arguing that reconstruction often behaves like denoising and reconstructs both normal and anomalous patterns too well (Jiang et al., 2023).

The limitations are equally consistent. ASGAN can only generate utterances of a pre-specified duration, struggles with full sentences, is evaluated mostly on SC09, and depends on a HiFi-GAN vocoder (Baas et al., 2022). GANtron does not use subjective MOS because its emphasis is controllable emotional generation rather than maximum naturalness, and Tacotron 2 training remains computationally heavy (Hortal et al., 2021). DarkGAN reports only moderate attribute control, with teacher dependence and imperfect correspondence between attribute correlation and human perception (Nistal et al., 2021). AEGAN-AD requires different scoring methods for different machine types and uses a localization procedure that remains heuristic (Jiang et al., 2023). AudioGAN has simple inference but a training recipe that is heavily engineered through multiple contrastive losses and specialized attention blocks, and its evaluation is confined to AudioCaps (Chung, 17 Dec 2025).

Taken together, these systems define AudioGAN as a technically diverse but conceptually coherent research area: adversarial audio modeling in which distribution-level realism, structured conditioning, and task-specific inductive bias are combined to address denoising, synthesis, controllable generation, and unsupervised detection.