Adversarial Step Distillation

• Adversarial Step Distillation is a method that compresses iterative diffusion processes into one- or few-step models using a blend of adversarial losses and score or distribution matching.
• The approach improves synthesis quality across applications such as image, video, and voice conversion by addressing capacity mismatches and correcting errors like blur and artifacts.
• Empirical results from models like ADD and SDXL-Lightning demonstrate competitive performance in single-step generation while highlighting challenges in stability and high-fidelity reconstruction.

Adversarial Step Distillation is a family of distillation procedures that compress a pretrained multi-step diffusion sampler, or an analogous iterative generative process, into a one-step or few-step student by combining adversarial supervision with score, consistency, or distribution matching. In these methods, the student is trained under an extreme neural function evaluation budget, while an adversarial module penalizes the perceptual errors, distributional gaps, temporal drift, or conditioning failures that regression-only distillation tends to leave unresolved. The resulting paradigm now spans text-to-image synthesis, autoregressive and diffusion-based video generation, adversarial purification, and voice conversion, with representative formulations in ADD, SDXL-Lightning, OSCP, POSE, and AAD-1 (Sauer et al., 2023, Lin et al., 2024, Lei et al., 2024, Cheng et al., 28 Aug 2025, Li et al., 2 Jun 2026).

1. Conceptual scope and emergence

In the image-generation literature, adversarial step distillation first became prominent as a remedy for the low-step failure modes of pure distillation. ADD trains a student denoiser to operate in just $1\!-\!4$ steps, using a frozen diffusion teacher for score distillation and an adversarial loss for fidelity, and is presented as the first method to unlock single-step, real-time image synthesis with foundation models (Sauer et al., 2023). SDXL-Lightning then systematized a progressive schedule, explicitly combining progressive and adversarial distillation to reduce SDXL from many steps to $8$, $4$, $2$, and $1$ steps at $1024$px (Lin et al., 2024).

The same principle has since been generalized beyond unconditional image sampling. POSE formulates one-step distillation for large-scale video diffusion models through a two-phase adversarial equilibrium in Gaussian noise space, with an additional conditional adversarial consistency mechanism for image-to-video generation (Cheng et al., 28 Aug 2025). AAD-1 addresses one-step autoregressive image-to-video generation by pairing a strictly causal generator with a bidirectional spatiotemporal discriminator, specifically to prevent motion collapse and long-range drift (Li et al., 2 Jun 2026). In adversarial purification, OSCP compresses diffusion-based purification into one NFE through Adversarial Consistency Distillation and Gaussian Adversarial Noise Distillation (Lei et al., 2024). In voice conversion, FasterVoiceGrad uses Adversarial Diffusion Conversion Distillation to obtain a one-step conversion model while simultaneously distilling the content encoder (Kaneko et al., 25 Aug 2025).

A recurring motive across these works is that aggressive step reduction creates a severe capacity and distribution mismatch. The student must approximate a multi-step reverse process in one or a few evaluations, and adversarial supervision is introduced because purely reconstruction- or consistency-based training often produces blur, artifacts, collapse, or semantic drift under this regime (Sauer et al., 2023, Lin et al., 2024, Li et al., 2 Jun 2026).

2. Objective formulations and distillation targets

A central formulation couples a teacher-guided distillation term with an adversarial generator loss. In ADD, the student minimizes

$$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$

where the adversarial term uses a hinge GAN objective and the distillation term is SDS-equivalent, implemented as an explicit denoised teacher target on a noised version of the student output (Sauer et al., 2023). This formulation is representative of image ASD systems in which the adversarial term corrects the blur and artifact profile of few-step denoisers, while the teacher term preserves the original diffusion model’s trajectory structure.

A second family uses score or distribution matching as the warm-up or primary supervision and then adds adversarial training to repair reverse-KL or consistency limitations. OSCP introduces Gaussian Adversarial Noise Distillation with

$$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$

where mixed Gaussian and adversarial latents are used to bridge natural and adversarial manifolds in latent space (Lei et al., 2024). DMDX replaces DMD’s reverse-KL alignment with Adversarial Distribution Matching, training a time-conditioned discriminator to distinguish teacher and fake score predictions at matched timesteps, and explicitly motivates this replacement by the mode-seeking and zero-forcing behavior of reverse KL in one-step distillation (Lu et al., 24 Jul 2025).

A third formulation adversarially aligns adjacent student trajectories rather than directly contrasting student and teacher outputs. In the causal video framework titled "Towards One-step Causal Video Generation via Adversarial Self-Distillation," the ASD loss is

$$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$

with a relativistic pairing objective that aligns the student’s $n$-step and $8$0-step noisy outputs (Yang et al., 3 Nov 2025). This design is explicitly motivated by the claim that the intra-student gap is smaller and smoother than the direct gap between an extreme few-step student and a multi-step teacher.

Teacher-aligned adversarial targets also appear in two-step image generation. Z-Image Turbo++ trains a $8$1-step student against teacher-generated images rather than external real images, using the standard logistic GAN losses

$8$2

$8$3

and argues that teacher outputs provide a stronger yet more attainable adversarial target for two-step generation (Liu et al., 10 Jun 2026).

3. Discriminator architectures, asymmetry, and the definition of “real”

The discriminator in adversarial step distillation is rarely a generic GAN critic. ADD uses a projection-style discriminator built on top of a frozen feature network $8$4, with trainable heads at multiple feature layers and conditioning on both text embeddings and image embeddings; the paper emphasizes that the adversarial signal is applied to high-level features rather than raw pixels alone (Sauer et al., 2023). SDXL-Lightning further specializes the critic by conditioning it on the anchor state $8$5, the predicted transition target, timestep indices, and text conditioning, so that the discriminator learns a local ODE transition rather than only a final image realism judgment (Lin et al., 2024).

Later work diversifies what is meant by “real” and “fake.” DMDX does not adversarially compare only final images; instead, its discriminator is trained on teacher and fake score predictions after a small PF-ODE step to $8$6, with the stated goal of aligning score fields over matched noise levels (Lu et al., 24 Jul 2025). Z-Image Turbo++ goes in a different direction and defines teacher-generated images, not external photographs, as the real distribution for GAN training, arguing that this removes intrinsic photograph-versus-diffusion discrepancies that a $8$7-step student cannot close (Liu et al., 10 Jun 2026).

The strongest architectural asymmetry appears in video. AAD-1 keeps the generator strictly causal, with sink tokens and a local sliding window, but trains a bidirectional discriminator over the entire clip and reduces the output to a single holistic realism score $8$8 (Li et al., 2 Jun 2026). The paper explicitly states that this asymmetry lets the discriminator detect global temporal failures and long-range drift that frame-wise or causal critics miss. POSE uses a different asymmetry: the discriminator backbone is the EMA of the generator itself, augmented with a cross-attention semantic head that fuses multimodal conditions and video features while operating in low-SNR Gaussian noise space (Cheng et al., 28 Aug 2025).

Some systems remove the distinction between score network and discriminator almost entirely. SiDA reuses the encoder of the fake-score network as a discriminator, produces a $8$9D discriminator map rather than a single scalar, and batch-normalizes the adversarial loss within each GPU batch before fusing it with the score-identity objective (Zhou et al., 2024). This design is presented as parameter-free on the discriminator side and specifically intended to correct teacher-score bias with real-image supervision.

4. Progressive schedules and stabilization mechanisms

A consistent result across the literature is that direct one-step adversarial training is unstable when the student is far from the teacher or data manifold. AAD-1 states this explicitly for autoregressive video: in one-step regimes trained from scratch, the student’s rollouts are far off-manifold, the mismatch compounds over time, and the discriminator overwhelms the generator with gradients that do not correspond to feasible one-step corrections (Li et al., 2 Jun 2026). The proposed remedy is a three-stage schedule: ODE initialization for $4$0 steps, a DMD warm-up for $4$1 student steps with early stopping and generator-to-fake-score update ratio $4$2, and then adversarial refinement for $4$3 generator steps, together with timestep-dependent Gaussian noise on the discriminator inputs and approximate R1/R2 regularization with $4$4 and $4$5 (Li et al., 2 Jun 2026).

SDXL-Lightning adopts a broader progressive curriculum. Its schedule first performs $4$6 step MSE distillation and then adversarially distills $4$7. The paper also hard-swaps the training input at $4$8 to pure noise, because SDXL’s native schedule does not reach pure noise during training even though inference starts from pure noise. For $4$9-step and $2$0-step students, the discriminator is additionally trained at multiple renoised timesteps $2$1, with the sampling ratio later biased toward $2$2 to boost detail (Lin et al., 2024).

POSE turns stabilization into the organizing principle of the method. Phase I, “stability priming,” performs score-difference learning across an SNR curriculum from higher to lower SNR, using a Light LoRA fake model to approximate the student’s score or velocity. Phase II, “unified adversarial equilibrium,” then runs a min–max game in Gaussian noise space with an EMA-backed discriminator and a spatiotemporal R1 penalty. The implementation details state $2$3 steps for priming, $2$4 steps for the adversarial phase, Adam with $2$5 in Phase I and $2$6 in Phase II, and EMA decay $2$7 (Cheng et al., 28 Aug 2025).

DMDX likewise separates support-overlap creation from adversarial score-field alignment. Its pipeline first runs Adversarial Distillation Pre-training on teacher ODE pairs and then performs ADM fine-tuning, with reported one-step SDXL settings of $2$8K ADP iterations plus $2$9K ADM iterations (Lu et al., 24 Jul 2025). SiDA introduces a different stabilization device: the adversarial term is averaged over spatial locations and the local batch on each GPU before being combined with the pixel-weighted SiD loss, which the paper describes as a way to avoid brittle loss-balancing heuristics (Zhou et al., 2024).

5. Representative applications and empirical results

In image synthesis, adversarial step distillation has produced competitive or teacher-surpassing one-step systems. On COCO zero-shot FID5k, ADD-M at $1$0 step reports time $1$1s, FID $1$2, and CLIP $1$3, outperforming earlier one-step baselines in the same table, while ADD-XL at $1$4 step runs at $1$5 seconds per image and at $1$6 steps at $1$7 seconds per image on $1$8 A100 mixed-precision inference (Sauer et al., 2023). SDXL-Lightning reports, for its full models, FID-Whole/FID-Patch/CLIP of $1$9 at $1024$0 step, $1024$1 at $1024$2 steps, $1024$3 at $1024$4 steps, and $1024$5 at $1024$6 steps (Lin et al., 2024). SiDA goes further on teacher surpassing: it reports FID $1024$7 on ImageNet $1024$8, and on ImageNet $1024$9 it reports FIDs $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$0 for XS, $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$1 for S, $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$2 for M, $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$3 for L, $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$4 for XL, and $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$5 for XXL, all in one step and without CFG, while the teacher EDM2-XXL is reported at FID $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$6 using CFG and $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$7 generation steps (Zhou et al., 2024). For $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$8-step generation, Z-Image Turbo++ reports OneIG $$\mathcal{L} = \mathcal{L}_{\mathrm{adv}}^{\mathrm G}(\hat{x}_{\theta}(x_s,s), \phi) + \lambda \mathcal{L}_{\mathrm{distill}}(\hat{x}_{\theta}(x_s,s), \psi),$$9, GenEval $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$0, DPG-Bench $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$1, LongText-CN $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$2, and LongText-EN $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$3, substantially narrowing the gap to its $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$4-step teacher (Liu et al., 10 Jun 2026).

Video results show that adversarial step distillation is not limited to endpoint realism. On VBench-I2V $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$5s, $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$6p, AAD-1 Stage-III at $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$7 NFE reports Subject Consistency $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$8, Background Consistency $$\mathcal{L}_{\mathrm{Total}}(\theta,\theta^-) = \mathcal{L}_{\mathrm{GAND}}(\theta,\theta^-) + \lambda_{\mathrm{CIG}} \mathcal{L}_{\mathrm{CIG}}(\theta),$$9, Motion Smoothness $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$0, Imaging Quality $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$1, I2V Subject $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$2, and I2V Background $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$3 (Li et al., 2 Jun 2026). POSE reports a $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$4 latency reduction from $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$5 s to $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$6 s for $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$7s $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$8 video, together with an average $$\mathcal{L}_{\mathrm{ASD}}(\theta,\psi) = \mathbb{E} \left[ f\!\left( D_\psi^n(\Psi(G_\theta^n(\mathbf{z}_1))) - D_\psi^n(\Psi(G_\theta^{n+1}(\mathbf{z}_2))) \right) \right],$$9 improvement over acceleration baselines on VBench-I2V, and gives representative $n$0-NFE scores of I2V $n$1, overall quality $n$2, semantic alignment $n$3, motion smoothness $n$4, dynamic degree $n$5, aesthetics $n$6, and image quality $n$7 (Cheng et al., 28 Aug 2025). The causal video ASD framework reports, with First-Frame Enhancement, VBench totals of $n$8 for $n$9-step generation and $8$00 for $8$01-step generation, compared with $8$02 and $8$03 for the Self Forcing baselines at $8$04 and $8$05 steps, respectively (Yang et al., 3 Nov 2025).

In adversarial purification, OSCP reports a $8$06 defense success rate on ImageNet AutoAttack $8$07 with $8$08, runtime $8$09 s per $8$10 image, and a claimed $8$11-fold speedup compared to conventional diffusion purification approaches (Lei et al., 2024). In voice conversion, FasterVoiceGrad reports GPU real-time factor reductions from $8$12 to $8$13 on VCTK and from $8$14 to $8$15 on LibriTTS, corresponding to $8$16 and $8$17 times faster conversion, together with UTMOS $8$18, DNSMOS $8$19, CER $8$20, and SECS $8$21 on VCTK (Kaneko et al., 25 Aug 2025).

6. Limitations, misconceptions, and open problems

A common misconception is that adversarial loss alone solves the one-step distillation problem. In the works surveyed here, adversarial supervision is consistently paired with score, consistency, or distribution matching, and several papers state that adversarial training by itself is unstable in the extreme low-step regime. ADD attributes one-step blur to the inadequacy of non-adversarial distillation but still relies on SDS-style teacher guidance (Sauer et al., 2023). AAD-1 states that one-step adversarial training from scratch leads to rollouts that are far off-manifold and to collapse unless the student is first warmed up by diffusion forcing and DMD (Li et al., 2 Jun 2026). POSE likewise separates stability priming from adversarial equilibrium rather than optimizing both simultaneously from an unprimed student (Cheng et al., 28 Aug 2025).

Another misconception is that any discriminator topology is adequate. The video papers are explicit that causal or frame-wise critics can fail catastrophically. AAD-1 reports Dynamic Degree $8$22 for a causal plus frame-wise discriminator, describing this setting as static collapse, and reports Drift Score $8$23 for causal video-wise heads versus $8$24 for bidirectional video-wise heads on $8$25s clips (Li et al., 2 Jun 2026). SDXL-Lightning documents a different failure mode, “Janus” artifacts, and addresses it by switching from a conditional flow-preserving discriminator to an unconditional mode-relaxed fine-tuning phase (Lin et al., 2024). FasterVoiceGrad identifies a further shortcut: if the content encoder is trained in reconstruction mode, the system can learn an identity mapping rather than the intended conversion process (Kaneko et al., 25 Aug 2025).

The definition of the adversarial target remains an active design axis rather than a settled standard. DMDX argues that reverse-KL-based DMD is intrinsically mode-seeking and motivates learned adversarial discrepancies over time-conditioned score predictions (Lu et al., 24 Jul 2025). Z-Image Turbo++ argues, by contrast, that using external real photographs as GAN targets can produce persistent artifacts because a $8$26-step student cannot close the full photograph-versus-diffusion gap; its solution is teacher-generated images as real samples (Liu et al., 10 Jun 2026). SiDA chooses real images, but only through their noisy versions, and uses them to correct teacher-score bias rather than to replace the teacher entirely (Zhou et al., 2024).

The current frontier is defined less by the possibility of one-step generation than by where that compression remains fragile. AAD-1 lists fast motion, complex structures such as faces and hands, and long-horizon extrapolation beyond $8$27s training clips as open limitations (Li et al., 2 Jun 2026). OSCP states that adaptive selection of the purification strength $8$28 remains open and that performance under other norm bounds or semantic and physical attacks may vary (Lei et al., 2024). Z-Image Turbo++ states that its step-decoupling and teacher-aligned adversarial target could scale to $8$29 steps and to video or audio diffusion, but also notes the increased parameter footprint and residual gap on dense text rendering (Liu et al., 10 Jun 2026). These reports suggest that future work will likely continue to focus on discriminator design, support-overlap initialization, and modality-specific structure rather than on adversarial loss in isolation.