Model-Based Adversarial Training

• Model-based adversarial training is a framework that leverages learned models to generate adaptive perturbations, improving sample efficiency and robustness.
• It employs techniques like learned optimizers, energy functions, and latent manifold regularization to outperform traditional fixed-attack methods.
• Empirical results indicate enhanced robustness, generative capability, and out-of-distribution detection across various benchmarks and data modalities.

Model-based adversarial training refers to a family of adversarial defense and generative modeling strategies in which the adversarial component of training is informed by a learned model—typically, the underlying deep neural network or an auxiliary model—rather than being restricted to hand-crafted perturbations or fixed algorithms such as PGD. These frameworks capitalize on the internal learned representations or energy functions to generate informative perturbations, enhance sample efficiency, stabilize optimization, and often yield strong adversarial and out-of-distribution robustness, as well as competitive generative performance. Methods in this class include adversarial training of energy-based models (EBMs), learned-optimizer-driven adversaries, model-based attacks in discrete input spaces, and manifold-guided adversarial objectives.

1. Core Principles and Formulations

Model-based adversarial training formalizes adversarial robustness as a minimax optimization problem, typically of the form: $$\min_\theta\ \mathbb{E}_{(x, y) \sim D}\left[\max_{\|x' - x\| \leq \epsilon}\, \mathcal{L}(f_\theta(x'), y)\right]$$ or, for model classes such as EBMs,

$$\max_{\theta} \mathbb{E}_{x \sim p_{\text{data}}}[\log D_\theta(x)] + \mathbb{E}_{x^* \sim p_T^*}[\log (1 - D_\theta(x^*))]$$

where the inner maximization is often implemented not by a fixed attack routine, but by an adaptive procedure tied to the model, such as a trainable optimizer, energy function, or manifold regularizer. This use of a model-aware inner adversary characterizes the "model-based" aspect of the framework (Yin et al., 2020, Xiong et al., 2020).

2. Energy-Based and Contrastive Approaches

Energy-based model (EBM) variants parameterize the unnormalized log-density via a scalar network, leading to Gibbs distributions: $$p_\theta(x) = \exp(f_\theta(x)) / Z(\theta), \qquad E_\theta(x) = -f_\theta(x)$$ Adversarial training within EBMs consists of an outer update that adapts the energy function by contrasting real data and "adversarial negatives" produced by running projected gradient steps on $f_\theta$. For instance, the approach in (Yin et al., 2020) interprets binary adversarial training as learning a discriminator $D(x) = \sigma(f_\theta(x))$, with inner maximization samples generated by PGD from a diverse, out-of-distribution pool $p_0$. This aligns model-based AT with short-run MCMC maximum likelihood, where adversarial perturbations explore and suppress spurious modes in the energy landscape. Theoretical results indicate that the optimum for such objectives recovers the support of the true data distribution, unlike MLE EBMs which target density matching.

Contrastive energy-based models (CEMs) generalize this view by showing that adversarial training’s core step, $$\hat{x} = \arg\max_{\|\hat{x} - x\| \leq \epsilon} -\log p_\theta(y \mid \hat{x})$$, implements a biased negative-phase sample for maximum likelihood, leading to robust classifiers with unexpectedly strong generative capabilities (Wang et al., 2022).

3. Learned Optimizers and Adaptive Adversaries

Model-based adversarial training encompasses learned-optimizer frameworks, where the inner maximization in adversarial training is replaced by a trainable optimizer—often a coordinate-wise RNN. In (Xiong et al., 2020), the attack is parameterized as: $$(\delta_t, h_{t+1}) = m_\phi(g_t, h_t), \qquad x_{t+1}' = \Pi_{B(x,\epsilon)}(x_t' + \delta_t)$$ with $g_t = \nabla_{x'} \mathcal{L}(f_\theta(x_t'), y)$ and $m_\phi$ trained to generate stronger adversarial examples than fixed-step PGD. Model and optimizer parameters are co-trained under a bilevel regime, with the learned optimizer adapting attack strength and direction per sample and step. This approach produces stronger adversaries, accelerates convergence, yields smoother minimax solutions, and increases robustness over vanilla PGD or CNN-based inner loops.

4. Latent Space and Manifold-guided Methods

Manifold Adversarial Training (MAT) (Zhang et al., 2018) extends adversarial training into model-based regimes by linking adversarial objectives with explicit latent-space regularization. The underlying principle is that the latent representation—often modeled by a class-conditioned GMM—offers a more informative adversarial playground than the output space. The worst-case perturbations are those that maximally disrupt the smoothness of the latent-data manifold, as quantified by the KL-divergence between Gaussian mixtures: $$\delta_{\rm adv} = \arg\max_{\|\delta\| \leq \sigma} KL\left[P_{\rm GM}(f_\theta(x)) \,\|\, P_{\rm GM}(f_\theta(x + \delta))\right]$$ The training objective penalizes both output and latent distributional roughness while simultaneously regularizing feature compactness via mutual information, generalizing center loss approaches. Empirically, MAT improves supervised and semi-supervised robustness and yields locally flatter latent manifolds relative to classic adversarial or virtual adversarial training.

5. Domain-Specific Model-Based Strategies

In discrete domains such as NLP, model-based adversarial training involves generating input perturbations through model-guided search, such as best-first search (BFF) or random sampling in the combinatorial space of label-preserving transformations (Ivgi et al., 2021). The inner loop adapts attacks online, aligning with the model's evolving weaknesses. Empirical results show that online augmentation with search-based attacks yields substantial gains in robust accuracy—at significant computational cost—whereas random sampling offers competitive robustness at dramatically reduced cost.

Feature-Scattering Adversarial Training (FSAT) (Zhang et al., 2019) exemplifies another model-based approach, generating adversarial inputs by maximizing the optimal-transport distance between batchwise feature distributions—without directly using label information in the perturbation. This collaborative, batch-level scattering of features enhances robustness and mitigates issues such as label leakage common in classic supervised perturbation methods.

6. Key Algorithms and Empirical Results

The following table summarizes representative model-based adversarial training approaches and their core mechanisms:

Approach	Inner Maximization	Outer Optimization Target
EBM/AT (Yin et al., 2020)	PGD on $f_\theta$, diverse pool	Max-min on $J(\theta)$ over support
RNN-AT (Xiong et al., 2020)	Learned RNN optimizer	Minimax over strong adversaries
MAT (Zhang et al., 2018)	KL-divergence in latent	Cross-entropy + smoothness + MI
FSAT (Zhang et al., 2019)	OT distance in feature	Classification loss on scattered batch
BFF/Discrete (Ivgi et al., 2021)	Best-first search/random	Minimax with online augmentation

In EBM-based adversarial training, experiments on CIFAR-10 show that the modeled support of $p_{\text{data}}$ yields Inception Score = 9.10, FID = 13.21—competitive with explicit EBMs—and robust OOD detection (AUC ~70–83%). Learned-optimizer-based AT achieves robust accuracies exceeding standard PGD-based baselines by 2–7 percentage points on CIFAR-10 depending on evaluation protocol. MAT registers lower error rates on MNIST and CIFAR-10 than virtual adversarial training (0.42% vs 0.72% on MNIST, 4.40% vs 5.81% on CIFAR-10). FSAT improves robust accuracy (e.g., PGD20: 70.5% vs. Madry-PGD's 44.9% on CIFAR-10), while BFF-based online augmentation in NLP improves robust accuracy on IMDB to 78.9% albeit at high computational cost (Ivgi et al., 2021).

7. Theoretical Analysis and Interpretation

A unifying perspective frames model-based adversarial training as a biased form of maximum likelihood in contrastive or energy-based models, where adversarial samples function as negative-phase surrogates (Wang et al., 2022). This demystifies the generative competence of robust classifiers: inner-loop adversarial maximization lowers the model probability of synthetic, off-manifold points, thus flattening the energy landscape outside the data manifold and enabling model inversion for generative sampling. The theoretical optimum in EBM-AT formulations ensures that the learned classifier precisely supports the data manifold (level set of $D(x)=1/2$ for $x \in \text{supp}\ p_{\text{data}}$), but does not recover the full data density, distinguishing it from MLE-trained EBMs (Yin et al., 2020).

These perspectives collectively clarify that model-based adversarial training is not solely a defense mechanism but a principled framework for driving model’s internal representations and output distributions to manifest desirable geometric, statistical, and generative properties across diverse data modalities and application domains.