---
title: Wasserstein Adversarial Examples
url: https://www.emergentmind.com/topics/wasserstein-adversarial-examples
type: topic
---

# Wasserstein Adversarial Examples

Wasserstein adversarial examples are inputs specifically crafted to induce misclassification in machine learning models, where the perturbation constraint is expressed not by $\ell_p$-norms but by the Wasserstein distance—a measure derived from optimal transport theory. Unlike traditional pixelwise norms, the Wasserstein metric quantifies the minimal "cost" of transforming a source example into a perturbed one by optimally "moving" mass among components (e.g., pixels or time points). This paradigm more naturally encodes spatial or temporal invariances, human-perceived similarity, and structural fidelity. Recent advances in both attack and defense under the Wasserstein threat model span a broad range of domains, encompassing robust image classification, generative modeling, imitation learning, and certified verification.

## 1. Mathematical Formulation of the Wasserstein Threat Model

The Wasserstein threat model departs from classical $\ell_p$-norm bounded perturbations by considering the Wasserstein distance $d_\mathcal{W}(x, x')$ defined over distributions induced by $x$ and its adversarially modified version $x'$. The adversarial example $x'$ satisfies:
$$
\mathcal{B}_\mathcal{W}(x, \epsilon) = \{x' : d_\mathcal{W}(x, x') \leq \epsilon\}
$$
where $d_\mathcal{W}$ is calculated as the optimal transport cost with respect to a cost matrix $C$, typically encoding spatial displacements or transformations. For images, this may involve moving pixel mass within local windows, capturing deformations such as translation, scaling, or blur that cannot be succinctly represented by $\ell_p$-norms [1902.07906]. For time series, the one-dimensional structure allows a closed-form characterization via cumulative distributions [2303.12357].

By relaxing the attack region to Wasserstein balls, the threat model encompasses "natural" and perceptually aligned perturbations, which are often imperceptible even when large under $\ell_p$-metrics.

## 2. Algorithmic Methodologies for Generating Wasserstein Adversarial Examples

Adapting projected gradient descent (PGD) to the Wasserstein ball introduces unique optimization challenges. The projection step,
$$
\min_{x' : d_\mathcal{W}(x, x') \leq \epsilon} \|x' - w\|_2^2
$$
is nontrivial and requires bespoke solvers:

- **Projected Sinkhorn Iteration:** A modified Sinkhorn scaling procedure is used to solve a regularized OT problem by iteratively updating dual variables, analogous to entropic regularized transport. Updates for dual vectors $(\alpha, \beta, \psi)$ are block-coordinate-wise, employing logarithmic and exponential transforms to compute efficient scaling and cost estimation [1902.07906]. Local transport restriction (e.g., $k \times k$ image patches) reduces computational complexity from $O(n^2)$ to $O(nk^2)$.

- **Dual Projection Operators and Frank–Wolfe Methods:** Later works advance exact dual projection algorithms based on Lagrangian duality, allowing line-search or bisection for dual variables and simplex projection row-wise. Entropic regularization within the Frank–Wolfe linear minimization oracle (LMO) grants further efficiency and strong convexity, providing primal solutions with closed-form updates per batch [2008.02883].

- **Time Series Specialization:** In one-dimensional domains, the Wasserstein projection leverages the closed-form expression of $d_\mathcal{W}$, enabling efficient gradient-descent-based projection and, optionally, a two-step routine: first constraining the search via an $\ell_\infty$ ball, then refining within the Wasserstein ball to balance attack imperceptibility and efficacy [2303.12357].

Under these methodologies, adversarial examples can reduce the accuracy of high-performing ResNet models on CIFAR-10 from $\sim94\%$ to $3\%$ for Wasserstein radii $\epsilon = 0.1$ [1902.07906], or as low as $3.4\%$ for radius $0.005$ with exact projections [2008.02883].

## 3. Distributional and Semantic Extensions

The distributional robustness perspective generalizes adversarial attacks to perturb entire probability measures within a Wasserstein ball. The adversarial risk is expressed as:
$$
\sup_{Q: \mathcal{W}_c(P, Q) < \epsilon} \mathbb{E}_Q[f(z)]
$$
for a base distribution $P$ and measurable cost $c$. This framework unifies and extends methods such as PGD-AT, TRADES, and MART by reinterpreting them as special cases where adversaries act pointwise rather than over distributions [2202.13437, 2306.09844].

Novel constructions such as the *Internal Wasserstein Distance* (IWD) compare empirical distributions of local features or patches, allowing generation of "semantically similar" adversarial examples. IWD-based attacks maximize classifier loss while ensuring the Wasserstein distance between internal feature sets (patch representations) remains small, producing diverse adversaries that respect the data manifold more closely than those derived via $\ell_p$-norms [2103.07598].

Layerwise or high-level feature perturbations further extend the approach. By perturbing intermediate decoder layers in generative autoencoder architectures and constraining Wasserstein distance in image space, adversarial examples exhibiting semantic alteration (such as added texture, structure, or color shifts) are achieved in a targeted, interpretable manner [2110.07182].

## 4. Certified Robustness and Verification

Defense against Wasserstein adversaries centers on both empirical approaches (adversarial training) and certified techniques. Certified robustness via **Wasserstein Smoothing** employs the relation that 1-Wasserstein distance between probability distributions of images can be upper bounded by the $\ell_1$ norm in a "flow" domain. Here, a *local flow plan* $\delta$ is injected with Laplacian noise:
$$
f^{WS}(x) = \mathbb{E}_{\delta \sim\, \mathrm{Laplace}(0, \sigma)}[f(\Delta(x, \delta))]
$$
where $\Delta(x, \delta)$ denotes the image after flow application. Robustness is then certified using bounds that guarantee invariance of the classifier's top prediction within a Wasserstein ball, leveraging prior theory from randomized smoothing in $\ell_1$ space [1910.10783].

A complementary verification technique “lifts” the certification and attack problem into the flow domain and applies convex polytope or $\ell_1$-ball methods. When the relevant layerwise affine mappings and feasibility constraints are enforced, this approach provides either complete or incomplete certification for the absence (or existence) of adversarial examples within Wasserstein balls [2110.06816].

## 5. Empirical Results and Applications

Substantial empirical evidence demonstrates the effectiveness of Wasserstein-based adversarial examples and defenses:

- On CIFAR-10, exact Wasserstein attacks (dual projection or Frank–Wolfe) reduce model accuracy to $\sim3\%$ for moderate $\epsilon$, outperforming previous approximate projections which yield only $65.6\%$ [2008.02883].
- Adversarial training with Wasserstein examples increases robustness to geometric deformations not captured by $\ell_p$-norm constraints, with adversarial accuracy improving from $3\%$ (no defense) to $76\%$ after PGD-based adversarial training [1902.07906].
- In time series (ECG) classification, Wasserstein adversaries achieve $100\%$ attack success rate on certain datasets, using much smaller and more imperceptible perturbations than $\ell_\infty$-based attacks [2303.12357].
- In unsupervised generative modeling, Wasserstein-based objectives allow drastic reductions in model size while maintaining or improving sample quality (e.g., a $20\times$ reduction in required CNNs compared to previous INN methods) [1711.08875].
- Certified defense strategies can guarantee nonexistence of adversaries within provable Wasserstein radii, although certified radii are often small for complex data modalities [1910.10783, 2303.12357].

## 6. Extensions, Limitations, and Open Directions

Research highlights several extensions and unresolved issues:

- The adaptation of Wasserstein adversarial example frameworks to non-image data such as univariate and multivariate time series, point clouds, and scientific simulations is an expanding area [1802.03325, 2303.12357].
- Certified robustness techniques remain limited by computational constraints—exact certificates are feasible for lower-dimensional or specially structured transformations, while high-dimensional image or flow domains often require incomplete or conservative relaxations [2110.06816].
- Adversarial training under Wasserstein or distributional threat models yields models with improved generalization to previously unseen or unmodeled distributional shifts, but practical scaling and trade-offs (e.g., selectivity and statistical efficiency) are still under study [2202.13437, 2306.09844].
- The interplay between metric uncertainty, model compactness, and robustness indicates lower bounds: universal robustness against an unknown class of transport metrics may require capacity-prohibitive models or additional cryptographic primitives [2007.06993].
- Semantically-aware threat models (internal Wasserstein, layerwise transport) offer increased attack diversity and challenge standard defense mechanisms, necessitating improved strategies for semantic robustness [2103.07598, 2110.07182].

## 7. Broader Impact and Research Directions

Wasserstein adversarial examples are reshaping the understanding of machine learning robustness by exposing model vulnerabilities to geometrically meaningful, theoretically principled, and perceptually aligned perturbations. The integration of optimal transport metrics into adversarial training, certified verification, and generative adversarial learning broadens the scope of robust ML, demanding both deeper theoretical foundations and practical algorithmic innovation. Current research continues to address computational bottlenecks (exact projection, scalable certification), domain extensions (multimodal and distributional settings), and adaptive reward/cost functions tailored for both robust prediction and generation.

Papers and code repositories relevant for practical implementations include [1902.07906], [2008.02883], [1910.10783], [2110.06816], [2303.12357], and [2306.09844].

Source: https://www.emergentmind.com/topics/wasserstein-adversarial-examples