Duality for the Adversarial Total Variation

Abstract: Adversarial training of binary classifiers can be reformulated as regularized risk minimization involving a nonlocal total variation. Building on this perspective, we establish a characterization of the subdifferential of this total variation using duality techniques. To achieve this, we derive a dual representation of the nonlocal total variation and a related integration of parts formula, involving a nonlocal gradient and divergence. We provide such duality statements both in the space of continuous functions vanishing at infinity on proper metric spaces and for the space of essentially bounded functions on Euclidean domains. Furthermore, under some additional conditions we provide characterizations of the subdifferential in these settings.

• The paper derives duality characterizations for adversarial total variation with explicit subdifferential formulas in both C₀(X) and L∞(Ω).
• It recasts adversarial training in binary classification as a regularized risk minimization using a nonlocal total variation functional.
• The paper provides rigorous optimality conditions that facilitate the design of primal-dual algorithms for robust adversarial learning.

Duality Characterizations for Adversarial Total Variation

Motivation and Context

The adversarial vulnerability of machine learning models—especially deep neural networks—has motivated significant developments in robust optimization. Adversarial training, as formalized by Madry et al., replaces standard empirical risk minimization with a risk computed on adversarial perturbations, resulting in a maximization-minimization problem with non-differentiable objectives. Recent mathematical studies have established connections between adversarial training in binary classification and regularization using a nonlocal total variation functional, particularly in the non-parametric regime. This paper, "Duality for the Adversarial Total Variation" (2604.18540), rigorously develops duality characterizations for the adversarial total variation, with a focus on explicit subdifferential formulas in both the spaces of continuous vanishing functions on proper metric spaces ($C_0(X)$) and essentially bounded functions on Euclidean domains ($L^\infty(\Omega)$).

Adversarial Training and Nonlocal Total Variation

The paper recasts adversarial training in binary classification as a regularized risk minimization involving a nonlocal total variation functional:

$$\inf_{u \in H} \mathbb{E}_{(x, y)\sim\mu}[\ell(u(x), y)] + \lambda \mathcal{R}(u),$$

where $\mathcal{R}$ is a nonlocal total variation regularizer constructed from the adversarial risk. For binary classification, previous work showed existence of minimizers and connected the nonlocal total variation to geometric properties of decision boundaries. Gamma-convergence results have established that in the small adversarial budget regime, the nonlocal total variation reduces to an anisotropic local total variation, justifying its nomenclature and mathematical fidelity.

Duality Formulation in $C_0(X)$

For $C_0(X)$, the space of continuous functions vanishing at infinity on proper metric spaces, the paper derives a dual representation:

$$\text{TV}_\varepsilon(u) = \max_{\underline{\mathbf{m}} \in \mathfrak{M} \times \mathfrak{M}} \left\{ -\int_X u(x) \, d \operatorname{div}[\underline{\mathbf{m}}](x) \right\},$$

where $\mathfrak{M}$ represents random walks (measurable families of probability measures with support in adversarial balls), and $\operatorname{div}$ is a nonlocal divergence operator dual to a nonlocal gradient $$grad_\varepsilon[u](x,y) = \frac{u(y) - u(x)}{\varepsilon}$$.

The dual representation leverages measurable selector theorems and properties of convex analysis to characterize the subdifferential explicitly: elements are nonlocal divergences of maximizer random walks, i.e., for $L^\infty(\Omega)$0 and compact $L^\infty(\Omega)$1, $L^\infty(\Omega)$2 if and only if there exists random walks $L^\infty(\Omega)$3 such that $L^\infty(\Omega)$4 and $L^\infty(\Omega)$5.

Duality Formulation in $L^\infty(\Omega)$6

On Euclidean domains, the paper develops a dual representation for essentially bounded functions, where the adversarial total variation is expressed using test functions from $L^\infty(\Omega)$7:

$L^\infty(\Omega)$8

with $L^\infty(\Omega)$9 the set of admissible (jointly measurable) test functions satisfying normalization and support constraints. Unlike the $$\inf_{u \in H} \mathbb{E}_{(x, y)\sim\mu}[\ell(u(x), y)] + \lambda \mathcal{R}(u),$$0 setting, the subdifferential characterization relies on weak-* closure in the dual space $$\inf_{u \in H} \mathbb{E}_{(x, y)\sim\mu}[\ell(u(x), y)] + \lambda \mathcal{R}(u),$$1, which is the space of bounded finitely additive measures absolutely continuous with respect to Lebesgue measure. The maximization may not be attained; instead, subgradients are characterized as limits (nets) of nonlocal divergences generated by the admissible test functions.

The nonlocal divergence operator acts as an adjoint to the nonlocal gradient:

$$\inf_{u \in H} \mathbb{E}_{(x, y)\sim\mu}[\ell(u(x), y)] + \lambda \mathcal{R}(u),$$2

which, under smoothness and $$\inf_{u \in H} \mathbb{E}_{(x, y)\sim\mu}[\ell(u(x), y)] + \lambda \mathcal{R}(u),$$3, approximates the classical divergence.

Integral and Subdifferential Characterizations

For $$\inf_{u \in H} \mathbb{E}_{(x, y)\sim\mu}[\ell(u(x), y)] + \lambda \mathcal{R}(u),$$4, the subdifferential admits an explicit integral characterization, allowing applications such as formulation of total variation flows and numerical saddle point optimization schemes. For $$\inf_{u \in H} \mathbb{E}_{(x, y)\sim\mu}[\ell(u(x), y)] + \lambda \mathcal{R}(u),$$5, the characterization is more abstract, requiring limit arguments with nets due to the lack of separability. The duality approach enables the extension of primal-dual algorithms to adversarial total variation regularization—an avenue hitherto unexplored in adversarial machine learning.

Key implications include:

• The nonlocal divergence and gradient operators are consistent across the two settings and coincide with local differential operators as $$\inf_{u \in H} \mathbb{E}_{(x, y)\sim\mu}[\ell(u(x), y)] + \lambda \mathcal{R}(u),$$6, providing theoretical justification for geometric adversarial regularization.
• Explicit formulas for the subdifferential provide rigorous optimality conditions, facilitating well-posedness analysis and algorithmic development for adversarially regularized learning.

Practical and Theoretical Implications

The duality framework and subdifferential characterizations open new possibilities for analytical treatment of adversarially robust learning objectives. In particular:

• Algorithmic Formulations: The saddle point structures and explicit optimality conditions allow for primal-dual and projected-gradient schemes analogous to those in classical TV-regularized inverse problems, but now in the adversarial training context.
• Statistical Properties: The duality reveals connections to robust risk, geometry of decision boundaries, and ramifications for generalization in the presence of adversarial noise.
• Analytical Extensions: The methods may enable further study of existence, uniqueness, and regularity properties for adversarially regularized learning, including multiclass extensions and general loss functions.

Future directions identified in the paper include extending measurable selector constructions to general reference measures, further refining the closure of subdifferential sets, and developing nonlocal versions of Anzellotti pairings for pointwise characterizations.

Conclusion

This work provides a mathematically rigorous duality framework for adversarial total variation regularization in robust learning, with explicit characterization of subdifferentials in both continuous and essentially bounded function settings. The duality formulas underpin both theoretical analysis and the design of computational algorithms for adversarial training, offering a foundation for further research in nonlocal geometric regularization, robust optimization, and adversarial machine learning.