Second-Order Duality (Hessian-KR)

• Second-Order Duality (Hessian-KR) is a framework that generalizes classical optimal transport dualities by incorporating second-order differential constraints and Hessian structures.
• It bridges convex geometry, partial differential equations, and variational analysis with practical applications in mechanics, optimal transport, and deep learning.
• The framework supports efficient algorithm design through quasi-Newton methods and rank-one decompositions by leveraging Hessian metrics and dual potential formulations.

Second-order duality (Hessian–Kantorovich–Rubinstein, or Hessian-KR) theory generalizes classical optimal transport dualities to settings governed by second-order differential information, typically encoded via Hessian constraints or second-order variational structures. This theoretical framework bridges optimal transport, convex geometry, partial differential equations, and variational analysis, and enables a unified treatment of second-order structures in analysis, geometry, mathematical optimization, and applied settings such as mechanics and deep learning.

1. Primal and Dual Formulations in Hessian–KR Theory

Second-order duality in the sense of Kantorovich–Rubinstein for the Hessian centers on a duality between a three-marginal optimal transport problem and a variational supremum over functions with prescribed Hessian bounds. Specifically, for probability measures $\mu, \nu \in \mathcal{P}_2(\mathbb{R}^n)$ of finite second moment and equal barycenter, with $f = \nu - \mu$, the primal is: $$J(\mu, \nu) = \inf_{\pi \in \Sigma(\mu, \nu)} \iiint c(x,y,z)\,\pi(dxdydz)$$ where $c(x,y,z) = \frac12|z-x|^2 + \frac12|z-y|^2$ and $\Sigma(\mu, \nu)$ is the set of couplings with marginals $\mu$, $\nu$, and a third marginal $\rho$ dominating both in the convex order (Bołbotowski et al., 2024).

The dual form is: $$I(f) = \sup \left\{ \int u\,d\nu - \int u\,d\mu : u \in C^{1,1}(\mathbb{R}^n), -I \le D^2 u(x) \le I \right\}$$ with $D^2 u(x)$ denoting the Hessian and $C^{1,1}$ the space of functions with Lipschitz gradient.

The duality theorem asserts $I(f) = J(\mu, \nu)$, both with achieving optimizers, and the extremality condition is characterized by a first-order optimality identity linking the dual potential $u$ and transport plan $\pi$ (Bołbotowski et al., 2024).

2. Second-Order Structures: Hessian Metrics and Monge–Ampère

The Hessian–KR duality emerges naturally on Hessian manifolds, where the metric tensor is locally given by the Hessian of a potential $\Phi$. In optimal transport, the quadratic cost induces the real Monge–Ampère equation: $\det D^2 \Phi(x) = \frac{f(x)}{g(\nabla \Phi(x))}$ governing the transport map $T(x) = \nabla \Phi(x)$. Dual potentials are connected by the Legendre transform, with $D^2 \Phi^*(y) = [D^2 \Phi(x)]^{-1}$ under $y = \nabla \Phi(x)$ and $x = \nabla \Phi^*(y)$. This establishes a geometric duality between the “primal” Hessian metric $g = D^2 \Phi$ and its dual $g^* = D^2 \Phi^*$ (Hultgren, 2023).

This dual structure enables symmetric treatment of source and target measures and translates analytic questions about transport into Hessian geometry. In mirror symmetry, the existence and uniqueness of Monge–Ampère potentials correspond to moduli of Calabi–Yau metrics, with SYZ fibrations arising as limits of these dualities.

3. Beckmann Representation and Rank-One Decomposition

The Hessian–KR framework admits an equivalent second-order Beckmann problem: $$I'(f) = \min \left\{ \int \rho^0(\sigma) : \sigma \in \mathcal{M}(\mathbb{R}^n; S^n), \; \operatorname{Div}^2 \sigma = f \right\}$$ where $\rho^0$ is the Schatten-1 norm, $S^n$ the symmetric matrices, and $\operatorname{Div}^2$ the double divergence operator (Bołbotowski et al., 2024). There is no duality gap: $I(f) = I'(f)$, and every minimizer $\sigma$ admits a representation as an integral of elementary rank-one measures $\sigma^{x,y,z}$ supported on straight segments between $x, z$ and $z, y$. In two dimensions, this describes the static elasticity of grillages, with the measure $\sigma$ concentrated on the bars of an optimal supporting graph.

The support of $\sigma$ lies inside unions of balls centered at midpoints between support points of $\mu$ and $\nu$, demonstrating geometric localization of optimal structures.

4. Second-Order Duality in Quasi-Newton Methods and Deep Learning

Beyond geometry, Hessian–KR duality underpins modern scalable quasi-Newton methods via Kronecker factorization of blockwise Hessians. For deep networks (MLPs or CNNs), the Hessian of the loss with respect to layer weights is approximated by $A \otimes G$, where $A$ and $G$ are empirical covariance and Gram matrices associated with layer inputs and outputs.

The dual variables are $H_A \approx A^{-1}$ and $H_G \approx G^{-1}$, so that the Newton step for each layer can be written as

$\Delta W = - H_G (\nabla_W f) H_A$

This decomposition enables efficient updates via BFGS/L-BFGS procedures without explicit large-matrix inversion, providing strong convergence guarantees and practical acceleration over first-order methods. Empirical results on autoencoders and ConvNet classifiers demonstrate performance parity or superiority relative to state-of-the-art optimizers such as KFAC and ADAM (Ren et al., 2021).

Algorithm	Storage Requirement	Per-Iteration Time
K-BFGS	$O(J^2 |\Delta|^2 + I^2 + IJ|\Delta|)$	$O(m J^2 |\Delta|^2/T_{curv} + m I^2/T_{curv})$
KFAC	$O(J^2 |\Delta|^2 + I^2 + IJ|\Delta|)$	$$O(m(J^2|\Delta|^2+I^2)/T_1 + J^3|\Delta|^3/T_2 + I^3/T_2)$$
SGD-m/Adam	$O(IJ|\Delta|)$	$O(IJ|\Delta|)$

Here, $I, J$ are output and input channels, $|\Delta|$ is filter size, $m$ minibatch size, and $T_{curv}$ curvature update frequency (Ren et al., 2021).

5. Second-Order Duality in Multiobjective Optimization

Second-order duality principles also appear in constrained, nonsmooth multiobjective fractional programming, with the Mond–Weir dual problem involving second-order generalized derivatives. The primal problem optimizes a vector-fractional objective $\frac{f}{F}(x)$ over a feasible set $X$ defined by locally Lipschitz, Clarke-regular maps with suitable constraints. The dual variables $(\lambda, \mu, \nu)$ satisfy stationarity, second-order curvature, and slackness conditions, involving first-order Gâteaux derivatives and second-order Páles–Zeidan derivatives (Chen et al., 2024).

Key duality results include:

• Weak duality: dual feasible solutions give lower bounds to all primal outcomes.
• Strong duality: under generalized second-order Abadie-type regularity and second-order KKT conditions, a primal solution yields an optimal dual solution.
• Multiplier relations: second-order complementary slackness links multipliers to generalized critical cones.

These results generalize classical strong/weak duality from smooth, scalar programs to multiobjective, nonsmooth, and second-order regimes.

6. Applications and Geometric Interpretations

Hessian–KR duality enables explicit structural results in mechanics, notably in elastic grillages and plate theory. In $n=2$, the double divergence equation for the optimal measure encodes equilibrium of thin plates under planar loading. The solution's support is a union of bars between atomic loads, facilitating design and analysis in structural optimization. In differential geometry, the role of Hessian potentials and the Monge–Ampère equation is central to the metric geometry of special affine and Hessian manifolds, with deep connections to mirror symmetry and moduli of Calabi–Yau structures (Bołbotowski et al., 2024, Hultgren, 2023).

7. Proof Strategies and Analytical Tools

Well-posedness and attainment of the dual are secured by direct methods, leveraging the closedness of the set of symmetric matrix-valued fields under convex, positive-homogeneous functionals, with existence supplied by the Arzelà–Ascoli theorem and truncation arguments. The absence of duality gap is established via Fenchel–Moreau conjugacy and inf-convolution perturbation. Martingale and convex order techniques furnish the relation between three-marginal OT couplings and monotone transport, with Strassen’s theorem playing a crucial role in characterizing convex order dominance (Bołbotowski et al., 2024, Hultgren, 2023).

In optimization contexts, chain-rule expansions of generalized derivatives and explicit curvature inequalities are exploited to bridge primal and dual statements, with the Clarke directional and Páles–Zeidan second-order derivatives governing the analysis in nonsmooth settings (Chen et al., 2024).

---

Second-order duality (Hessian-KR) thus forms a foundational pillar for advanced optimal transport, variational problems with second-order constraints, scalable optimization in modern machine learning, and geometric analysis, all unified under a rigorous duality framework explicitly involving the Hessian and its geometric-functional properties.