Canonical Primal-Dual Method

• Canonical primal-dual method is a reformulation strategy that converts optimization problems into saddle point structures, eliminating the duality gap under mild assumptions.
• It underpins advanced proximal algorithms, including generalized Chambolle-Pock schemes, to accelerate convergence and improve computational performance in structured problems.
• The method facilitates global optimization for nonconvex problems by providing rigorous convergence analysis and often outperforming standard SDP relaxations in scalability and accuracy.

The canonical primal-dual method, rooted in canonical duality theory, provides a unified approach for addressing both convex and nonconvex optimization problems by reformulating the original problem into a saddle point program or a convex-concave form. This methodology underpins a wide spectrum of modern optimization algorithms, ranging from the Chambolle-Pock primal-dual scheme for convex saddle point problems to globally convergent algorithms for highly nonconvex polynomial and quadratic programs. The theoretical foundation guarantees strong duality (no duality gap) under mild regularity assumptions, enabling rigorous convergence analysis and often superior numerical performance in practice.

1. The Canonical Primal-Dual Reformulation

The canonical primal-dual approach begins with a general minimization problem, potentially nonconvex, of the form

$$\min_{x \in X} P(x) = W(x) + \frac{1}{2} x^T A x - f^T x,$$

where $x \in \mathbb{R}^n$, $W$ is possibly nonconvex, $A$ is symmetric, and $f$ is given. Canonical duality theory introduces an auxiliary variable via a transformation $\xi = \Lambda(x)$ such that $W(x) = V(\Lambda(x))$, where $V$ is strictly convex and differentiable. Using the Legendre-Fenchel conjugate, $V^*(\varsigma)$, the original problem is equivalently reformulated as a convex–concave saddle point problem: $$\min_{x \in X} \max_{\varsigma \in S^+} \Xi(x, \varsigma),$$ where the total complementarity function is

$x \in \mathbb{R}^n$0

with dual feasible set $x \in \mathbb{R}^n$1 defined by $x \in \mathbb{R}^n$2, $x \in \mathbb{R}^n$3. The canonical dual function $x \in \mathbb{R}^n$4 is obtained by minimizing $x \in \mathbb{R}^n$5 with respect to $x \in \mathbb{R}^n$6 for fixed $x \in \mathbb{R}^n$7: $x \in \mathbb{R}^n$8 where $x \in \mathbb{R}^n$9 (Wu et al., 2012, zhou, 2012, Latorre, 2014).

2. Complementary-Duality Principle and Global Optimality

The critical point theory of canonical duality establishes that if $W$0 is a stationary point of the dual maximization and $W$1, then the corresponding $W$2 is a stationary point of the primal, and $W$3 (complementary-duality equality). When $W$4, $W$5 is in fact a global minimizer. This eliminates the duality gap, an essential difference from classical Lagrangian duality for nonconvex problems, where duality gaps can be significant (Wu et al., 2012, zhou, 2012, Latorre, 2014).

3. Proximal Algorithms and Generalized Chambolle-Pock Primal-Dual Schemes

For convex-concave saddle point structures, the canonical primal-dual methodology underpins the design of proximal algorithms with explicit primal and dual updates. The iteration for the classical Chambolle–Pock scheme for

$W$6

is given by

$W$7

where $W$8 denotes the proximal operator. The original convergence condition is $W$9. The scheme generalizes to incorporate additional parameters (proximal weights, extrapolation) to improve step size bounds. For instance, the improved convergence condition

$A$0

yields $A$1 for $A$2, allowing up to 33% larger product of step sizes and substantially fewer iterations in practice (He et al., 2021, Li et al., 2022, Malitsky et al., 2016).

4. Structure-Exploiting Heuristics and Large-Scale Computation

The standard canonical primal-dual bounds consider worst-case spectral properties, but for structured problems (e.g., assignment problems where $A$3 has much larger maximal than average eigenvalue), it is effective in practice to relax the step-size constraint to depend on the average spectral radius: $A$4 instead of $A$5. Empirical results show that in assignment problems, this enables speed-ups of up to $A$6–$A$7 over conservative, globally valid step-size rules, while preserving convergence and practical optimality (He et al., 2021).

Practical guidelines include:

• In absence of special structure, set $A$8 and pick $A$9 such that $f$0.
• If $f$1 admits fast factorization, increase $f$2 (or $f$3) toward this limit for maximal step-size.
• For problems with favorable average spectrum, further enlarge $f$4 toward $f$5, monitoring primal-dual residuals for stability.
• Always monitor the primal-dual gap or residual norm and terminate upon reaching a prescribed tolerance (He et al., 2021).

5. Algorithms for Nonconvex Global Optimization

Canonical duality enables the transformation of nonconvex polynomial optimization problems into concave maximization problems over a convex domain (no duality gap). The dual feasible region often admits an equivalence to an SDP via Schur complements, but canonical dual algorithms—for instance, unconstrained maximization with penalized positive semidefiniteness—can provide superior performance and scalability compared to general-purpose SDP solvers.

The main algorithmic strategies comprise:

• Coupled stationarity solves (Newton or quasi-Newton systems over primal and dual variables),
• Dual-only criticality iterations,
• Unconstrained dual maximization with penalization of constraint violations,
• Direct unconstrained primal minimization initialized by dual-feasible points.

These strategies are demonstrated, e.g., on quartic polynomial benchmarks and large-dimensional nonconvex functions, consistently delivering global optimizers in seconds with full reproduction of optimal values, even when standard SDP relaxations fail or return only approximately feasible solutions (zhou, 2012, Wu et al., 2012).

6. Interior-Point and Potential-Reduction Methods

For large-scale nonconvex instances, the canonical primal-dual saddle systems can be solved using interior-point potential reduction approaches. KKT conditions corresponding to saddle points of the total complementarity function $f$6 are encoded into a system of nonlinear equations, augmented by slack variables and multipliers to maintain strict feasibility. The algorithm proceeds by damped Newton steps minimizing a barrier-like potential composed with the KKT residual, with global convergence established under mild assumptions ensuring the search direction is always a descent and iterate sequence remains bounded (Latorre, 2014).

This framework has enabled globally convergent and highly accurate solvers for challenging applications such as large-scale sensor network localization, where it outperforms contemporary SDP-based relaxations in both accuracy and efficiency.

7. Connections to Classical and Modern Primal-Dual Algorithms

The canonical primal-dual method provides the theoretical foundation for a broad range of classical and modern primal-dual algorithms, unifying:

• Chambolle-Pock primal-dual splitting (proximal gradient and extragradient-based schemes),
• ADMM and augmented Lagrangian methods as special cases in the excessive-gap and smoothing frameworks,
• Decomposition-based and block-separable methods via appropriate proximal and smoothing modifications,
• Potential-reduction and interior-point methods for NP-hard nonconvex optimization (He et al., 2021, Li et al., 2022, Tran-Dinh et al., 2014).

This demonstrates the universality of the canonical formulation: various accelerated, adaptive, and structure-exploiting generalizations all operate as instantiations or extensions of the groundwork provided by the canonical saddle-point perspective.

---

References:

(He et al., 2021, zhou, 2012, Wu et al., 2012, Li et al., 2022, Tran-Dinh et al., 2014, Malitsky et al., 2016, Latorre, 2014)