Anytime Primal-Dual Framework

• Anytime primal-dual framework is a design principle that ensures each iteration yields a usable primal-dual state via techniques like fixed-stepsize and asynchronous updates.
• It encompasses diverse algorithmic patterns—including distributed consensus, bundle-based augmented Lagrangian, and explicit primal recovery methods—with proven convergence regimes.
• The framework supports both online and asynchronous execution, balancing trade-offs in convergence speed, communication overhead, and computational complexity.

Searching arXiv for the cited primal-dual framework papers to ground the article in the current arXiv record. “Anytime primal-dual framework” denotes a family of primal-dual algorithmic constructions in which intermediate primal-dual states are intended to remain usable before asymptotic convergence, while progress is organized through a Lagrangian, augmented-Lagrangian, smoothed-gap, or saddle-point structure. In the cited literature, the phrase is used for several distinct but related design patterns: fixed-stepsize distributed methods with multiple primal steps per dual update, asynchronous block methods whose progress is measured by completed operations rather than synchronized rounds, single-loop bundle-based augmented Lagrangian schemes that output a primal candidate at every iteration, universal dual-driven methods with explicit primal recovery, online primal-dual updates that act immediately on arrivals, and symmetric-cone schemes that maintain approximate primal and dual certificates throughout execution (Mansoori et al., 2019, Hendrickson et al., 2020, Liao et al., 12 Feb 2025, Yurtsever et al., 2015, Zheng et al., 2024).

1. Scope of the term

Across the literature, “anytime” is not attached to a single update formula. Rather, it refers to a recurring requirement that one can stop after any number of iterations, operations, or arrivals and still retain a meaningful primal-dual state. This suggests that the term functions as a design principle rather than a single algorithmic family.

Interpretation	Mechanism	Representative papers
Tunable outer-iteration work	Multiple primal gradient/consensus steps per dual update	(Mansoori et al., 2019)
Operation-count progress under asynchrony	Updates and communications occur without global iteration barriers	(Hendrickson et al., 2020, Hale et al., 2016, Combettes et al., 2015)
Single-loop primal-dual refinement	Descent and null steps refine an inner approximation without a nested inner solver loop	(Liao et al., 12 Feb 2025)
Explicit primal recovery from dual iterates	Weighted averaging of primal sharp-oracle points or excessive-gap iterates	(Yurtsever et al., 2015, Tran-Dinh et al., 2014)
Online irrevocable response	Primal and dual variables are updated immediately when requests arrive	(Thang, 2017)
Approximate certificate maintenance in conic settings	Witness iterates and averaged dual solutions with stopping logic	(Zheng et al., 2024)

In distributed consensus optimization, the anytime property may mean that the parameter $T$ controls the trade off between performance and execution complexity because the method performs $T$ primal steps per outer iteration while reusing a single gradient evaluation (Mansoori et al., 2019). In totally asynchronous constrained optimization, it means that primal agents and dual agents can operate whenever information is available, with progress expressed in terms of completed operations rather than synchronized clocks (Hendrickson et al., 2020). In bundle-based augmented Lagrangian methods, it means that every iteration yields a concrete primal candidate and that null steps refine the model without invoking a separate inner routine (Liao et al., 12 Feb 2025). In universal primal-dual methods, it means that each iterate already carries explicit objective-residual and feasibility-gap guarantees for a recovered primal point (Yurtsever et al., 2015).

2. Canonical variational and Lagrangian formulations

The framework appears over several standard optimization templates. In distributed consensus form, the problem is

$$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$

where $A$ is the edge-node incidence matrix, and the augmented Lagrangian is

$$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$

with $B\succeq 0$, the same null space as $A$, and graph-compatible sparsity (Mansoori et al., 2019).

For general constrained convex programs of the form

$$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$

a regularized Lagrangian

$$L_\delta(x,\mu)=h(x)+\mu^\top g(x)-\frac{\delta}{2}\|\mu\|^2$$

is used to make the dual side strongly concave and to control asynchronous dual updates (Hendrickson et al., 2020). A closely related cloud-agent model employs the Tikhonov-regularized Lagrangian

$$L_{\alpha,\beta}(x,\mu)=f(x)+\frac{\alpha}{2}\|x\|^2+\mu^T g(x)-\frac{\beta}{2}\|\mu\|^2,$$

which yields a unique regularized saddle point and explicit speed-accuracy tradeoffs (Hale et al., 2016).

For linearly constrained convex problems, augmented Lagrangian formulations of the form

$T$0

support both classical ALM equivalence on the dual side and single-loop bundle approximations on the primal side (Liao et al., 12 Feb 2025). A related conic framework rewrites composite convex inequality-constrained problems in an augmented saddle form $T$1, and for $T$2 the dual restriction $T$3 becomes optional (Zhu et al., 2022).

Other anytime primal-dual formulations start directly from a dual composite problem

$T$4

with primal information recovered from the sharp operator

$T$5

or from a smoothed excessive-gap quantity $T$6 that simultaneously tracks objective residual and feasibility (Yurtsever et al., 2015, Tran-Dinh et al., 2014). In bilinear saddle problems,

$T$7

new proximal terms centered at history-dependent points produce another anytime-style variant (Liu et al., 23 Apr 2025).

A common structural theme is that the primal and dual variables are coupled strongly enough that each update can be interpreted as a meaningful approximate saddle-point step. This suggests that “anytime” behavior is obtained not by abandoning dual structure, but by preserving it while relaxing synchronization, subproblem exactness, or inner-loop completion.

3. Recurrent algorithmic patterns

One prominent pattern is the fixed-stepsize distributed method with multiple primal updates per dual update. The MCPD framework computes $T$8 once, performs $T$9 primal inner updates, and then updates the dual variables once. Its compact form is

$$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$0

with

$$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$1

The key feature is that increasing $$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$2 improves primal accuracy per outer iteration while increasing communication and computation per outer loop (Mansoori et al., 2019).

A second pattern is single-loop augmented Lagrangian refinement. BALA replaces the expensive exact ALM subproblem over $$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$3 with an exact solve over a simpler inner approximation $$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$4,

$$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$5

forms

$$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$6

and accepts the step only if

$$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$7

Otherwise, it takes a null step, keeps the outer iterates fixed, and still updates the model. Since $$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$8 can be as simple as $$\min_x f(x)=\sum_{i=1}^n f_i(x_i)\quad \text{s.t.}\quad Ax=0,$$9, the method uses null steps themselves as the approximation-refinement mechanism rather than a separate inner solver loop (Liao et al., 12 Feb 2025).

A third pattern is explicit primal recovery from dual iterations. The universal primal-dual framework performs a dual proximal-gradient step

$A$0

while constructing the primal estimate by weighted averaging,

$A$1

with

$A$2

The method is universal because backtracking adapts automatically to unknown Hölder smoothness of the dual smooth part $A$3, and it avoids the proximity operator of the primal objective (Yurtsever et al., 2015).

A fourth pattern is smoothing-based excessive-gap iteration. The 2P1D and 1P2D schemes maintain primal-dual iterates $A$4 together with smoothing parameters $A$5, and enforce a contraction of the smoothed gap

$A$6

Through different smoothers and center-point choices, this framework subsumes decomposition algorithms, augmented Lagrangian methods, and ADMM-like schemes (Tran-Dinh et al., 2014).

A fifth pattern is to modify the proximal centers themselves. NPDA introduces recursively averaged points

$A$7

and middle points

$A$8

then centers the primal and dual proximal subproblems at $A$9 and $$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$0 instead of the latest iterates. A line-search variant removes the need to know $$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$1 in advance (Liu et al., 23 Apr 2025).

The term also appears in direct inequality handling without slack variables. An augmented Lagrangian

$$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$2

supports both a standard “central” dual update and an “any-time” non-central update that advances dual variables before the current augmented Lagrangian has been minimized exactly. The paper interprets this as a parallel primal-dual algorithm analogous in spirit to a primal-dual Newton method (Toussaint, 2014).

4. Convergence regimes and theoretical guarantees

The convergence theory of anytime primal-dual frameworks is heterogeneous. In the exact distributed consensus setting, strong convexity and Lipschitz gradient assumptions together with appropriate constant stepsizes yield global $$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$3-linear convergence in a weighted norm,

$$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$4

and $$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$5 $$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$6-linearly. This is an exact method rather than a neighborhood method, despite using fixed stepsizes (Mansoori et al., 2019).

In the bundle-based ALM setting, the basic guarantees are asymptotic convergence of the dual iterates, primal feasibility, and primal objective value under mild assumptions, together with explicit complexity estimates: $$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$7 iterations for a dual iterate with $$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$8, $$L(x,\lambda)=f(x)+\lambda^\prime Ax+\frac{1}{2}x^\prime Bx,$$9 iterations for a primal iterate with $B\succeq 0$0 and $B\succeq 0$1, and improved $B\succeq 0$2 and $B\succeq 0$3 bounds when $B\succeq 0$4. For averaged descent-step iterates, the primal error is $B\succeq 0$5. Under quadratic growth and quadratic closeness of the model, every iteration eventually becomes a descent step and the dual, feasibility, and objective errors all converge linearly (Liao et al., 12 Feb 2025).

The universal dual-driven framework gives explicit primal recovery rates that adapt to unknown Hölder smoothness. UniPDGrad has complexity

$B\succeq 0$6

while the accelerated version has

$B\succeq 0$7

The earlier excessive-gap framework provides non-ergodic, separate bounds on objective residual and primal feasibility, with $B\succeq 0$8 feasibility for the augmented-Lagrangian smoother and $B\succeq 0$9 behavior for the Bregman smoother, and stronger bounds when $A$0 is strongly convex (Yurtsever et al., 2015, Tran-Dinh et al., 2014).

In bilinear saddle-point problems, NPDA is globally convergent with an ergodic $A$1 primal-dual gap, the accelerated ANPDA attains $A$2-type behavior when one block is strongly convex, and the line-search variant preserves global convergence and the $A$3 ergodic rate without requiring prior knowledge of $A$4 (Liu et al., 23 Apr 2025). A unified augmented-Lagrangian framework for conic inequality constraints preserves $A$5 ergodic convergence, does not require prior knowledge of the magnitude of the optimal Lagrangian multiplier, and yields linear convergence of the last iterate for affine equality constrained problems under a local error bound (Zhu et al., 2022).

Asynchronous frameworks typically replace wall-clock iteration counts by operation-based measures. For fixed dual variable $A$6, the totally asynchronous block method proves max-norm primal contraction

$A$7

with $A$8, and then measures actual progress by $A$9. The dual side contracts only up to explicit asynchrony-dependent error terms, including the term $$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$0, which is identified as the asynchrony penalty (Hendrickson et al., 2020). In the cloud-agent regularized model, primal and dual rate bounds are geometric in the number of completed local cycles, but smaller regularization parameters improve final approximation accuracy at the cost of slower convergence (Hale et al., 2016).

This range of results shows that anytime behavior is compatible with exact linear convergence, sublinear ergodic convergence, linear last-iterate convergence, and operation-count contractions. A plausible implication is that “anytime” specifies how iterates are produced and certified, not a single asymptotic rate class.

5. Asynchrony, heterogeneity, and online execution

A central issue in asynchronous primal-dual design is the status of the dual variable. In the totally asynchronous block framework, four forms of asynchrony are considered: asynchronous primal computations, asynchronous communication of primal variables, asynchronous dual computations, and asynchronous communication of dual variables. The paper proves a counterexample theorem showing that arbitrarily small disagreement in dual vectors can induce arbitrarily large disagreement in primal minimizers, so dual broadcasts must remain globally consistent across primal agents (Hendrickson et al., 2020). The cloud-agent framework reaches the same conclusion: the synchronized dual copy is the lone point of synchrony, and a concrete counterexample shows that allowing agents to use different dual values can produce non-decaying oscillations (Hale et al., 2016).

Asynchronous execution can nevertheless be pushed far when the dual synchronization rule is respected. In monotone-inclusion form, block-iterative primal-dual projective splitting activates only subsets of operators at each iteration, allows bounded lags $$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$1, constructs a separating half-space from the currently available graph points, and then performs either a Fejér projection step or a Haugazeau-type step. Under deterministic block coverage and bounded delays, one method yields weak convergence, while another converges strongly to the best approximation of the initial point to the Kuhn–Tucker set (Combettes et al., 2015).

Federated settings introduce a different form of execution heterogeneity. FedHybrid splits clients into $$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$2, which use gradient-type updates, and $$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$3, which use Newton-type updates, while the server performs the consensus update

$$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$4

The main theorem gives last-iterate $$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$5-linear convergence,

$$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$6

regardless of which clients choose gradient-type or Newton-type updates, provided the step-size conditions hold (Niu et al., 2021).

The online setting yields a different meaning of anytime execution. Configuration-LP-based primal-dual algorithms react immediately and irrevocably to arriving requests. In the general resource-cost model, the greedy rule chooses the strategy minimizing the marginal cost increase

$$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$7

and competitiveness is certified by $$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$8-smoothness via the ratio $$\min_{x\in X}\ h(x)\quad \text{s.t.}\quad g(x)\le 0,$$9. For online covering with arbitrary monotone objectives, a multilinear extension and local smoothness replace convex duality requirements (Thang, 2017).

6. Unification, limitations, and formalization

A notable property of the area is unification. The distributed exact framework identifies EXTRA and DIGing as special cases with one primal step per iteration and particular choices of $$L_\delta(x,\mu)=h(x)+\mu^\top g(x)-\frac{\delta}{2}\|\mu\|^2$$0 and $$L_\delta(x,\mu)=h(x)+\mu^\top g(x)-\frac{\delta}{2}\|\mu\|^2$$1 (Mansoori et al., 2019). The conic inequality framework recovers PDHG, Chambolle–Pock, GDA, OGDA, linearized ALM, and introduces SOGDA through parameter choices in a single template (Zhu et al., 2022). The excessive-gap framework explicitly subsumes decomposition algorithms, augmented Lagrangian methods, and ADMM-like schemes through smoother and center-point selection (Tran-Dinh et al., 2014). The universal dual-driven method positions itself against generalized conditional gradient methods by using the sharp operator without requiring differentiability of the primal objective and while handling linear inclusion constraints (Yurtsever et al., 2015). The symmetric-cone framework extends the Arora–Kale SDP paradigm from PSD cones to general symmetric cones through symmetric-cone multiplicative weights updates (Zheng et al., 2024).

The limitations are equally structural. Exact linear convergence in distributed consensus requires $$L_\delta(x,\mu)=h(x)+\mu^\top g(x)-\frac{\delta}{2}\|\mu\|^2$$2-strongly convex, twice differentiable, $$L_\delta(x,\mu)=h(x)+\mu^\top g(x)-\frac{\delta}{2}\|\mu\|^2$$3-Lipschitz-gradient local functions and conservative step-size conditions depending on spectral radii and on $$L_\delta(x,\mu)=h(x)+\mu^\top g(x)-\frac{\delta}{2}\|\mu\|^2$$4 (Mansoori et al., 2019). Totally asynchronous contraction requires $$L_\delta(x,\mu)=h(x)+\mu^\top g(x)-\frac{\delta}{2}\|\mu\|^2$$5-diagonal dominance of the primal Hessian and forbids inconsistent dual communication (Hendrickson et al., 2020). The cloud-agent model attains robustness through regularization, but this also creates a speed-accuracy tradeoff because the regularized saddle point is not exactly the original one (Hale et al., 2016). The inequality-based non-central update relies on moderate stability of constraint activity; if active sets change, the method may destabilize, which motivates smoothing, damping, and adaptive step-size safeguards (Toussaint, 2014). The bundle-based ALM obtains linear convergence only under stronger regularity such as quadratic growth and quadratic closeness of the bundle model (Liao et al., 12 Feb 2025).

These constraints clarify two common misconceptions. First, anytime primal-dual methods are not automatically exact; some are exact and linearly convergent, while others converge only to a neighborhood because of asynchrony penalties or regularization (Mansoori et al., 2019, Hendrickson et al., 2020, Hale et al., 2016). Second, anytime does not mean unconstrained asynchrony; multiple papers show that globally consistent dual information is indispensable in asynchronous architectures (Hendrickson et al., 2020, Hale et al., 2016).

A separate line of work addresses the analysis itself. An Isabelle/HOL framework formalizes matrix-based LP duality, weak duality, complementary slackness, scaled complementary slackness, and invariant-preserving primal-dual arguments for matching and online matching algorithms. The paper states explicitly that it does not itself develop an anytime primal-dual algorithm in the modern interruptible sense, but it formalizes progressively maintained primal and dual certificates. This suggests a route toward machine-checked anytime analyses in which intermediate primal feasibility, dual feasibility, and slackness conditions are verified as algorithmic invariants (Abdulaziz et al., 22 Apr 2026).