Anchored Gradient Descent Ascent

• The paper introduces AGDA, which adds a vanishing anchoring term to simultaneous GDA to stabilize updates in convex-concave saddle-point problems.
• AGDA leverages a fixed initialization anchor to guide both primal and dual variables, yielding concrete convergence rates in smooth and composite settings.
• The method extends to proximal and constrained problems through operator-theoretic formulations, achieving O(1/t) last-iterate decay in smooth cases and O(1/√T) residual bounds in complex settings.

Anchored Gradient Descent Ascent (AGDA) is a stabilization of simultaneous gradient descent-ascent for minimax and saddle-point problems in which each iterate is pulled toward a fixed reference point, typically the initialization. In the smooth unconstrained convex-concave setting, with saddle operator $G(z)=(\nabla_x L(z),-\nabla_y L(z))$ and $z=(x,y)$, the method takes the form

$z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$

or, coordinatewise,

$$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$

The anchor $z_0$ is fixed, the anchoring weight $\beta_t$ vanishes over time, and the method remains a single-gradient-call modification of simultaneous GDA. Modern analyses place AGDA within monotone operator theory, connect it to Halpern-type regularization, and establish last-iterate guarantees that were unavailable for vanilla GDA on classical oscillatory examples such as bilinear games (Ryu et al., 2019, Surina et al., 4 Apr 2026, Cai et al., 14 Apr 2026).

1. Core formulation and problem classes

AGDA is studied primarily for convex-concave saddle-point problems

$$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$

where $L(\cdot,y)$ is convex for every $y$, $L(x,\cdot)$ is concave for every $z=(x,y)$0, and the associated saddle operator

$z=(x,y)$1

is monotone and Lipschitz. In this setting, anchoring adds the restoring term $z=(x,y)$2 to simultaneous GDA, so the primal variable is pulled toward $z=(x,y)$3 and the dual variable toward $z=(x,y)$4. The 2019 formulation explicitly identifies the anchor as the fixed initial iterate and relates the mechanism to Halpern’s method and James–Stein shrinkage (Ryu et al., 2019).

A broader operator-theoretic formulation treats AGDA as a method for the monotone inclusion

$z=(x,y)$5

where $z=(x,y)$6 is single-valued, monotone, and $z=(x,y)$7-Lipschitz, $z=(x,y)$8 is maximally monotone, and the solution set $z=(x,y)$9 is nonempty. This encompasses monotone variational inequalities and convex-concave saddle-point problems with constraints or regularization. The proximal anchored iteration is

$z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$0

which reduces to standard AGDA when $z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$1 (Cai et al., 14 Apr 2026).

For composite saddle problems

$z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$2

with $z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$3 continuously differentiable, convex in $z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$4, concave in $z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$5, and $z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$6 proper closed convex, one sets

$z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$7

The resolvent then separates, yielding proximal anchored gradient descent-ascent: $z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$8

$z_{t+1}=z_t-\alpha_t G(z_t)+\beta_t(z_0-z_t),$9

If $$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$0 and $$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$1, these become projected updates onto closed convex sets $$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$2 and $$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$3 (Cai et al., 14 Apr 2026).

2. Early theory: ODE motivation, deterministic rates, and stochastic convergence

The first AGDA analysis in this corpus appears as “anchored simultaneous gradient descent” in a 2019 ODE-guided treatment of convex-concave minimax dynamics. In the deterministic smooth setting the update is

$$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$4

and in the stochastic subgradient setting the anchored term is

$$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$5

The continuous-time analogue is

$$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$6

with Lyapunov-type quantity

$$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$7

The paper derives the continuous-time decay

$$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$8

and proves in discrete time that, under convex-concavity and smoothness, AGDA satisfies

$$x_{t+1}=x_t-\alpha_t\nabla_x L(x_t,y_t)+\beta_t(x_0-x_t),\qquad y_{t+1}=y_t+\alpha_t\nabla_y L(x_t,y_t)+\beta_t(y_0-y_t).$$9

for $z_0$0, $z_0$1. Its stochastic anchored theorem is stronger in a different sense: under convex-concavity and a second-moment/Lipschitz-type oracle condition, but without strict convexity/concavity and without differentiability, the iterates converge in $z_0$2 to

$z_0$3

the projection of the anchor onto the saddle-point set (Ryu et al., 2019).

This early theory established two points that remained central in later work. First, anchoring regularizes the last iterate rather than only ergodic averages. Second, the anchor does not merely damp motion; it also selects a solution when the saddle set is nonunique, because the fixed point $z_0$4 remains present in the dynamics throughout the run (Ryu et al., 2019).

3. Exact last-iterate rates in smooth convex-concave problems

A 2026 analysis sharpened the AGDA theory for smooth convex-concave minimax problems by resolving an open question left by the 2019 rate. It studies

$z_0$5

under monotonicity of $z_0$6, $z_0$7-Lipschitz continuity, and existence of a saddle point $z_0$8 with $z_0$9. The update remains

$\beta_t$0

but the schedule changes to

$\beta_t$1

Under this schedule the paper proves the exact last-iterate bound

$\beta_t$2

Equivalently,

$\beta_t$3

This closes the gap left by the earlier $\beta_t$4 analysis and shows that the borderline $\beta_t$5 decay for the squared gradient norm is achievable in the actual iterate, not merely in an average (Surina et al., 4 Apr 2026).

The proof is purely discrete-time. Its structure has three parts: a boundedness argument giving

$\beta_t$6

a recurrence for consecutive differences $\beta_t$7 implying

$\beta_t$8

and the reconstruction identity

$\beta_t$9

which converts iterate stability into gradient decay. The result is notable not only for its rate but also for its formal provenance: the paper reports that it was discovered autonomously by an AI system capable of writing formal proofs in Lean, and the natural-language proof is presented as an informalization of the formal development (Surina et al., 4 Apr 2026).

A common misconception is that AGDA only supports suboptimal last-iterate rates. That statement is accurate for the general composite monotone inclusion analysis discussed below, where the primary theorem is an $$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$0 bound on a residual norm, but it is not accurate for the smooth unconstrained convex-concave case: there the 2026 result gives the exact $$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$1 rate for $$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$2 (Surina et al., 4 Apr 2026).

4. Composite, constrained, and operator-theoretic AGDA

The operator-theoretic extension of 2026 studies AGDA in its most general monotone inclusion form and derives a concrete anchored gradient descent-ascent method for constrained and regularized min-max problems. The standing assumptions are that $$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$3 is monotone and $$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$4-Lipschitz, $$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$5 is maximally monotone, and $$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$6. The proposed schedule is

$$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$7

The convergence metric is the tangent residual

$$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$8

which reduces to $$\min_{x\in \mathbb{R}^n}\max_{y\in \mathbb{R}^m} L(x,y),$$9 when $L(\cdot,y)$0. The paper also recalls the natural residual

$L(\cdot,y)$1

and proves

$L(\cdot,y)$2

for every $L(\cdot,y)$3. Consequently, a tangent-residual bound immediately implies a natural-residual bound (Cai et al., 14 Apr 2026).

Its main theorem states that for the iterates generated by

$L(\cdot,y)$4

one has

$L(\cdot,y)$5

where $L(\cdot,y)$6 is induced by the resolvent step,

$L(\cdot,y)$7

Using $L(\cdot,y)$8, the explicit version becomes

$L(\cdot,y)$9

This is a last-iterate guarantee for $y$0 itself, and by residual domination it also yields an $y$1 last-iterate bound for the natural residual (Cai et al., 14 Apr 2026).

The proof introduces auxiliary vectors

$y$2

so that the proximal iteration can be rewritten as

$y$3

This “$y$4-vector trick” makes the composite recursion resemble the unconstrained anchored recursion while preserving the nonsmooth or constrained contribution of $y$5 (Cai et al., 14 Apr 2026).

In saddle-point language, the operator formulation shows that AGDA is not limited to smooth unconstrained min-max. The same fixed-anchor mechanism extends directly to proximal and projected updates, with the resolvent enforcing nonsmooth regularization or feasibility while the anchor continues to regularize both players’ motion (Cai et al., 14 Apr 2026).

5. Position within the stabilized GDA landscape

The modern literature places AGDA alongside several other mechanisms for suppressing the rotational behavior of simultaneous GDA. The comparison is explicit in the 2026 composite monotone-inclusion paper:

Method family	Stabilization mechanism	Stated last-iterate behavior
Vanilla GDA / forward-backward	None	Can diverge or oscillate even on simple bilinear problems
AGDA	Fixed anchor $y$6	$y$7 tangent residual in the composite setting; $y$8 for $y$9 in the smooth unconstrained setting
EG / OGDA	Optimism or lookahead	$L(x,\cdot)$0 last-iterate residual
EAG / FEG and composite variants	Optimism plus anchoring	$L(x,\cdot)$1 last-iterate, stated as optimal

This comparison shows that anchoring alone is sufficient for last-iterate convergence, but not for the fastest rates currently stated in the same generality; the optimal $L(x,\cdot)$2 last-iterate rate is attributed to methods that combine anchoring with optimism or extragradient structure (Cai et al., 14 Apr 2026).

Several nearby methods are anchor-like without being standard AGDA. “Dissipative Gradient Descent Ascent” augments the state with auxiliary variables $L(x,\cdot)$3 and pulls the iterate toward a dynamically updated filtered reference: $L(x,\cdot)$4 and similarly for $L(x,\cdot)$5. The paper is explicit that this is not literal anchoring to a fixed center; the reference evolves as an exponential moving average, and the method is best described as dynamically anchored and dissipative rather than standard AGDA. Its theory gives linear convergence in bilinear and strongly convex-strongly concave settings (Zheng et al., 2024).

A different competitor dispenses with anchors entirely. “Slingshot stepsize schedules” make plain GDA converge by using time-varying, asymmetric, and periodically negative stepsizes. That work presents anchoring as prior art among stabilization devices, but its claim is that convergence can be recovered solely by changing the stepsize schedule, without adding anchoring, optimism, or extragradient corrections (Shugart et al., 2 May 2025).

The continuous-time Newtonian dissipation analysis of gradient descent-ascent is also adjacent rather than direct AGDA theory. It contains no anchoring term, but it explains how antisymmetric coupling can generate rotational behavior and how dissipative structure can restore convergence. This supplies conceptual background for why anchor terms may act as damping-like regularizers even though the paper itself does not analyze AGDA (Seung, 2019).

6. Variants, scope, and limitations

Anchoring has also been adapted beyond the standard smooth convex-concave fixed-anchor setting. A notable example is the “semi-anchored multi-step gradient descent-ascent method” for structured nonconvex-nonconcave composite minimax problems,

$L(x,\cdot)$6

whose saddle-subdifferential operator satisfies a weak Minty variational inequality. SA-MGDA is derived from a Bregman proximal point method and introduces anchoring only on the dual side. The exact dual subproblem is centered at

$L(x,\cdot)$7

so the method is explicitly asymmetric: the ascent variable is tethered to a previous-iterate-based anchor while the descent variable is updated by a proximal gradient step. Under weak MVI it obtains a nonergodic $L(x,\cdot)$8 stationarity guarantee, and under strong MVI it obtains linear convergence in Bregman distance (Lee et al., 2021).

These developments delimit the meaning of AGDA. Fixed-anchor AGDA refers to a method that pulls iterates toward a constant reference point $L(x,\cdot)$9. Dynamic-anchor methods such as DGDA, dual-only semi-anchored methods such as SA-MGDA, and dissipation-based continuous-time analyses are structurally related, but they are not the same algorithmic object (Zheng et al., 2024, Lee et al., 2021, Seung, 2019).

The limitations of standard AGDA are also explicit in the literature. In the general monotone inclusion setting, the main guarantee is for residual convergence—specifically tangent residual, and hence natural residual—rather than a direct primal-dual objective-gap theorem (Cai et al., 14 Apr 2026). The composite note does not claim that $z=(x,y)$00 is optimal; on the contrary, it states that stronger $z=(x,y)$01 rates are known when anchoring is combined with optimism (Cai et al., 14 Apr 2026). The 2019 stochastic anchored theorem requires a small $z=(x,y)$02 in the anchor exponent for the proof, while also stating that this is likely a proof artifact (Ryu et al., 2019). The 2026 exact $z=(x,y)$03 result applies to smooth unconstrained convex-concave problems, not to the full composite inclusion setting (Surina et al., 4 Apr 2026).

Taken together, these results support a precise characterization. AGDA is a single-call, last-iterate-oriented regularization of simultaneous GDA in which a vanishing pull toward a fixed anchor suppresses oscillation without introducing lookahead. In smooth unconstrained convex-concave problems it now has an exact $z=(x,y)$04 last-iterate rate for the squared saddle-gradient norm (Surina et al., 4 Apr 2026). In the general composite monotone inclusion framework it admits a proximal-resolvent generalization with an $z=(x,y)$05 last-iterate residual bound (Cai et al., 14 Apr 2026). Its broader significance lies in showing that anchoring alone, without optimism, is already sufficient to stabilize a large class of minimax dynamics that vanilla simultaneous GDA does not control (Ryu et al., 2019, Cai et al., 14 Apr 2026).