First-Order Augmented Lagrangian Method

• First-Order Augmented Lagrangian Method is an algorithmic framework for solving constrained minimax problems via gradient and proximal operations.
• It integrates a safeguarded augmented Lagrangian formulation with a two-stage first-order subsolver that exploits strong concavity for acceleration.
• The method achieves improved operation complexity, reducing iterations and facilitating faster convergence in large-scale nonconvex optimization scenarios.

A first-order augmented Lagrangian method is an algorithmic framework for solving constrained minimax optimization problems, particularly those exhibiting nonconvexity in the minimization variable and strong concavity in the maximization variable. The first-order approach utilizes only gradient and proximal operations, eschewing second-order information, which enables scalability to high-dimensional problems and facilitates efficient subproblem solutions by harnessing structure such as strong concavity. Recent advances, notably the method introduced by Z. Lu and S. Mei, establish operation complexity bounds for finding approximately stationary (ε-KKT) solutions in nonconvex–strongly-concave constrained minimax settings, sharply improving upon prior results by an order in ε (Lu et al., 28 Dec 2025).

1. Constrained Nonconvex–Strongly-Concave Minimax Problem

The method targets constrained minimax programs of the form

$$\min_{x\in\mathbb R^n}\;\max_{y\in\mathbb R^m}\;\Bigl\{F(x,y)\!:=f(x,y)+p(x)-q(y)\Bigr\} \quad \text{s.t. } c(x)\le0,\;d(x,y)\le0$$

where $f$ is continuously differentiable, $p$ and $q$ are proper closed convex regularizers (with efficient proximal mappings), $c$ and $d$ are smooth constraint mappings, and the domains of $p,q$ are compact. The minimization in $x$ may be nonconvex, while $f(x,\cdot)$ is assumed $\sigma$-strongly concave.

Assumptions include:

• $f$ is $L_{\nabla f}$-Lipschitz,
• $c$ and $d$ are smooth and Lipschitz; $d_i(x,\cdot)$ convex,
• robust MFCQ and uniform Slater conditions for feasible sets,
• existence of an $O(\sqrt\varepsilon)$-feasible initial point.

2. Safeguarded Augmented Lagrangian Formulation

The core algorithmic step is the construction of a safeguarded augmented Lagrangian: $$\mathcal L_\rho(x,y,\lambda_x,\lambda_y) = F(x,y) + \frac1{2\rho}\Big(\|[\lambda_x+\rho c(x)]_+\|^2-\|\lambda_x\|^2\Big) - \frac1{2\rho}\Big(\|[\lambda_y+\rho d(x,y)]_+\|^2-\|\lambda_y\|^2\Big)$$ where $\lambda_x,\lambda_y$ are dual multipliers for the respective constraints, and $[\,\cdot\,]_+$ denotes componentwise max with zero. The positive quadratic term enforces feasibility for minimization ($c$), the negative counterpart for maximization ($d$).

The outer loop iterates over penalty parameter $\rho_k$ and dual variables, at each stage solving the unconstrained nonconvex–strongly-concave minimax subproblem

$$\min_x\max_y \mathcal L_{\rho_k}(x,y,\lambda_x^k,\lambda_y^k)$$

via a first-order subsolver described below.

3. First-Order Subproblem Solver Leveraging Strong Concavity

Each AL subproblem is solved using an inner two-stage algorithm:

• Proximal-point regularization: Transforms the nonconvex–strongly-concave objective $H(x,y) = h(x,y) + p(x) - q(y)$ into a strongly-convex–strongly-concave variant

$H_k(x,y) = h(x,y) + L_h\|x-x^k\|^2 + p(x) - q(y)$

which is strongly convex in $x$, strongly concave in $y$, and globally smooth.

• Optimal first-order primal-dual method: A variant of accelerated primal-dual schemes (cf. [Kovalev–Gasnikov ’22]) achieves $O(\kappa^{1/2}\epsilon^{-2}\log(1/\epsilon))$ complexity for subproblem stationarity, where $\kappa=L_h/\sigma$.

The inner solver alternates regularization and primal-dual updates until the norm $\|x^{t+1}-x^t\|$ drops below a prescribed tolerance.

4. Algorithm Structure and ε-KKT Characterization

The full algorithm comprises:

• Outer loop: iteratively increases the penalty parameter $\rho_k$ (typically set as $\epsilon_k^{-1}$ where $\epsilon_k$ geometrically decays), updates multipliers, and solves subproblems to progressively higher accuracy.
• Inner loop: performs strongly-convex–strongly-concave minimax optimization as described above.

The output $(x,y,\lambda_x,\lambda_y)$ is defined as an ε-KKT solution if: $$\begin{aligned} &\mathrm{dist}(0,\,\partial_xF(x,y)+\nabla c(x)\lambda_x-\nabla_x d(x,y)\lambda_y)\le\varepsilon \ &\mathrm{dist}(0,\,\partial_yF(x,y)-\nabla_y d(x,y)\lambda_y)\le\varepsilon \ &\|[c(x)]_+\|\le\varepsilon, \quad |\langle\lambda_x,c(x)\rangle|\le\varepsilon \ &\|[d(x,y)]_+\|\le\varepsilon, \quad |\langle\lambda_y,d(x,y)\rangle|\le\varepsilon \end{aligned}$$ which certifies near-stationarity, near-feasibility, and near-complementarity.

5. Complexity Results and Accelerated Guarantees

The main theoretical advance is an improved operation complexity for finding an $\varepsilon$-KKT solution: $O(\varepsilon^{-3.5}\log\varepsilon^{-1})$ fundamental operations (gradients of $f,c,d$ and proximal mappings of $p,q$) under suitable assumptions. This improves the previous best-known complexity by a factor of $\varepsilon^{-0.5}$, enabled by exploiting strong concavity in $y$ for accelerated subproblem solves. Specifically, each outer iteration costs $O(\epsilon_k^{-7/2}\log(1/\epsilon_k))$ operations, and $O(\log(1/\varepsilon))$ outer iterations suffice.

Comparison:

Problem Structure	Best Known Complexity Before	New Complexity	Key Advance
Concave, not strongly-concave	$O(\varepsilon^{-4}\log\varepsilon^{-1})$	$O(\varepsilon^{-3.5}\log\varepsilon^{-1})$	$(-1/2)$ order exploitation of strong concavity

6. Numerical Experiments and Empirical Evidence

Experiments validate theoretical results:

• Unconstrained quadratic minimax (random $A,B,C$): Compared Algorithm 2 (inner solver) vs. Alternating Gradient Projection [Xu–Lan ’23]; both obtain similar objective values, but the new method is about $4\times$ faster for dimensions $n=m=200$, with even larger speedup as $n,m$ increase.
• Constrained quadratic minimax (linear inequalities): Compared full AL method to previous ALM [Lu–Mei ’24]; again matched solution quality, but ran $2$–$3\times$ faster on medium-scale problems ($n\approx200$) due to fewer inner iterations required.

7. Implications and Scope

This first-order augmented Lagrangian method establishes a new benchmark for large-scale nonconvex–strongly-concave minimax optimization under functional constraints. The framework’s modularity permits integration with various proximal mappings and constraint sets, provided regularity and qualification hold. The pivotal role of strong concavity enables strict acceleration, relevant in adversarial machine learning, saddle-point optimization, and robust control.

The approach relies fundamentally on first-order oracles, multiplier updates, and safeguarding of dual variables—ensuring both theoretical and empirical computational superiority over earlier methods in this class (Lu et al., 28 Dec 2025).