---
title: Arrow–Hurwicz Iteration
url: https://www.emergentmind.com/topics/arrow-hurwicz-iteration
type: topic
---

# Arrow–Hurwicz Iteration

Arrow–Hurwicz iteration denotes a class of primal-dual descent-ascent methods for saddle-point problems and constrained optimization. In its canonical form, it evolves a primal variable by descent in the Lagrangian and a dual variable by ascent in the constraint residual; in continuous time this gives a differential system, while in discrete time it gives stationary splitting, projection, proximal, or semi-implicit iterations. Recent work treats Arrow–Hurwicz both as a classical method in its own right and as a structural template underlying distributed least-squares solvers, incompressible Navier–Stokes iterations, sphere-packing algorithms, and modern primal-dual schemes such as PDHG [2301.00177] [1701.03908] [2203.01534] [2403.11139].

## 1. Canonical saddle-point structure

The basic setting is a convex-concave Lagrangian. In the Hilbert-space formulation for linearly constrained convex minimization,
\[
\inf\{\,f(x)\mid Ax-b=0\,\},
\]
the associated Lagrangian is
\[
L(x,\lambda)=f(x)+\langle \lambda,Ax-b\rangle_Y,
\]
and saddle points satisfy
\[
\nabla f(\bar x)+A^*\bar\lambda=0,\qquad A\bar x-b=0.
\]
The continuous-time Arrow–Hurwicz system is then
\[
\dot x+\nabla f(x)+A^*\lambda=0,\qquad \dot\lambda+b-Ax=0,
\]
that is, primal steepest descent coupled to dual ascent in the feasibility defect [2301.00177].

A more general projected form arises for convex-concave saddle problems
\[
\inf_{u\in C}\sup_{p\in K}L(u,p),\qquad L(u,p)=\langle Au,p\rangle+F(u)-G(p),
\]
with closed convex sets \(C\subset V\) and \(K\subset W\). The discrete Arrow–Hurwicz update is a projected ascent in \(p\) and a projected descent in \(u\), with an extrapolated primal variable in the generalized scheme studied for nonsmooth saddle systems [1712.03888]. In the linear special case
\[
L(u,p)=\langle Au,p\rangle+\langle f,u\rangle-\langle g,p\rangle,
\]
the scheme reduces to a Chambolle–Pock-type update, convergent under
\[
\alpha\beta\|A\|^2<1
\]
[1712.03888].

The same template appears in network optimization. For a distributed least-squares problem with global system
\[
z=Hy,\qquad h_i^\top y=z_i,\quad i=1,\dots,N,
\]
equation \(i\) is assigned to node \(i\), each node stores \(x_i\in\mathbb R^m\), and consensus is enforced by the graph Laplacian \(L\). The distributed problem becomes
\[
\min_x U(x)\qquad \text{s.t.}\qquad (L\otimes I_m)x=0,
\]
with
\[
U(x_1,\dots,x_N)=\sum_{i=1}^N |h_i^\top x_i-z_i|^2.
\]
The resulting continuous-time Arrow–Hurwicz–Uzawa-type flow is
\[
\dot{x}=-(L\otimes I_m)v-\nabla U(x),\qquad \dot{v}=(L\otimes I_m)x,
\]
so the primal variable is the stack of node states and the dual variable enforces consensus [1701.03908].

## 2. Continuous-time dynamics and asymptotic theory

For the classical differential system, the governing operator
\[
T(x,\lambda)=\big(\nabla f(x)+A^*\lambda,\; b-Ax\big)
\]
is monotone and continuous, hence maximally monotone, and the Cauchy problem is well posed for every initial datum [2301.00177]. A nonexpansiveness property holds: for two solutions \((x,\lambda)\) and \((y,\eta)\),
\[
\frac{d}{dt}\Big(\|x(t)-y(t)\|_X^2+\|\lambda(t)-\eta(t)\|_Y^2\Big)\le 0.
\]
The velocity norm is non-increasing, and if the saddle set is nonempty then trajectories are bounded [2301.00177].

A central identity is the “Lagrangian identity”
\[
\frac{d}{dt}L(x(t),\lambda(t))+\|\dot x(t)\|_X^2=\|\dot\lambda(t)\|_Y^2.
\]
More generally, for any saddle point \((\xi,\eta)\),
\[
\frac{d}{dt}\frac12\big(\|x(t)-\xi\|_X^2+\|\lambda(t)-\eta\|_Y^2\big)+L(x(t),\eta)-L(\xi,\lambda(t))\le 0.
\]
This yields boundedness, integrability of the primal-dual gap, and existence of limits of distances to saddle points [2301.00177].

In the merely convex case, the Cesàro averages
\[
(\sigma(t),\omega(t))=\frac1t\int_0^t(x(\tau),\lambda(\tau))\,d\tau
\]
satisfy
\[
L(\sigma(t),\eta)-L(\xi,\omega(t))=O(1/t),
\]
and converge weakly to a saddle point. Under strong convexity, the actual trajectory satisfies
\[
L(x(t),\eta)-L(\xi,\lambda(t))=o(t^{-1/2}),\qquad \|(\dot x(t),\dot\lambda(t))\|=o(t^{-1/2}),
\]
with weak convergence of \((x(t),\lambda(t))\) to a saddle point [2301.00177]. If \(A\) or \(A^*\) is bounded below, this strengthens to strong convergence and improved decay such as
\[
L(x(t),\eta)-L(\xi,\lambda(t))=o(t^{-1}).
\]
When \(f\) is \(C^2\), \(\nabla^2 f\) is bounded, and either \(A\) or \(A^*\) is bounded below, the paper derives exponential-type decay, including the underdamped, critically damped, and overdamped regimes familiar from damped harmonic oscillators [2301.00177].

A regularized variant augments the flow by vanishing Tikhonov terms:
\[
\dot{x}+\nabla f(x)+A^{\ast}\lambda+\varepsilon(t)x=0,\qquad
\dot{\lambda}+b-Ax+\varepsilon(t)\lambda=0.
\]
Under \(C^2\) assumptions on \(\varepsilon\), the differential inequalities
\[
\varepsilon^2(t)+\dot\varepsilon(t)\ge0,\qquad 2\varepsilon(t)\dot\varepsilon(t)+\ddot\varepsilon(t)\le0,
\]
and nonemptiness of the saddle set, solutions converge strongly to the minimum-norm saddle point
\[
(x(t),\lambda(t))\to \operatorname{proj}_{S\times M}(0,0).
\]
If, in addition,
\[
\|T(z)-T(z')\|^2\ge \alpha\|z-z'\|^2,
\]
then the zero is unique and stronger distance estimates follow [2411.17656]. For \(\varepsilon(t)=t^{-p}\), \(p\in(0,1]\), the framework yields explicit decay bounds, and for \(p=1\) the key residual and velocity estimates simplify to \(O(t^{-2})\) rates for the quantities controlled by the theorem [2411.17656].

## 3. Discrete iterations, projections, and step-size restrictions

The explicit projected scheme for
\[
L(u,p)=\langle Au,p\rangle+F(u)-G(p)
\]
updates \(p\) by
\[
p_{n+1}=\Pi_K\big(p_n+\alpha(Au_n-G'(p_n))\big)
\]
and \(u\) by
\[
u_{n+1}=\Pi_C\big(u_n-\beta(A^*p_{n+1}+F'(u_n))\big),
\]
followed by primal extrapolation. Convergence is proved under
\[
0<\alpha<\frac{2}{L_g},\qquad 0<\beta<\frac{2}{L_f},
\]
and
\[
\alpha \beta \left(\lVert A \rVert^2 - \frac{L_f L_g}{4} \right)+ \frac{\alpha L_g}{2} + \frac{\beta L_f}{2} < 1.
\]
These conditions explicitly couple the Lipschitz moduli of \(F'\) and \(G'\), the operator norm \(\|A\|\), and the product \(\alpha\beta\) [1712.03888].

A semi-implicit variant replaces the descent direction by the filtered term
\[
(A^*A+I)^{-1}(A^*p_{n+1}+F'(u_n)),
\]
so that the primal update becomes
\[
u_{n+1}=\Pi_C\Big(u_n-\beta (A^*A + I)^{-1}(A^* p_{n+1} +F'(u_n))\Big).
\]
The corresponding convergence condition is
\[
0<\alpha <\frac{2}{L_g},\qquad 0 <\beta < \frac{2}{L_f},\qquad \alpha \beta + \frac{\alpha L_g}{2} < 1,
\]
notably without any dependence on \(\|A\|\). The paper identifies this removal of \(\|A\|\) from the admissible parameter range as the key robustness advantage of the semi-implicit scheme [1712.03888].

For distributed least squares over a fixed connected graph, Euler discretization of the Arrow–Hurwicz–Uzawa flow yields
\[
x(k+1)=x(k)-\epsilon(L\otimes I_m)v(k)-\epsilon\nabla U(x(k)),
\]
\[
v(k+1)=v(k)+\epsilon(L\otimes I_m)x(k).
\]
In shifted stacked form,
\[
\begin{bmatrix}\hat x(k+1)\\ \hat v(k+1)\end{bmatrix}=(I+\epsilon M)\begin{bmatrix}\hat x(k)\\ \hat v(k)\end{bmatrix}.
\]
There exists a critical step size
\[
\epsilon^\ast=\min_{\lambda\in \sigma^\ast(M)}\left[-\frac{2\Re(\lambda)}{|\lambda|^2}\right]
\]
such that \(0<\epsilon<\epsilon^\ast\) gives exponential convergence of all node states to the least-squares solution, whereas \(\epsilon>\epsilon^\ast\) yields divergence for some initial conditions. If the graph/equation spanning condition fails, divergence can occur for any \(\epsilon>0\) [1701.03908].

## 4. Oscillation, nonconvergence, and stabilization

A recurrent theme in the literature is that the classical Arrow–Hurwicz mechanism can be only marginally stable. In the distributed least-squares flow, the shifted dynamics are linear,
\[
\dot{\hat x}=-(L\otimes I_m)\hat v-\tilde H\hat x,\qquad \dot{\hat v}=(L\otimes I_m)\hat x,
\]
with Lyapunov function
\[
V(\hat x,\hat v)=\tfrac12(\|\hat x\|^2+\|\hat v\|^2),
\qquad \dot V=-\hat x^\top \tilde H\hat x\le 0.
\]
The flow converges to consensus least squares if and only if for every Laplacian eigenvector \(\alpha\),
\[
\operatorname{span}\{h_i:i\in\mathcal I_\alpha\}=\mathbb R^m.
\]
If this spanning condition fails, the system matrix acquires a purely imaginary eigenvalue and oscillatory solutions occur [1701.03908].

A similar phenomenon appears in the proximal Arrow–Hurwicz discretization of
\[
\min_x f(x)+g(Fx)=\min_x\max_y \Phi(x,y),
\]
where the primal step is followed by a dual step using \(x_{k+1}\). For the example
\[
\Phi(x,y)=x-xy+y,
\]
the iteration becomes
\[
\frac{x_{k+1}-x_k}{s}=y_k-1,\qquad \frac{y_{k+1}-y_k}{s}=-x_{k+1}+1,
\]
which is the symplectic-Euler discretization of
\[
\dot X=Y-1,\qquad \dot Y=-X+1.
\]
The associated Hamiltonian
\[
H(x,y)=\frac12(x^2+y^2)-x-y
\]
is conserved by the continuous flow, so trajectories lie on closed level sets rather than converging to the saddle. The paper uses this to explain the periodic behavior and six-point closed loop observed for proximal Arrow–Hurwicz [2403.11139].

In sphere packing, classical Arrow–Hurwicz is likewise reported to be too oscillatory. For the smooth two-sphere one-dimensional constraint
\[
\phi(X)=d^2-|X|^2,
\]
the linearized algorithm has characteristic polynomial
\[
P(\lambda)=\lambda^2+4d^2\alpha\beta,
\]
so the eigenvalues are purely imaginary and the equilibrium is a center. The damped multi-step variant instead yields a cubic polynomial
\[
P(\lambda)=\lambda^3+c\lambda^2+2\alpha\beta d^2\lambda+4d^2\beta\alpha^2,
\]
with sufficient stability condition
\[
c-2\alpha>0.
\]
For the nonsmooth constraint \(\phi(X)=d-|X|\), the undamped equilibrium is already asymptotically stable in the two-sphere one-dimensional case, with fastest convergence at \(\alpha=4\beta\), while the damped version is stable under
\[
(\alpha+\beta d)c-\beta\alpha>0
\]
[1605.05473].

Across these settings, damping, implicit filtering, and vanishing regularization recur as remedies for purely imaginary spectrum, periodic orbits, and slow decay [1605.05473] [1712.03888] [2411.17656].

## 5. Major applications and specialized formulations

Arrow–Hurwicz iteration is best understood as a reusable saddle-point mechanism whose concrete form depends on the objective, the constraint operator, and the discretization.

| Setting | Arrow–Hurwicz formulation | Reported result |
|---|---|---|
| Linearly constrained convex minimization | Continuous primal-dual flow \(\dot x+\nabla f(x)+A^*\lambda=0\), \(\dot\lambda+b-Ax=0\) | \(O(1/t)\) ergodic gap decay; stronger rates under curvature [2301.00177] |
| Network least squares | AHU flow with consensus constraint \((L\otimes I_m)x=0\) | Necessary and sufficient graph-dependent convergence criterion [1701.03908] |
| Steady Navier–Stokes FEM | Decoupled velocity solve plus pressure correction | Grad-div and Anderson acceleration improve convergence [2203.01534] |
| Divergence-free VEM Navier–Stokes | A-H iteration on nonlinear saddle system from VEM discretization | Geometric convergence with mesh-independent contraction factor [2507.12036] |
| Sphere packing | Damped, second-order, multi-step Arrow–Hurwicz | Better stability than classical AHA in tested cases [1605.05473] |

In incompressible steady Navier–Stokes discretizations, the attraction of Arrow–Hurwicz is the decoupling of velocity and pressure. A typical finite-element form computes \(u^{m+1}\) from a convection-diffusion-type problem,
\[
\frac{1}{\rho}(\nabla(u^{m+1}-u^m),\nabla v) + \nu(\nabla u^m,\nabla v) + b(u^m;u^{m+1},v) - (p^m,\nabla\cdot v) = (f,v),
\]
followed by a pressure correction
\[
\alpha(p^{m+1}-p^m,q)+\rho(\nabla\cdot u^{m+1},q)=0.
\]
Grad-div stabilization adds
\[
\gamma(\nabla\cdot u^{m+1},\nabla\cdot v)
\]
to the momentum equation, improving incompressibility and nonlinear convergence. Under \(\nabla\cdot X_h=Q_h\) and
\[
\rho=\nu^{-1},\qquad \alpha=\frac{\epsilon}{\nu},\qquad \gamma=\epsilon^{-1},
\]
the method becomes equivalent to the iterated Picard penalty method, with a linear convergence rate bounded by the known IPP rate under
\[
\kappa<1,\qquad \epsilon < \frac{\nu\beta^2}{M^2}
\]
[2203.01534].

The numerical consequences are substantial. For the \(Re=100\) lid-driven cavity, earlier Arrow–Hurwicz results reported around \(731\) iterations with Taylor–Hood elements, whereas the stabilized Scott–Vogelius version converged in about \(80\) iterations under good parameters. The same study reports successful computations at \(Re=5000\) and \(Re=10000\), and finds that the combination of grad-div stabilization and Anderson acceleration is the best-performing variant [2203.01534].

For divergence-free mixed virtual element discretizations of steady incompressible Navier–Stokes, the Arrow–Hurwicz iteration is analyzed directly at the algebraic level. Starting from a discrete Stokes solve, the update reads
\[
\rho^{-1} a_h(u_h^{n+1} - u_h^n,v_h) + \nu a_h(u_h^n,v_h) + \widetilde N_h(u_h^n;u_h^{n+1},v_h) = -b(v_h,p_h^n)+(f_h,v_h),
\]
\[
\alpha(p_h^{n+1}-p_h^n,q_h)-\rho\, b(u_h^{n+1},q_h)=0.
\]
Under the discrete smallness condition \(\widetilde\Lambda<1\) and
\[
\alpha^* |1-\rho \nu| + \rho (\alpha_* \nu)\widetilde{\Lambda} + \frac{\rho^2}{2\alpha} < \alpha_*,
\]
the paper proves
\[
\mathcal{F}  |  {E}^{n+1}_h | ^2_1+ \alpha \|  e^{n+1}_h \| ^2_0 \leq \gamma (\mathcal{F} |  {E}^{n}_h | ^2_1 +\alpha \|  e^n_h \| ^2_0 ),
\qquad 0<\gamma<1,
\]
with \(\gamma\) independent of the mesh size [2507.12036].

## 6. Relation to PDHG, Chambolle–Pock, and modern primal-dual methods

Contemporary work often interprets Arrow–Hurwicz as the baseline member of a broader family of primal-dual algorithms. In the generalized Lasso setting,
\[
\min_{x\in\mathbb R^{d_1}} f(x)+g(Fx)
\]
is rewritten as
\[
\min_x\max_y \Phi(x,y)=f(x)+\langle Fx,y\rangle-g^\star(y).
\]
The proximal Arrow–Hurwicz algorithm performs
\[
x_{k+1}=\arg\min_x\left\{f(x)+\langle Fx,y_k\rangle+\frac{1}{2s}\|x-x_k\|^2\right\},
\]
\[
y_{k+1}=\arg\max_y\left\{-g^\star(y)+\langle Fx_{k+1},y\rangle-\frac{1}{2s}\|y-y_k\|^2\right\}.
\]
PDHG keeps the same primal update but uses the extrapolated point
\[
\overline{x}_{k+1}=x_{k+1}+(x_{k+1}-x_k)
\]
in the dual step. Expanded in increment form,
\[
\frac{x_{k+1}-x_k}{s}-F^\top(y_{k+1}-y_k)=-F^\top y_{k+1}-\nabla f(x_{k+1}),
\]
\[
\frac{y_{k+1}-y_k}{s}-F(x_{k+1}-x_k)=Fx_{k+1}-\nabla g^\star(y_{k+1}).
\]
The terms \(F^\top(y_{k+1}-y_k)\) and \(F(x_{k+1}-x_k)\) are the coupled \(y\)-correction and \(x\)-correction emphasized by the paper [2403.11139].

A high-resolution ODE model for PDHG is
\[
\dot X-sF^\top \dot Y=-F^\top Y-\nabla f(X),\qquad
\dot Y-sF\dot X=FX-\nabla g^\star(Y),
\]
with second-derivative corrections retained in the full derivation. Discrete PDHG is identified as the implicit Euler discretization of this system, whereas proximal Arrow–Hurwicz corresponds to a mixed explicit-implicit Euler discretization of the lower-resolution system obtained by dropping the \(O(s)\) coupling terms [2403.11139]. This gives a precise dynamical explanation for a widely observed algorithmic contrast: Arrow–Hurwicz-type discretizations may preserve periodic structure, while PDHG’s coupled correction acts as a small but essential perturbation that yields convergence.

The resulting rates are correspondingly different. For PDHG, the averaged iterates satisfy an ergodic \(O(1/N)\) bound, and when one component is strongly convex the averaged primal iterates satisfy
\[
\|\overline x_N-x^\star\|^2\le \frac{\text{constant}}{\mu s N}
\]
in the one-step-size setting described by the paper [2403.11139]. The discrete numerical-error Lyapunov quantity is also monotone decreasing, which the paper treats not as a nuisance term but as a dissipative mechanism.

This suggests that Arrow–Hurwicz iteration is less a single fixed algorithm than a foundational primal-dual architecture. Chambolle–Pock extrapolation, PDHG’s coupled correction, semi-implicit filtering, grad-div stabilization, Anderson acceleration, damping, and Tikhonov regularization all modify the same descent-ascent core in order to recover stronger contraction, suppress oscillation, or improve robustness [1712.03888] [2203.01534] [2403.11139] [2411.17656].

Source: https://www.emergentmind.com/topics/arrow-hurwicz-iteration