Proximal ADMM for Nonconvex Optimization

• Proximal ADMM is an augmented Lagrangian-based method that adds quadratic regularization to stabilize and ensure convergence in nonconvex optimization problems.
• The algorithm sequentially updates each block using proximal terms, achieving stability and global Lyapunov descent even under weak regularity assumptions.
• It is effectively applied in dynamic settings such as 4D-Var data assimilation and implicit PDE time-stepping, demonstrating robust performance in nonconvex and distributed scenarios.

A proximal ADMM (Alternating Direction Method of Multipliers) is a class of augmented Lagrangian-based operator-splitting schemes in which explicit quadratic (or more general proximal) regularization is added to the subproblems for stabilization, computational tractability, or to achieve convergence under relaxed or nonconvex settings. Proximal ADMM architectures have played a central role across unconstrained, convex, and nonconvex optimization, including settings with multiple variable blocks, nonlinear constraints, nonsmooth components, and distributed or stochastic architectures.

1. Core Algorithmic Structure and Principles

The proximal ADMM modifies the classical ADMM by introducing (often variable) blockwise proximal terms to the sequential minimization of the augmented Lagrangian. In a general multi-block nonlinear and nonconvex setting such as

$$\min f(x) = \sum_{i=0}^n f_i(x_i) \quad \text{subject to}\quad x_{i+1} = \phi(x_i),\quad i=0,\ldots,n-1$$

with $f_i$ and $\phi$ (possibly nonconvex and nonlinear), the proximal ADMM scheme updates each block by solving

$$x_i^{k+1} = \operatorname{argmin}_{x_i} \left\{ \mathcal{L}_\rho(x_{<i}^{k+1}, x_i, x_{>i}^k, \lambda^k) + \frac{1}{2\eta_i}\|x_i - x_i^k\|^2 \right\}$$

and then executes blockwise multiplier (dual) ascent, typically via

$$\lambda_j^{k+1} = \lambda_j^k + \rho_j (x_{j+1}^{k+1} - \phi(x_j^{k+1}))$$

Crucially, the choice of blockwise proximal regularization $\frac{1}{2\eta_i}\|x_i - x_i^k\|^2$ is essential for restoring stability, ensuring a Lyapunov-type descent, and compensating for the lack of contractivity/multiblock divergence common in direct ADMM generalizations (Li et al., 20 Jun 2025).

2. Global and Local Convergence Guarantees

Proximal ADMMs are analyzed via Lyapunov or potential-function frameworks to obtain convergence to critical/KKT points, even in settings with nonconvex objectives or nonlinear constraints. For the nonconvex chain-constrained case (Li et al., 20 Jun 2025), under local Lipschitz and $L$-smooth assumptions on the component gradients, and a bounded level-set assumption, it is shown that:

• The sequence of iterates remains bounded, with accumulation points corresponding to first-order KKT points of the problem.
• The Lyapunov function

$$E(k) = \mathcal{L}_\rho(x^k,\lambda^k) + \sum_{i=0}^n \frac{1}{4\eta_i}\|x_i^k - x_i^{k-1}\|^2$$

is monotonically decreasing and bounded below, guaranteeing sufficient decrease.

• All differences $\|x^{k+1} - x^k\|$, $\|\lambda^{k+1}-\lambda^k\|$ vanish in the limit, and there exists a subsequence along which the blockwise residuals decay at rate $f_i$0 (Li et al., 20 Jun 2025). This establishes strong convergence/ergodic rates for the best block.

In convex settings, more refined rates—including nonergodic $f_i$1 and even linear convergence under error-bound or metric subregularity conditions—are available (Jin, 2023). For multi-block and composite scenarios, similar arguments yield global subsequential convergence to critical (KKT) points of the penalized Lagrangian (Yuan, 2024, Yang et al., 2022, Kong et al., 2021).

3. Parameter Selection, Blockwise Penalties, and Regularity

Achieving descent and convergence in the multiblock nonconvex context is subtle. For the chain-constrained ODE/PDE discretization structure:

• Penalty parameters $f_i$2 and proximal weights $f_i$3 are selected inductively: for any set positive $f_i$4, $f_i$5 are chosen large enough so that

$f_i$6

where $f_i$7 depend on local smoothness/Lipschitz constants and previous penalty choices. This ensures a global descent property holds across blocks.

• Only local $f_i$8-smoothness and local Lipschitz continuity are needed (as opposed to global regularity), provided level-sets of the penalized objective are bounded.
• Proximal weights $f_i$9 control the update size per block; larger $\phi$0 increase penalization of movement and stabilize the nonconvex subproblem, but may slow convergence.
• For distributed or composite settings, similar blockwise or variable metric terms are required to adapt to the local curvature and structure (Yuan, 2024, Maia et al., 2024).

4. Applications and Practical Implementation

Proximal ADMM has been applied in settings where chains of nonlinear state or control variables interact:

Example A: 4D-Var Data Assimilation (Lorenz-63)

For reconstructing chaotic dynamical system trajectories from noisy/sparse measurements with time-discrete ODE constraints, proximal ADMM is applied with each block update corresponding to small nonlinear least squares in each $\phi$1, solvable with Levenberg–Marquardt or derivatives-free methods. Gradient-based methods (e.g., adjoint+CG, LBFGS) are shown to often fail in escaping local minima in this nonconvex landscape. In contrast, the proximal ADMM sharpens constraint satisfaction and tracks truth, with final objective and constraint error dropping rapidly after initialization. The method is robust but slower per iteration than quasi-Newton, reflecting a convergence/nonconvexity trade-off (Li et al., 20 Jun 2025).

Example B: Implicit Time-Integration for PDEs

The method is used as a nonlinear solver for implicit time-stepping (e.g., Lax–Friedrichs for viscous Burgers’ equation). In this context, the ODE/PDE constraints are enforced as blockwise equality constraints, and proximal ADMM solves for the state at each time step. The scheme remains stable and convergent, unlike explicit schemes, at large time steps; the error diminishes with grid refinement and the number of time steps (Li et al., 20 Jun 2025).

5. Comparison to Classical, Multiblock, and ADMM Variants

Classical two-block ADMM is convergent for convex $\phi$2 separable problems. However, naive extensions to $\phi$3 blocks or nonlinear (even convex) constraints can lead to instability and divergence. The introduction of per-block or variable metric proximal terms in the update of each block is the critical modification that ensures global Lyapunov descent and convergence under relatively weak assumptions, without requiring strong convexity or blockwise invertibility (Li et al., 20 Jun 2025).

This approach is also distinguished from classical Jacobi/parallel updates, which tend to be less stable in the presence of nonconvexity or nonlinearity; proximal ADMM sequentially and adaptively regularizes each block update.

6. Computational Aspects and Subproblem Solving

Due to the blockwise structure, each subproblem in proximal ADMM is typically a small, well-posed nonlinear optimization or least squares:

• For ODE/PDE-constrained data assimilation, each $\phi$4-update is a small nonlinear least squares problem, efficiently solvable with standard algorithms.
• The method is amenable to parallelization across time or spatial indices depending on the structure.
• The additional proximal term is simple to implement and computationally lightweight compared to the full Hessian or adjoint-based modifications.

In implicit time-stepping contexts, the ability to perform blockwise updates with proximal regularization makes the approach an optimizer-based alternative to Newton or fixed-point iterative solvers for nonlinear systems arising from ODE/PDE discretizations.

7. Impact and Broader Developments

The proximal ADMM paradigm for multiblock, nonconvex, and dynamics-constrained optimization has established provable convergence under relatively weak assumptions, overcoming critical divergence issues plaguing direct generalizations of ADMM. It has demonstrated robust performance in practical high-dimensional, nonconvex domains—especially inverse problems and time-dependent control or estimation. The parameter-selection heuristics, Lyapunov descent formalism, and sequential-proximal architecture developed in (Li et al., 20 Jun 2025) enable practical and theoretically guaranteed optimization even in settings where classical ADMM fails to provide meaningful iterates.

---

Key Reference: "Convergent Proximal Multiblock ADMM for Nonconvex Dynamics-Constrained Optimization" (Li et al., 20 Jun 2025)