Proximal Augmented Lagrangian Method (PALM)

• Proximal Augmented Lagrangian Method (PALM) is a framework that stabilizes augmented Lagrangians using an explicit proximal term to improve subproblem conditioning in both convex and nonconvex optimization.
• It leverages tools like the Moreau envelope and Bregman regularization to transform nonsmooth or weakly convex problems into smooth, strongly convex subproblems with provable convergence guarantees.
• PALM is versatile, finding applications in decentralized consensus, quadratic programming, and complex nonlinear optimization while offering both global nonasymptotic and local convergence analyses.

The proximal augmented Lagrangian method (PALM) is a family of constrained-optimization methods in which an augmented Lagrangian is stabilized by an explicit proximal regularization term. In Euclidean settings, PALM is commonly interpreted as a proximal point algorithm applied to a dual or KKT operator; in composite formulations it can also be obtained by eliminating a nonsmooth block through a proximal map and the Moreau envelope, which yields a continuously differentiable reduced augmented Lagrangian. Across recent literature, PALM appears in convex linearly constrained optimization, nonconvex equality- and inequality-constrained problems, weakly convex programming, decentralized consensus, and non-Euclidean Bregman path-following schemes (Adeoye et al., 2 Sep 2025, Dhingra et al., 2016, Laude, 17 Feb 2026).

1. Foundational formulation

A canonical PALM model augments the Lagrangian and then adds a quadratic regularizer centered at a reference point. For nonlinear equality- and inequality-constrained problems with a smooth part $f_1$, a convex proximable part $f_2$, equality constraints $h(x)=0$, and inequality constraints $g(x)\le 0$, the proximal augmented Lagrangian used in one recent formulation is

$$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$

with $\rho>0$, $\nu>0$, and proximal parameter $\gamma>0$. The proximal term $\frac{1}{2\gamma}\|x-v\|^2$ is introduced precisely to stabilize the subproblem, improve conditioning, and, for sufficiently small $\gamma$, make the subproblem strongly convex even when $f_2$0 is nonconvex (Adeoye et al., 2 Sep 2025).

For nonsmooth composite optimization of the form

$f_2$1

another construction introduces an auxiliary variable $f_2$2, minimizes the standard augmented Lagrangian over $f_2$3, and substitutes

$f_2$4

The resulting proximal augmented Lagrangian is

$f_2$5

where $f_2$6 is the Moreau envelope of $f_2$7. This replacement turns the reduced function into a continuously differentiable object in both primal and dual variables, even though $f_2$8 itself may be nondifferentiable (Dhingra et al., 2016).

A broader abstraction replaces Euclidean proximal regularization by Bregman regularization. With a Legendre function $f_2$9 and KKT operator $h(x)=0$0, the outer step becomes

$h(x)=0$1

In that view, classical Rockafellar PALM is the Euclidean special case, whereas non-Euclidean choices of $h(x)=0$2 produce Bregman PALM variants that encompass exponential-multiplier and interior-point proximal augmented Lagrangian schemes (Laude, 17 Feb 2026).

2. Problem classes and algorithmic archetypes

PALM is not tied to a single problem class. One line of work treats broad nonlinear programs with smooth equality and inequality constraints and a convex nonsmooth term $h(x)=0$3 with easy proximal mapping. The PALM update there is an inexact minimization in $h(x)=0$4 of the proximal augmented Lagrangian, followed by classical multiplier updates

$h(x)=0$5

and adaptive updates of both penalties and the proximal parameter (Adeoye et al., 2 Sep 2025).

A second archetype is the weakly convex inequality-constrained method QPALM, designed for

$h(x)=0$6

with all functions smooth and weakly convex. At iteration $h(x)=0$7, QPALM builds quadratic models

$h(x)=0$8

forms the augmented Lagrangian of the quadratic model,

$h(x)=0$9

and computes

$g(x)\le 0$0

The paper explicitly characterizes this as a PALM variant based on quadratic approximations and as a “sequentially strongly convex programming” method (Zhang et al., 5 May 2026).

In convex linearly constrained optimization, PALM can take a linearized single-block form. For

$g(x)\le 0$1

with $g(x)\le 0$2 convex and differentiable with Lipschitz gradient and $g(x)\le 0$3 convex prox-friendly, the PALM step replaces $g(x)\le 0$4 by its quadratic upper model and solves an augmented Lagrangian subproblem regularized by $g(x)\le 0$5. The accelerated version, Fast PALM, introduces Nesterov-type extrapolation and achieves a sharper rate in that convex setting (Lu et al., 2015).

A decentralized archetype arises in consensus optimization over a network. With the consensus constraint $g(x)\le 0$6, the PALM subproblem is

$g(x)\le 0$7

and D-ripALM solves it only approximately inside a double-loop framework, with the inexactness controlled by a relative criterion tied to the current feasibility residual and proximal displacement (Zhu et al., 6 Feb 2026).

Taken together, these formulations indicate that PALM is best understood as a design pattern: an augmented Lagrangian coupled with explicit proximal regularization, rather than a single immutable update rule.

3. Regularization mechanisms and surrogate models

The defining regularization mechanism in PALM is the proximal term, but the literature combines it with several distinct smoothing and modeling devices.

In nonsmooth composite optimization, the decisive device is the Moreau envelope. Eliminating the nonsmooth block produces a differentiable reduced augmented Lagrangian, and its gradients can be written in terms of the proximal operator of $g(x)\le 0$8. For instance, the Moreau-envelope identity

$g(x)\le 0$9

gives explicit formulas for the primal and dual gradients of the reduced PAL function (Dhingra et al., 2016). This mechanism explains why PALM can handle nondifferentiable convex regularizers while still enabling smooth optimization tools such as line search, quasi-Newton updates, and primal-dual gradient flows.

In weakly convex programming, the decisive device is model curvature. QPALM does not insert the original nonconvex functions directly into the augmented Lagrangian; it replaces both objective and constraints by quadratic models and selects matrices $$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$0 so that the augmented model is convex in $$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$1. In particular, the paper allows

$$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$2

which yields a strongly convex primal subproblem after adding the proximal term (Zhang et al., 5 May 2026). The same work emphasizes that this differs from classical SQP: the subproblems are explicitly forced to be strongly convex, and the analysis targets global non-asymptotic complexity rather than local second-order convergence.

In the Bregman generalization, the decisive device is non-Euclidean geometry. The Bregman divergence

$$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$3

is inserted into the dual regularization, and after marginalization the resulting Bregman augmented Lagrangian becomes a smooth convex-composite function. The paper shows that, thanks to non-Euclidean geometries, the marginal function is generalized self-concordant, placing the inner problem within the regime of Newton’s method (Laude, 17 Feb 2026). This suggests that PALM can absorb interior-point and exponential-multiplier behavior without abandoning the proximal-point interpretation.

4. Convergence regimes and stationarity metrics

A recurring misconception is to treat “the” convergence theory of PALM as uniform across settings. The literature instead gives regime-dependent statements tied to the geometry of the problem, the smoothness model, and the inexactness criterion.

For weakly convex inequality-constrained optimization, QPALM analyzes three quantities associated with $$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$4-KKT conditions: the squared norm of the gradient of the Moreau envelope of the Lagrangian, the average constraint violation, and the average complementarity violation. With the choices

$$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$5

the paper proves that all three metrics converge to zero at order $$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$6, and translates this into an outer-iteration requirement $$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$7 for an $$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$8-KKT point (Zhang et al., 5 May 2026).

For nonconvex structured optimization with equality and inequality constraints, the inexact proximal ALM of another study does not give a worst-case iteration bound, but it proves finite $$\begin{aligned} \mathcal{L}_{\rho,\nu,\gamma}(x,\lambda,\mu;v) &= f_1(x) + \langle \lambda, h(x)\rangle + \frac{\rho}{2}\|h(x)\|^2 \ &\quad + \frac{1}{2\nu}\|\max\{0,\nu g(x)+\mu\}\|^2 - \frac{1}{2\nu}\|\mu\|^2 + \frac{1}{2\gamma}\|x-v\|^2, \end{aligned}$$9-KKT termination for any $\rho>0$0, together with vanishing primal residuals and subsequential vanishing of dual residuals. The analysis depends critically on the existence of an initial feasible point and on explicit control of the augmented Lagrangian along the iterates (Adeoye et al., 2 Sep 2025).

For nonconvex conic programming, the local theory is sharper. An inexact PALM with primal proximal regularization

$\rho>0$1

is shown to converge Q-linearly under a conic second-order sufficient condition and calmness of the multiplier mapping, without requiring uniqueness of Lagrange multipliers. If $\rho>0$2, the same framework yields Q-superlinear convergence (Zhang et al., 15 Sep 2025).

In convex linearly constrained settings, the rates are of a different kind. The single-block convex PALM of the accelerated paper has an $\rho>0$3 convergence function bound, while Fast PALM achieves $\rho>0$4 in that model class (Lu et al., 2015). In decentralized consensus optimization, D-ripALM proves global convergence under general convexity, and under an error-bound condition it obtains asymptotic superlinear convergence of the outer ALM iterates (Zhu et al., 6 Feb 2026). The Bregman path-following framework similarly derives fast outer rates from metric subregularity of the KKT operator and couples them with inner Newton complexity estimates (Laude, 17 Feb 2026).

These results are not interchangeable. PALM theory is organized by problem regime: convex versus nonconvex, local versus global analysis, Euclidean versus Bregman geometry, and exact versus relative-type inexactness.

5. Numerical realization and application domains

PALM implementations are defined less by a single inner solver than by a recurring numerical strategy: regularize the augmented Lagrangian subproblem until it is smooth or strongly convex, then solve it aggressively with first-order acceleration or semismooth Newton machinery.

In weakly convex QPALM, the strongly convex primal subproblem is solved approximately by Nesterov’s accelerated projected gradient. The same paper reports experiments on nonconvex QCQP and on nonconvex Neyman–Pearson classification with sigmoid loss on MNIST, CINA, and gisette. In the QCQP tests, a data profile over 20 instances shows QPALM fastest on 19/20 instances, with median CPU time about $\rho>0$5 seconds versus $\rho>0$6 for pALM and $\rho>0$7 seconds for classical ALM; the reported decay of stationarity, feasibility, and complementarity metrics is consistent with the $\rho>0$8 theory (Zhang et al., 5 May 2026).

In the adaptive nonconvex P-ALM, the subproblems are solved by PANOC from the Alpaqa library with L-BFGS search directions, memory size 20, and a maximum of 2000 inner iterations. The same work also proposes a phase-I problem,

$\rho>0$9

when an initial feasible point is unavailable (Adeoye et al., 2 Sep 2025).

For quadratic programming, PALM has been implemented with semismooth Newton directions, exact line search, and factorization updates. In one convex QP solver, the inner function

$\nu>0$0

is minimized by semismooth Newton, and the one-dimensional exact line search reduces to the zero of a monotone, piecewise affine scalar function (Hermans et al., 2019). A related nonconvex QP solver retains the same PALM architecture, solves strongly convex proximal subproblems by semismooth Newton, and interprets the outer loop as inexact proximal point iterations on an extended-real-valued objective (Hermans et al., 2020).

High-dimensional convex QP has motivated two-phase designs. QPPAL uses Phase I to obtain a low- to medium-accuracy point by an sGS-based inexact semi-proximal augmented Lagrangian method, and Phase II to obtain a more accurate solution by a proximal augmented Lagrangian method with semismooth Newton inner solves (Liang et al., 2021). This illustrates a broader implementation lesson: PALM is often paired with warm starts, decomposition, and specialized linear algebra rather than used as a monolithic black box.

Recent applications extend well beyond QP. D-ripALM applies the proximal augmented Lagrangian principle to decentralized consensus convex optimization and reports competitive performance on distributed logistic regression and LASSO, including fewer communication rounds than single-loop PG-EXTRA and NIDS in several high-precision regimes (Zhu et al., 6 Feb 2026). A semismooth Newton-based PALM has also been developed for joint estimation of clustered multiple Gaussian graphical models, where block-diagonal screening reduces the problem to smaller independent subproblems and the proximal ALM subproblems are then solved superlinearly by semismooth Newton (Wang et al., 12 Jun 2026). Another paper develops ripALM as a relative-type inexact pALM and demonstrates competitive efficiency on quadratically regularized optimal transport (Zhu et al., 2024).

6. Relations to adjacent methodologies and current directions

PALM sits at the intersection of augmented Lagrangian theory, proximal-point methods, operator splitting, and, in some variants, SQP or interior-point methodology.

Its relation to classical ALM is direct: PALM augments the Lagrangian exactly as ALM does, but adds a proximal term to the primal or primal-dual subproblem. Several papers explicitly interpret this as a proximal point algorithm on a dual or KKT operator, rather than merely as a heuristic regularization trick (Adeoye et al., 2 Sep 2025, Laude, 17 Feb 2026). This operator-theoretic viewpoint explains why multiplier updates, proximal stabilizers, and residual-based stopping rules recur across otherwise very different algorithms.

Its relation to ADMM is close but not identical. In the convex linearly constrained literature, PALM is described as ALM with a proximal or linearized treatment of the smooth part, and one paper states that PALM can be regarded as a special case of Proximal Linearized ADMM with Parallel Splitting in the single-block setting (Lu et al., 2015). Other work likewise places P-ALM near ADMM, primal-dual hybrid gradient methods, and predictor-corrector proximal multipliers, while stressing that the proximal augmented Lagrangian retains the method-of-multipliers structure and its penalty logic (Adeoye et al., 2 Sep 2025).

Its relation to SQP is clearest in weakly convex QPALM. There, quadratic models of objective and constraints make the method parallel SQP in spirit, but the paper distinguishes them sharply: classical SQP often solves indefinite quadratic programs geared toward local second-order behavior, whereas QPALM forces strong convexity through curvature choices and proximal regularization and is analyzed through global non-asymptotic complexity (Zhang et al., 5 May 2026).

Its relation to mirror and Bregman methods appears in generalized formulations. Incremental aggregated proximal and augmented Lagrangian algorithms for separable convex programs already include nonquadratic regularizations such as entropy and exponential penalties, which the paper interprets as generalized PALM-type constructions in non-Euclidean geometry (Bertsekas, 2015). The later Bregman path-following framework systematizes this viewpoint by showing that exponential multiplier methods and interior-point proximal augmented Lagrangian schemes are special cases of Bregman proximal point iterations on the KKT operator (Laude, 17 Feb 2026).

The current directions recorded in these papers are correspondingly diverse. They include matrix-valued or blockwise proximal parameters, explicit outer-inner complexity estimates, broader composite nonsmooth models, larger-scale applications in optimal control and machine learning, and further exploitation of error bounds or calmness properties to sharpen local rates (Adeoye et al., 2 Sep 2025, Laude, 17 Feb 2026). The cumulative picture is that PALM is no longer a single Euclidean variant of ALM, but a framework in which proximal regularization, augmented Lagrangian structure, and problem-specific geometry are combined to obtain implementable subproblems and analyzable primal-dual dynamics.