Proximal Descent Method Overview

• Proximal descent methods are a class of first-order optimization algorithms that achieve descent by solving a local, regularized subproblem rather than taking a bare gradient step.
• They employ tools such as proximal-gradient mappings, Bregman divergences, and momentum schemes to certify stationarity and accelerate convergence in both convex and nonconvex settings.
• Their versatility is evident in diverse applications, from signal processing to distributed optimization, with extensions to coordinate, block, and plug-and-play models.

Proximal descent method denotes a family of first-order optimization algorithms in which descent is produced by solving a local regularized subproblem rather than by taking a bare gradient step. In its canonical composite form, it treats problems of the form $\min_x F(x)=f(x)+g(x)$, where $f$ is smooth and $g$ is proper, closed, and convex, but the same paradigm extends to constrained problems, block-coordinate schemes, Bregman geometries, weakly convex objectives, and plug-and-play models with implicit priors (Bok et al., 2024, Elshiaty et al., 4 Jun 2025, Liao et al., 2 Sep 2025).

1. Formal problem classes and core operators

The standard convex-composite formulation is

$\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$

with $f$ convex and $L$-smooth and $g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$ proper, closed, and convex. In this setting the proximal operator is

$$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$

and the proximal gradient descent update is

$$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$

When $g=\iota_C$ is the indicator of a nonempty closed convex set $f$0, the proximal map becomes the Euclidean projection $f$1, so projected gradient descent is recovered as a special case (Bok et al., 2024).

A closely related object is the proximal-gradient mapping

$f$2

It provides a computable stationarity measure, and in the cited analysis it satisfies

$f$3

so $f$4 is equivalent to first-order optimality for the composite problem (Chen et al., 2022).

The same descent logic appears in stronger geometric or implicit regularizations. In Bregman proximal gradient descent, the Euclidean quadratic is replaced by a Bregman divergence

$f$5

and the update takes the form

$f$6

which is tailored to relative Lipschitz smoothness rather than Euclidean $f$7-smoothness (Elshiaty et al., 4 Jun 2025). For weakly convex optimization, the same family is often interpreted through Moreau envelopes and proximal-point regularization. One formulation minimizes an $f$8-weakly convex and possibly nonsmooth function by combining an inexact proximal point method with convex bundle techniques, using $f$9-inexact stationarity as the primary termination notion (Liao et al., 2 Sep 2025).

2. Descent mechanisms and certification

The defining feature of proximal descent is not merely the update formula but the availability of a descent certificate. For composite proximal gradient descent with $g$0, the refined descent identity

$g$1

strengthens the standard one-step decrease bound by adding a next-iterate proximal-gradient term. In the convex case $g$2, this becomes

$g$3

and it is accompanied by proximal-gradient mapping norm monotonicity: $g$4 For $g$5 and $g$6, the norm sequence is nonincreasing (Chen et al., 2022).

A different descent certificate arises in the geometric proximal gradient method. GeoPG extends geometric descent to nonsmooth strongly convex composite problems by maintaining two balls that both contain the optimizer $g$7, then replacing them by the minimum enclosing ball of their intersection. Its central inequality is

$g$8

which yields a ball around

$g$9

containing $\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$0 (Chen et al., 2016).

The proximal distance line of work replaces hard constraints by distance penalties and majorizes the distance term at each iterate. For constrained minimization with loss $\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$1 and constraint set $\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$2, the penalized objective is

$\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$3

and the key majorization is

$\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$4

The resulting surrogate

$\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$5

has minimizer

$\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$6

which guarantees monotone decrease of the penalized loss for fixed $\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$7 through the MM principle (Keys et al., 2016). A related survey emphasizes exact penalty theory, distance majorization, and modular reductions to projections and proximal mappings across convex and nonconvex constrained models (Lange et al., 2015).

3. Acceleration and rate theory

Acceleration in proximal descent has traditionally been associated with momentum-based schemes such as FISTA and Nesterov acceleration. In the composite setting, these methods provide the benchmark rates $\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$8 for smooth convex objectives and $\min_{x\in\mathbb{R}^d} F(x)=f(x)+g(x),$9 under strong convexity, which the cited literature describes as information-theoretically optimal among first-order methods (Bok et al., 2024).

A distinct line of work shows that proximal descent can also be accelerated without momentum. For the silver stepsize schedule,

$f$0

with normalized stepsizes $f$1, proximal gradient descent on horizons $f$2 satisfies the exact smooth-convex bound

$f$3

Equivalently, achieving $f$4 requires

$f$5

and in the strongly convex case the iteration complexity becomes

$f$6

The analysis combines recursive gluing with a Laplacian-structured sum-of-squares certificate (Bok et al., 2024).

GeoPG obtains a different kind of acceleration. For $f$7,

$f$8

and with $f$9 this yields the optimal strongly convex factor

$L$0

so the iteration complexity is

$L$1

GeoPG therefore matches the optimal linear rate among first-order methods for strongly convex composite minimization, but it does so through a geometric shrinking-of-balls mechanism rather than explicit momentum variables (Chen et al., 2016).

Acceleration also appears in coordinate and potential-based formulations. APPROX, the accelerated, parallel, proximal coordinate descent method, is the first method in its paper that is simultaneously accelerated, parallel and proximal. In the special case when the number of processors is equal to the number of coordinates, it converges at rate

$L$2

where $L$3 is an average degree of separability and $L$4 is the average of coordinate-wise Lipschitz constants (Fercoq et al., 2013). In parallel, a potential-function framework for proximal gradient descent shows that one can minimize the norm of the proximal-gradient mapping more rapidly than basic PGD: with a suitable accelerated construction,

$L$5

so $L$6 decays as $L$7 along the best iterate (Chen et al., 2022).

4. Generalizations beyond Euclidean convex composite minimization

The proximal descent paradigm extends to weakly convex and nonconvex settings, but the theory changes substantially. One recent formulation studies minimization of an $L$8-weakly convex and possibly nonsmooth function via an inexact proximal point method solved by convex bundle techniques. It proves that an $L$9-inexact stationary point can be found using at most

$g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$0

function value and subgradient evaluations, and consequently achieves the best-known complexity

$g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$1

for finding an approximate Moreau stationary point with $g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$2. The same method accelerates automatically to $g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$3 under smoothness and attains linear convergence under quadratic growth (Liao et al., 2 Sep 2025).

Under non-Euclidean geometry, multilevel Bregman proximal gradient descent treats constrained convex problems with relative Lipschitz smoothness. Its single-level step is Bregman proximal, while the multilevel extension introduces coarse models, transfer operators, and coarse correction conditions. The paper establishes a global linear convergence rate for multiple coarse levels and proves well-posedness of the multilevel framework (Elshiaty et al., 4 Jun 2025).

Plug-and-play variants reinterpret denoisers as proximal operators of weakly convex potentials. In the relaxed PnP-$g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$4PGD framework, the denoiser is designed so that $g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$5, and convergence with $g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$6 is obtained under

$g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$7

where $g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$8 is the weak convexity modulus induced by the denoiser. A relaxed denoiser $g:\mathbb{R}^d\to\mathbb{R}\cup\{+\infty\}$9 reduces that modulus and therefore widens the admissible range of $$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$0 (Hurault et al., 2023). A block-coordinate plug-and-play variant for hyperspectral anomaly detection uses an orthogonality-constrained nonconvex nonsmooth model and proves that any accumulation point of the PnP-PBCD sequence is a stationary point (Liu et al., 2024).

Proximal descent also generalizes to minimax problems. AGDA+ addresses double-regularized nonconvex-strongly concave minimax problems with alternating proximal descent in the primal block and proximal ascent in the dual block: $$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$1 With nonmonotone adaptive backtracking, AGDA+ achieves the optimal iteration complexity $$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$2 and is the first step-size search method for NCSC minimax problems that requires only $$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$3 calls to $$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$4 on average per backtracking iteration (Zhang et al., 2024).

5. Coordinate, block, operator-splitting, and distributed realizations

Many practically important proximal descent methods are coordinate or block structured. For SLOPE, a hybrid solver alternates full proximal gradient steps with proximal coordinate descent over clusters. The SLOPE proximal operator is computed by sorted soft-thresholding followed by isotonic regression, while the coordinate subproblems use a scalar SLOPE thresholding operator derived from the directional derivative of the sorted $$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$5 penalty (Larsson et al., 2022). For simultaneous filtering and parameter estimation in ODE models, BCD-prox alternates a parameter update and a state update regularized by a quadratic tether

$$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$6

which yields robustness to noise, parameter initialization, and hyperparameters in the reported experiments (Raziperchikolaei et al., 2018).

Asynchronous and parallel proximal descent variants target large-scale settings. DAP-SVRG decouples the proximal computation from the parameter server by letting workers compute

$$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$7

while the server performs only the update $$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$8. Under strong convexity and bounded delay, the method achieves linear convergence (Huo et al., 2016). AAPCD further combines asynchronous proximal coordinate descent with delay-aware momentum and sufficient descent for nonconvex nonsmooth problems, and the cited analysis proves that for both bounded delays and unbounded delays every limit point is a critical point (Kazemi et al., 2019).

Distributed realizations can also be written as proximal descent with consensus. In the decentralized proximal optimization method with consensus procedure, each outer iteration combines an accelerated proximal gradient step with $$\operatorname{prox}_{t g}(y)=\arg\min_x\left\{g(x)+\frac{1}{2t}\|x-y\|^2\right\},$$9 consensus rounds over a time-varying network. Under smoothness, strong convexity, and spectral contraction assumptions on the mixing matrices, the method reaches $$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$0-accuracy with

$$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$1

oracle calls per node and

$$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$2

communication rounds (Rogozin et al., 2023).

A more operator-theoretic strand embeds proximal descent into fixed-point methods. Hierarchical convex optimization by the hybrid steepest descent method uses nonexpansive operators constructed by forward-backward, Douglas–Rachford, or linearized augmented Lagrangian splittings, then performs a secondary gradient step over the fixed-point set to choose a preferred minimizer. This produces hierarchical enhancements of SVM and Lasso/TREX-type models (Yamada et al., 2022).

6. Application domains

Proximal descent is not tied to a single application family. In convex composite statistics, GeoPG is analyzed on linear regression and logistic regression with elastic net regularization, with $$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$3 handled by soft-thresholding (Chen et al., 2016). The same composite template underlies ISTA, projected gradient descent, and the $$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$4-regularized examples emphasized in silver-step acceleration (Bok et al., 2024). SLOPE extends this to sorted $$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$5 regularization, where the optimization challenge is nonseparability rather than nonconvexity (Larsson et al., 2022).

In signal and image processing, proximal descent appears in multiple guises. ML-BPGD is developed for image reconstruction under relative smoothness, with deconvolution, tomographic reconstruction, and D-optimal design as motivating instances (Elshiaty et al., 4 Jun 2025). PnP-PBCD is specialized to hyperspectral anomaly detection, combining low-rank background modeling, generalized group sparsity for anomalies, and a proximal denoiser prior for eigenimages (Liu et al., 2024). A different nonconvex proximal-gradient design, FPGD-SP, is used for locally regularized sparse graph construction in clustering, where the per-column proximal operator becomes weighted hard-thresholding and the final stage enjoys an $$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$6 rate after support stabilization (Sun et al., 2024).

Constrained and combinatorial optimization provide another substantial domain. Proximal distance algorithms have been instantiated for linear programming, nonnegative quadratic programming, projection to the closest kinship matrix, projection onto a second-order cone constraint, calculation of Horn’s copositive matrix index, linear complementarity programming, sparse principal components analysis, binary piecewise-linear programming, $$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$7 regression, matrix completion, and inverse sparse covariance estimation (Keys et al., 2016, Lange et al., 2015). In these examples the method’s modularity comes from reducing each iteration to a projection onto a structured set and a proximal or quadratic step.

Dynamical systems and hierarchical model selection also fit naturally into the paradigm. BCD-prox uses block coordinate proximal updates for simultaneous filtering and parameter estimation in ODE systems with up to $$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$8 dimensions (Raziperchikolaei et al., 2018). Hybrid steepest descent with proximal splitting operators has been used to define hierarchical enhancements of support vector machines and the Lasso estimator, where the fixed-point set of a proximal splitting method defines the base solution set and a secondary criterion chooses a preferred point within it (Yamada et al., 2022).

7. Limitations, misconceptions, and open directions

A common misconception is that proximal descent is synonymous with the basic proximal gradient method. The literature is broader: it includes proximal distance MM schemes, geometric ball-intersection methods, block-coordinate and asynchronous algorithms, Bregman mirror-like variants, operator-splitting fixed-point methods, and weakly convex bundle-proximal hybrids (Keys et al., 2016, Chen et al., 2016, Elshiaty et al., 4 Jun 2025, Liao et al., 2 Sep 2025).

A second misconception is that acceleration necessarily requires momentum. The silver stepsize schedule shows that asymptotically faster convergence can be obtained solely by a structured, nonmonotone stepsize pattern, even for proximal and projected gradient descent (Bok et al., 2024). At the same time, the same source states that momentum-based acceleration remains asymptotically faster in worst-case theory: $$x_{k+1}=\operatorname{prox}_{t_k g}(x_k-t_k\nabla f(x_k)).$$9 versus $g=\iota_C$0 in the smooth convex case, and $g=\iota_C$1 versus $g=\iota_C$2 in the strongly convex case (Bok et al., 2024).

Practical performance also depends strongly on oracle quality and geometry. The silver-step guarantees require exact proximal evaluations or sufficient accuracy, and they are outside the theory in nonconvex settings, where large steps may cause instability (Bok et al., 2024). GeoPG assumes a simple proximal operator and, in its strongest guarantees, strong convexity of the smooth component (Chen et al., 2016). Plug-and-play convergence in the cited analyses is not a generic property of arbitrary denoisers; it depends on denoisers that are provably proximal maps of weakly convex potentials (Hurault et al., 2023, Liu et al., 2024).

Open directions follow directly from these constraints. Extending geometric mechanisms beyond strong convexity is explicitly described as open and interesting in GeoPG (Chen et al., 2016). Weakly convex proximal descent has best-known general complexity bounds but still relies on problem constants such as the weak convexity parameter $g=\iota_C$3 (Liao et al., 2 Sep 2025). Multilevel Bregman methods require well-posed coarse models and transfer operators compatible with the constraint geometry (Elshiaty et al., 4 Jun 2025). This suggests that the long-term development of proximal descent will continue to revolve around three axes: tighter complexity theory, richer structural priors, and computational architectures that preserve the proximal subproblem’s tractability while enlarging the admissible problem class.