---
title: Variable Smoothing Alternating Proximal Gradient
url: https://www.emergentmind.com/topics/variable-smoothing-alternating-proximal-gradient
type: topic
---

# Variable Smoothing Alternating Proximal Gradient

Searching arXiv for the primary paper and closely related variable-smoothing proximal-gradient works.
Variable Smoothing Alternating Proximal Gradient denotes a class of first-order methods for nonsmooth optimization in which a nonsmooth weakly convex component is replaced at iteration \(n\) by a smoother surrogate with smoothing parameter \(\mu_n\searrow 0\), after which one performs a forward gradient step on the smoothed surrogate and a backward proximal step on the remaining term. In the formulation developed in "A Proximal Variable Smoothing for Nonsmooth Minimization Involving Weakly Convex Composite with MIMO Application" [2409.10934], the target problem is
\[
\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),
\]
where \(h\) is differentiable, \(g\) is Lipschitz continuous, prox-friendly, and \(\eta\)-weakly convex, \(\mathfrak{S}\) is continuously differentiable, and \(\phi\) is proper, lower semicontinuous, convex, and prox-friendly. The resulting method is a time-varying forward-backward splitting scheme designed to obtain stationary points of a nonsmooth, generally nonconvex, weakly convex composite objective [2409.10934].

## 1. Problem class and structural setting

The basic optimization model is the three-term composite problem
\[
\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},
\]
with \(\mathcal{X},\mathcal{Z}\) Euclidean spaces. The term \(\phi : \mathcal{X}\to \mathbb{R}\cup\{+\infty\}\) is proper, lower semicontinuous, convex, and prox-friendly; \(h:\mathcal{X}\to\mathbb{R}\) is differentiable with Lipschitz continuous gradient on \(\operatorname{dom}\phi\); \(\mathfrak{S}:\mathcal{X}\to\mathcal{Z}\) is continuously differentiable and may be nonlinear; and \(g:\mathcal{Z}\to\mathbb{R}\) is Lipschitz continuous, prox-friendly, and \(\eta\)-weakly convex in the sense that \(g+\frac{\eta}{2}\|\cdot\|^2\) is convex [2409.10934].

This structure is described as a weakly convex composite. Because \(g\) is weakly convex and \(\mathfrak{S}\) is smooth, the composite \(g\circ\mathfrak{S}\) is generally nonconvex but preserves exploitable regularity inherited from weak convexity and smoothness [2409.10934]. The optimization objective is not necessarily global minimization; rather, the goal is to locate stationary points. Using the Rockafellar–Wets general subdifferential \(\partial\), a stationary point \(\bm{x}^\star\) satisfies
\[
\bm{0} \in \partial(f+\phi)(\bm{x}^\star)=\partial f(\bm{x}^\star)+\partial \phi(\bm{x}^\star).
\]

The assumptions used in the convergence analysis emphasize asymptotic stationarity rather than coercivity or a Kurdyka–Łojasiewicz argument. In particular, the analysis in [2409.10934] requires Lipschitz continuity of \(\nabla h\) and of \(\nabla({}^{\mu}g\circ\mathfrak{S})\), a variable smoothing sequence \(\mu_n\searrow 0\) that is nonsummable but slowly decreasing, and a stepsize sequence satisfying an Armijo-type decrease together with uniform lower and upper bounds relative to the Lipschitz constants. Coercivity of \(F\) is not explicitly required, and convergence is formulated in terms of a stationarity measure and cluster points rather than uniqueness of the limit.

## 2. Moreau-envelope smoothing and the variable-smoothing principle

The defining mechanism of the method is the replacement of the nonsmooth weakly convex term \(g\) by a progressively less smoothed surrogate constructed from the Moreau envelope. For \(\mu\in (0,\eta^{-1})\), the Moreau envelope of \(g\) is
\[
{}^{\mu}g(\widebar{\bm{z}}) := \min_{\bm{z}\in\mathcal{Z}}
\left\{ g(\bm{z}) + \frac{1}{2\mu}\|\bm{z}-\widebar{\bm{z}}\|^2 \right\},
\]
with minimizer \(\operatorname{prox}_{\mu g}(\widebar{\bm{z}})\) [2409.10934]. Even when \(g\) is nonsmooth, \({}^{\mu}g\) is continuously differentiable and satisfies
\[
\nabla({}^{\mu}g)(\bm{z}) = \mu^{-1}\big(\bm{z} - \operatorname{prox}_{\mu g}(\bm{z})\big).
\]
Its gradient is Lipschitz continuous with constant \(O(\mu^{-1})\), and \({}^{\mu}g(\bm{z})\to g(\bm{z})\) as \(\mu\to 0\) [2409.10934].

Only the term \(g\circ \mathfrak{S}\) is smoothed. At iteration \(n\), the method uses
\[
f_n(\bm{x}) := h(\bm{x}) + {}^{\mu_n} g(\mathfrak{S}(\bm{x})).
\]
Then \(f_n\) is differentiable, with
\[
\nabla f_n = \nabla h + (\mathrm{D}\mathfrak{S})^\ast \nabla({}^{\mu_n} g)\circ\mathfrak{S},
\]
and its gradient is Lipschitz with constant
\[
L_{\nabla f_n} = \varpi_1 + \varpi_2 \mu_n^{-1}
\]
under mild regularity of \(\mathfrak{S}\) [2409.10934]. As \(\mu_n\to 0\), the smoothed objectives \(f_n\) converge pointwise to the original nonsmooth term \(f\).

The smoothing sequence is constrained by three conditions:
\[
\mu_n\to 0,\qquad \sum_{n=1}^\infty \mu_n = +\infty,\qquad \exists M\ge 1:\ M^{-1}\le \mu_{n+1}/\mu_n \le 1.
\]
A typical choice is
\[
\mu_n = \frac{1}{2\eta}\,n^{-1/\alpha},\qquad \alpha\ge 1,
\]
and the experiments in [2409.10934] use \(\mu_n=(2\eta)^{-1}n^{-1/3}\). The stated interpretation is that early iterations operate on more smoothed, easier problems with more stable gradients, while later iterations track the original nonsmooth objective more closely.

A related development in "A Proximal Variable Smoothing for Minimization of Nonlinearly Composite Nonsmooth Function -- Maxmin Dispersion and MIMO Applications" [2506.05974] uses the same basic principle: a single-loop proximal gradient-type method with a time-varying Moreau-smoothed surrogate and a decreasing smoothing schedule. A later two-block extension, "Variable Smoothing Alternating Proximal Gradient Algorithm for Coupled Composite Optimization" [2510.27156], applies the same idea to a coupled composite problem with one block updated by a prox step and the other by a smoothed gradient step. This suggests a broader family of methods in which variable smoothing is integrated with forward-backward or alternating proximal-gradient architectures.

## 3. Alternating proximal-gradient iteration and stepsize selection

In [2409.10934], the algorithm is explicitly formulated as a forward-backward splitting method with a time-varying forward operator. Given \(\bm{x}_1\in\mathcal{X}\), \(c\in(0,1)\), and a smoothing sequence \((\mu_n)\subset(0,(2\eta)^{-1}]\), each iteration performs:

1. Definition of the smoothed surrogate
   \[
   f_n(\bm{x}) := h(\bm{x}) + {}^{\mu_n} g(\mathfrak{S}(\bm{x})).
   \]

2. Choice of a stepsize \(\gamma_n>0\) satisfying an Armijo decrease, a lower bound \(\gamma_n \ge \beta L_{\nabla f_n}^{-1}\) for some \(\beta>0\), and a uniform upper bound \(\sup_n\gamma_n<+\infty\).

3. Forward step
   \[
   \bm{x}_{n+\frac{1}{2}} = \bm{x}_n - \gamma_n \nabla f_n(\bm{x}_n).
   \]

4. Backward step
   \[
   \bm{x}_{n+1} = \operatorname{prox}_{\gamma_n\phi}\big(\bm{x}_{n+\frac{1}{2}}\big).
   \]

Equivalently,
\[
\bm{x}_{n+1} = \operatorname{prox}_{\gamma_n\phi}\big(\bm{x}_n - \gamma_n \nabla f_n(\bm{x}_n)\big).
\]
The terminology “alternating proximal gradient” reflects the alternation between a gradient step on the smoothed composite \(h+{}^{\mu_n}g\circ \mathfrak{S}\) and a proximal step on \(\phi\), with the forward operator changing over time as \(\mu_n\) decreases [2409.10934].

The Armijo condition is expressed in terms of the smoothed problem:
\[
\begin{aligned}
&(f_n+\phi)\Big(\operatorname{prox}_{\gamma_n\phi}(\bm{x}_n - \gamma_n\nabla f_n(\bm{x}_n))\Big) \\
&\qquad \le (f_n+\phi)(\bm{x}_n) - c\gamma_n \big(\mathcal{M}_{\gamma_n}^{f_n,\phi}(\bm{x}_n)\big)^2.
\end{aligned}
\]
Two concrete stepsize strategies are given. The first is the explicit rule
\[
\gamma_n = \frac{2(1-c)}{L_{\nabla f_n}},
\]
and the second is backtracking,
\[
\gamma_n = \max\{\gamma_{\rm init}\rho^k : k\in\{0,1,\dots\},\;\text{Armijo holds}\},
\]
with \(\gamma_{\rm init}>0\) and \(\rho\in(0,1)\) [2409.10934]. The backtracking version removes the need for explicit knowledge of \(L_{\nabla f_n}\).

The practical implementation described in [2409.10934] begins with any \(\bm{x}_1\in\operatorname{dom}\phi\). Per iteration, one computes \(\nabla h(\bm{x}_n)\), evaluates \(\mathfrak{S}(\bm{x}_n)\), applies \(\operatorname{prox}_{\mu_n g}\) once to form \(\nabla({}^{\mu_n}g)\), applies \((\mathrm{D}\mathfrak{S})^\ast\), and computes \(\operatorname{prox}_{\gamma_n\phi}\). The dominant costs are therefore evaluating \(\mathfrak{S}\) and its derivative and one proximal map each for \(g\) and \(\phi\).

## 4. Stationarity measure and convergence theory

A central contribution of [2409.10934] is a stationarity measure tailored to the forward-backward structure. For any \(\gamma>0\) and \(\widebar{\bm{x}}\in\mathcal{X}\),
\[
\mathcal{M}_{\gamma}^{f,\phi}(\widebar{\bm{x}})
:= \inf\Big\{\gamma^{-1}\big\|\widebar{\bm{x}} - \operatorname{prox}_{\gamma\phi}(\widebar{\bm{x}} - \gamma\bm{v})\big\|
\;\Big|\; \bm{v} \in \partial f(\widebar{\bm{x}})\Big\}.
\]
When \(\phi\equiv 0\), this reduces to \(d(\bm{0},\partial f(\widebar{\bm{x}}))\). For general \(\phi\),
\[
\bm{0}\in\partial f(\widebar{\bm{x}})+\partial\phi(\widebar{\bm{x}})
\iff
\mathcal{M}^{f,\phi}_\gamma(\widebar{\bm{x}})=0
\quad (\forall\gamma>0),
\]
so the measure vanishes exactly at stationary points [2409.10934].

The measure is lower semicontinuous in \(\bm{x}\), and the smoothed version is especially simple because \(f_n\) is differentiable:
\[
\mathcal{M}_{\gamma}^{f_n,\phi}(\bm{x})
=
\gamma^{-1}\big\|\bm{x} - \operatorname{prox}_{\gamma\phi}(\bm{x}-\gamma\nabla f_n(\bm{x}))\big\|.
\]
This is the norm of the forward-backward residual for the smoothed problem. The key approximation theorem states that if \(\bm{x}_n\to \widebar{\bm{x}}\) and \(\mu_n\searrow 0\), then for fixed \(\gamma>0\),
\[
\liminf_{n\to\infty} \mathcal{M}_{\gamma}^{f_n,\phi}(\bm{x}_n)
\ge
\mathcal{M}_{\gamma}^{f,\phi}(\widebar{\bm{x}}).
\]
Hence, if the smoothed residuals tend to zero along a convergent sequence, the limit is stationary for the original nonsmooth objective [2409.10934].

Under the Lipschitz and stepsize assumptions, the main convergence theorem in [2409.10934] yields:

- Asymptotic stationarity in liminf sense:
  \[
  \liminf_{n\to\infty} \mathcal{M}^{f_n,\phi}_{\widebar{\gamma}}(\bm{x}_n) = 0,
  \]
  where \(\widebar{\gamma} := \sup_n \gamma_n <+\infty\).

- Existence of a subsequence \(\bm{x}_{m(l)}\) such that
  \[
  \lim_{l\to\infty}\mathcal{M}^{f_{m(l)},\phi}_{\widebar{\gamma}}(\bm{x}_{m(l)})=0.
  \]

- Every cluster point of that subsequence is a stationary point of \(f+\phi\).

The proof uses the Armijo decrease to derive descent inequalities for \(f_n+\phi\), summability of the smoothed stationarity violations, and the lower-semicontinuity link between \(\mathcal{M}_{\gamma}^{f_n,\phi}\) and \(\mathcal{M}_{\gamma}^{f,\phi}\) [2409.10934].

Later works refine this perspective. The 2025 nonlinear-composite extension [2506.05974] introduces a gradient-mapping-type asymptotic approximation and proves sublinear complexity bounds for the smoothed stationarity measure. The two-block coupled-composite method [2510.27156] establishes \(\mathcal{O}(\varepsilon^{-3})\) iteration complexity to reach an \(\varepsilon\)-approximate stationary point. These results indicate that variable smoothing has evolved from asymptotic stationarity guarantees toward explicit complexity statements in broader settings.

## 5. Relation to proximal-gradient, smoothing, and alternating schemes

Within the taxonomy of first-order methods, Variable Smoothing Alternating Proximal Gradient occupies an intermediate position between classical proximal gradient, fixed-parameter smoothing, and more elaborate composite nonconvex algorithms.

Classical forward-backward splitting assumes a smooth \(f\) with Lipschitz gradient and a convex proximable \(\phi\). In that setting, one directly applies
\[
\bm{x}_{n+1} = \operatorname{prox}_{\gamma_n\phi}\big(\bm{x}_n-\gamma_n\nabla f(\bm{x}_n)\big).
\]
The difficulty addressed by [2409.10934] is that \(f=h+g\circ \mathfrak{S}\) is nonsmooth and nonconvex. Variable smoothing restores differentiability by replacing \(g\) with its Moreau envelope at each iteration, without requiring \(\operatorname{prox}_{g\circ\mathfrak{S}}\), which is generally difficult.

Compared with fixed-parameter smoothing, the distinction is that \(\mu_n\to 0\). Fixed smoothing optimizes a permanently smoothed approximation and therefore retains a fixed approximation bias; variable smoothing gradually removes that bias, so cluster-point stationarity is obtained for the original objective rather than merely the surrogate [2409.10934]. The explicit trade-off is already visible in the earlier convex "Smoothing Proximal Gradient Method for General Structured Sparse Learning" [1202.3708], which notes that large \(\mu\) improves conditioning while small \(\mu\) reduces approximation error; it also reports that using a large \(\mu\) initially and reducing \(\mu\) over iterations leads to better empirical results, although convergence-rate analysis is harder in that setting.

The phrase “alternating proximal gradient” should not be conflated with block-coordinate PALM-type methods, although there are strong family resemblances. In [2409.10934], the alternation is between the forward step on the smoothed surrogate and the backward prox step on \(\phi\). By contrast, the coupled two-block algorithm in [2510.27156] alternates across variable blocks as well: a smoothed gradient step in the \(y\)-block for \(g(Ay)\) and a prox step in the \(x\)-block for \(f(x)\). A plausible implication is that the 2024 method can be viewed as the single-block, three-term prototype from which later block-structured variants descend.

Related but distinct lines include smoothing proximal gradient with extrapolation for exact continuous relaxations of \(\ell_0\) regularization [2112.01114], where smoothing is applied to a nonsmooth convex loss and coupled with inertial terms, and proximal ADMM with exponential smoothing of primal iterates [1812.10229], where “smoothing” refers to a smoothed sequence of primal centers rather than Moreau-envelope smoothing of an objective component. These methods share the theme of time-varying regularization or smoothing but operate with different algorithmic objects.

## 6. Applications, empirical behavior, and later developments

The 2024 paper develops a concrete application to multiuser MIMO signal detection with \(M\)-ary phase-shift keying [2409.10934]. The physical model is
\[
\mathsf{y} = \mathsf{H}\mathsf{s}^\star + \mathsf{e},
\]
with transmitted symbol vector \(\mathsf{s}^\star \in \mathsf{D}\subset \mathbb{C}^U\), \(U\) transmit antennas, \(B\) receive antennas, and emphasis on the underdetermined case \(U>B\). After real embedding, earlier formulations include LMMSE, modulus-constrained least squares, and the SOAV model. The stated limitation of SOAV is that it is convex and cannot make points in the discrete constellation unique minimizers, so it lacks contrast to enforce discrete constellations strongly [2409.10934].

The proposed remedy is a polar-coordinate formulation. With
\[
F(\bm{r},\bm{\theta})=
\begin{bmatrix}
\bm{r}\odot \mathbf{sin}(\bm{\theta})\\
\bm{r}\odot \mathbf{cos}(\bm{\theta})
\end{bmatrix},
\]
the optimization model becomes
\[
\begin{aligned}
\min_{\bm{r},\bm{\theta}} \quad &
\frac{1}{2}\|\bm{y}-\bm{H}F(\bm{r},\bm{\theta})\|^2
+
\lambda_{r}\sum_{u=1}^{U} [\bm{r}]_{u}^{-1}
+
\lambda_{\theta}\,\big\|\mathbf{sin}\big(\tfrac{M\bm{\theta}}{2}\big)\big\|_1 \\
\text{s.t.}\quad &
\bm{r} \in [r_{\min},1]^U,\;\bm{\theta}\in\mathbb{R}^U.
\end{aligned}
\]
The radius penalty encourages \(\bm{r}\to \bm{1}\), while the angular penalty drives \(\theta_u\) toward \(\{2\pi m/M:m\in\mathbb{Z}\}\). The model fits the general framework with \(g(\bm{z})=\lambda_\theta\|\bm{z}\|_1\), \(\mathfrak{S}(\bm{r},\bm{\theta})=\mathbf{sin}(M\bm{\theta}/2)\), and \(\phi\) the indicator of the box constraint [2409.10934].

The reported numerical setup uses random correlated MIMO channels, \(M=8\), typically \(U=128\), \(B=U\) or \(B=3U/4\), smoothing sequence \(\mu_n=(2\eta)^{-1}n^{-1/3}\) with \(\eta=1\), and Armijo/backtracking parameters \(c=2^{-13}\), \(\gamma_{\rm init}=1\), \(\rho=1/2\) [2409.10934]. Against proximal subgradient baselines applied to the same model, the proposed method converges much faster in objective value versus CPU time. Against LMMSE, modulus-constrained least squares, and SOAV, all advanced nonlinear models outperform LMMSE, while the proposed model consistently achieves the lowest BER across SNRs, especially in the challenging \(B=3U/4\) regime [2409.10934].

The 2025 follow-up [2506.05974] broadens the application portfolio to maxmin dispersion and MIMO signal detection while retaining the same single-loop proximal variable-smoothing architecture. Its experimental discussion again reports faster convergence than subgradient methods and strong detection performance. A broader contextual development appears in [2510.27156], where the same variable-smoothing principle is extended to coupled composite optimization and evaluated on sparse signal recovery and image denoising, with explicit \(\mathcal{O}(\varepsilon^{-3})\) iteration complexity.

Taken together, these works support a precise interpretation of Variable Smoothing Alternating Proximal Gradient: a time-varying forward-backward framework for nonsmooth nonconvex composite objectives in which Moreau-envelope smoothing is decreased over time, the forward operator is the gradient of the current smoothed surrogate, and the backward operator is the proximal map of the residual term. In the formulation of [2409.10934], its principal theoretical content is asymptotic stationarity via a smoothed surrogate residual, and its principal practical content is that a single-loop method based only on gradients and simple proximal mappings can be effective on structured signal-processing problems such as MIMO detection.

Source: https://www.emergentmind.com/topics/variable-smoothing-alternating-proximal-gradient