Variable Smoothing Alternating Proximal Gradient

• Variable Smoothing Alternating Proximal Gradient is a method that replaces a nonsmooth weakly convex component with a Moreau-envelope smoothed surrogate to achieve asymptotic stationarity.
• It integrates a forward gradient step on the adaptive smoothed function with a backward proximal step on the convex term, using an Armijo-based line search for efficient convergence.
• The approach has demonstrated practical success in MIMO detection, outperforming traditional methods with faster convergence and lower error rates through adaptive parameter tuning.

Searching arXiv for the primary paper and closely related variable-smoothing proximal-gradient works. Variable Smoothing Alternating Proximal Gradient denotes a class of first-order methods for nonsmooth optimization in which a nonsmooth weakly convex component is replaced at iteration $n$ by a smoother surrogate with smoothing parameter $\mu_n\searrow 0$, after which one performs a forward gradient step on the smoothed surrogate and a backward proximal step on the remaining term. In the formulation developed in "A Proximal Variable Smoothing for Nonsmooth Minimization Involving Weakly Convex Composite with MIMO Application" (Kume et al., 2024), the target problem is

$$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$

where $h$ is differentiable, $g$ is Lipschitz continuous, prox-friendly, and $\eta$-weakly convex, $\mathfrak{S}$ is continuously differentiable, and $\phi$ is proper, lower semicontinuous, convex, and prox-friendly. The resulting method is a time-varying forward-backward splitting scheme designed to obtain stationary points of a nonsmooth, generally nonconvex, weakly convex composite objective (Kume et al., 2024).

1. Problem class and structural setting

The basic optimization model is the three-term composite problem

$$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$

with $\mathcal{X},\mathcal{Z}$ Euclidean spaces. The term $\mu_n\searrow 0$0 is proper, lower semicontinuous, convex, and prox-friendly; $\mu_n\searrow 0$1 is differentiable with Lipschitz continuous gradient on $\mu_n\searrow 0$2; $\mu_n\searrow 0$3 is continuously differentiable and may be nonlinear; and $\mu_n\searrow 0$4 is Lipschitz continuous, prox-friendly, and $\mu_n\searrow 0$5-weakly convex in the sense that $\mu_n\searrow 0$6 is convex (Kume et al., 2024).

This structure is described as a weakly convex composite. Because $\mu_n\searrow 0$7 is weakly convex and $\mu_n\searrow 0$8 is smooth, the composite $\mu_n\searrow 0$9 is generally nonconvex but preserves exploitable regularity inherited from weak convexity and smoothness (Kume et al., 2024). The optimization objective is not necessarily global minimization; rather, the goal is to locate stationary points. Using the Rockafellar–Wets general subdifferential $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$0, a stationary point $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$1 satisfies

$$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$2

The assumptions used in the convergence analysis emphasize asymptotic stationarity rather than coercivity or a Kurdyka–Łojasiewicz argument. In particular, the analysis in (Kume et al., 2024) requires Lipschitz continuity of $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$3 and of $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$4, a variable smoothing sequence $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$5 that is nonsummable but slowly decreasing, and a stepsize sequence satisfying an Armijo-type decrease together with uniform lower and upper bounds relative to the Lipschitz constants. Coercivity of $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$6 is not explicitly required, and convergence is formulated in terms of a stationarity measure and cluster points rather than uniqueness of the limit.

2. Moreau-envelope smoothing and the variable-smoothing principle

The defining mechanism of the method is the replacement of the nonsmooth weakly convex term $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$7 by a progressively less smoothed surrogate constructed from the Moreau envelope. For $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$8, the Moreau envelope of $$\min_{\bm{x} \in \mathcal{X}} (h + g \circ \mathfrak{S} + \phi)(\bm{x}),$$9 is

$h$0

with minimizer $h$1 (Kume et al., 2024). Even when $h$2 is nonsmooth, $h$3 is continuously differentiable and satisfies

$h$4

Its gradient is Lipschitz continuous with constant $h$5, and $h$6 as $h$7 (Kume et al., 2024).

Only the term $h$8 is smoothed. At iteration $h$9, the method uses

$g$0

Then $g$1 is differentiable, with

$g$2

and its gradient is Lipschitz with constant

$g$3

under mild regularity of $g$4 (Kume et al., 2024). As $g$5, the smoothed objectives $g$6 converge pointwise to the original nonsmooth term $g$7.

The smoothing sequence is constrained by three conditions: $g$8 A typical choice is

$g$9

and the experiments in (Kume et al., 2024) use $\eta$0. The stated interpretation is that early iterations operate on more smoothed, easier problems with more stable gradients, while later iterations track the original nonsmooth objective more closely.

A related development in "A Proximal Variable Smoothing for Minimization of Nonlinearly Composite Nonsmooth Function -- Maxmin Dispersion and MIMO Applications" (Kume et al., 6 Jun 2025) uses the same basic principle: a single-loop proximal gradient-type method with a time-varying Moreau-smoothed surrogate and a decreasing smoothing schedule. A later two-block extension, "Variable Smoothing Alternating Proximal Gradient Algorithm for Coupled Composite Optimization" (Long et al., 31 Oct 2025), applies the same idea to a coupled composite problem with one block updated by a prox step and the other by a smoothed gradient step. This suggests a broader family of methods in which variable smoothing is integrated with forward-backward or alternating proximal-gradient architectures.

3. Alternating proximal-gradient iteration and stepsize selection

In (Kume et al., 2024), the algorithm is explicitly formulated as a forward-backward splitting method with a time-varying forward operator. Given $\eta$1, $\eta$2, and a smoothing sequence $\eta$3, each iteration performs:

1. Definition of the smoothed surrogate

$\eta$4

1. Choice of a stepsize $\eta$5 satisfying an Armijo decrease, a lower bound $\eta$6 for some $\eta$7, and a uniform upper bound $\eta$8.
2. Forward step

$\eta$9

1. Backward step

$\mathfrak{S}$0

Equivalently,

$\mathfrak{S}$1

The terminology “alternating proximal gradient” reflects the alternation between a gradient step on the smoothed composite $\mathfrak{S}$2 and a proximal step on $\mathfrak{S}$3, with the forward operator changing over time as $\mathfrak{S}$4 decreases (Kume et al., 2024).

The Armijo condition is expressed in terms of the smoothed problem: $\mathfrak{S}$5 Two concrete stepsize strategies are given. The first is the explicit rule

$\mathfrak{S}$6

and the second is backtracking,

$\mathfrak{S}$7

with $\mathfrak{S}$8 and $\mathfrak{S}$9 (Kume et al., 2024). The backtracking version removes the need for explicit knowledge of $\phi$0.

The practical implementation described in (Kume et al., 2024) begins with any $\phi$1. Per iteration, one computes $\phi$2, evaluates $\phi$3, applies $\phi$4 once to form $\phi$5, applies $\phi$6, and computes $\phi$7. The dominant costs are therefore evaluating $\phi$8 and its derivative and one proximal map each for $\phi$9 and $$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$0.

4. Stationarity measure and convergence theory

A central contribution of (Kume et al., 2024) is a stationarity measure tailored to the forward-backward structure. For any $$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$1 and $$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$2,

$$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$3

When $$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$4, this reduces to $$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$5. For general $$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$6,

$$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$7

so the measure vanishes exactly at stationary points (Kume et al., 2024).

The measure is lower semicontinuous in $$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$8, and the smoothed version is especially simple because $$\min_{\bm{x} \in \mathcal{X}} (f+\phi)(\bm{x}), \qquad f := h + g\circ \mathfrak{S},$$9 is differentiable: $\mathcal{X},\mathcal{Z}$0 This is the norm of the forward-backward residual for the smoothed problem. The key approximation theorem states that if $\mathcal{X},\mathcal{Z}$1 and $\mathcal{X},\mathcal{Z}$2, then for fixed $\mathcal{X},\mathcal{Z}$3,

$\mathcal{X},\mathcal{Z}$4

Hence, if the smoothed residuals tend to zero along a convergent sequence, the limit is stationary for the original nonsmooth objective (Kume et al., 2024).

Under the Lipschitz and stepsize assumptions, the main convergence theorem in (Kume et al., 2024) yields:

• Asymptotic stationarity in liminf sense:

$\mathcal{X},\mathcal{Z}$5

where $\mathcal{X},\mathcal{Z}$6.

• Existence of a subsequence $\mathcal{X},\mathcal{Z}$7 such that

$\mathcal{X},\mathcal{Z}$8

• Every cluster point of that subsequence is a stationary point of $\mathcal{X},\mathcal{Z}$9.

The proof uses the Armijo decrease to derive descent inequalities for $\mu_n\searrow 0$00, summability of the smoothed stationarity violations, and the lower-semicontinuity link between $\mu_n\searrow 0$01 and $\mu_n\searrow 0$02 (Kume et al., 2024).

Later works refine this perspective. The 2025 nonlinear-composite extension (Kume et al., 6 Jun 2025) introduces a gradient-mapping-type asymptotic approximation and proves sublinear complexity bounds for the smoothed stationarity measure. The two-block coupled-composite method (Long et al., 31 Oct 2025) establishes $\mu_n\searrow 0$03 iteration complexity to reach an $\mu_n\searrow 0$04-approximate stationary point. These results indicate that variable smoothing has evolved from asymptotic stationarity guarantees toward explicit complexity statements in broader settings.

5. Relation to proximal-gradient, smoothing, and alternating schemes

Within the taxonomy of first-order methods, Variable Smoothing Alternating Proximal Gradient occupies an intermediate position between classical proximal gradient, fixed-parameter smoothing, and more elaborate composite nonconvex algorithms.

Classical forward-backward splitting assumes a smooth $\mu_n\searrow 0$05 with Lipschitz gradient and a convex proximable $\mu_n\searrow 0$06. In that setting, one directly applies

$\mu_n\searrow 0$07

The difficulty addressed by (Kume et al., 2024) is that $\mu_n\searrow 0$08 is nonsmooth and nonconvex. Variable smoothing restores differentiability by replacing $\mu_n\searrow 0$09 with its Moreau envelope at each iteration, without requiring $\mu_n\searrow 0$10, which is generally difficult.

Compared with fixed-parameter smoothing, the distinction is that $\mu_n\searrow 0$11. Fixed smoothing optimizes a permanently smoothed approximation and therefore retains a fixed approximation bias; variable smoothing gradually removes that bias, so cluster-point stationarity is obtained for the original objective rather than merely the surrogate (Kume et al., 2024). The explicit trade-off is already visible in the earlier convex "Smoothing Proximal Gradient Method for General Structured Sparse Learning" (Chen et al., 2012), which notes that large $\mu_n\searrow 0$12 improves conditioning while small $\mu_n\searrow 0$13 reduces approximation error; it also reports that using a large $\mu_n\searrow 0$14 initially and reducing $\mu_n\searrow 0$15 over iterations leads to better empirical results, although convergence-rate analysis is harder in that setting.

The phrase “alternating proximal gradient” should not be conflated with block-coordinate PALM-type methods, although there are strong family resemblances. In (Kume et al., 2024), the alternation is between the forward step on the smoothed surrogate and the backward prox step on $\mu_n\searrow 0$16. By contrast, the coupled two-block algorithm in (Long et al., 31 Oct 2025) alternates across variable blocks as well: a smoothed gradient step in the $\mu_n\searrow 0$17-block for $\mu_n\searrow 0$18 and a prox step in the $\mu_n\searrow 0$19-block for $\mu_n\searrow 0$20. A plausible implication is that the 2024 method can be viewed as the single-block, three-term prototype from which later block-structured variants descend.

Related but distinct lines include smoothing proximal gradient with extrapolation for exact continuous relaxations of $\mu_n\searrow 0$21 regularization (Zhang et al., 2021), where smoothing is applied to a nonsmooth convex loss and coupled with inertial terms, and proximal ADMM with exponential smoothing of primal iterates (Zhang et al., 2018), where “smoothing” refers to a smoothed sequence of primal centers rather than Moreau-envelope smoothing of an objective component. These methods share the theme of time-varying regularization or smoothing but operate with different algorithmic objects.

6. Applications, empirical behavior, and later developments

The 2024 paper develops a concrete application to multiuser MIMO signal detection with $\mu_n\searrow 0$22-ary phase-shift keying (Kume et al., 2024). The physical model is

$\mu_n\searrow 0$23

with transmitted symbol vector $\mu_n\searrow 0$24, $\mu_n\searrow 0$25 transmit antennas, $\mu_n\searrow 0$26 receive antennas, and emphasis on the underdetermined case $\mu_n\searrow 0$27. After real embedding, earlier formulations include LMMSE, modulus-constrained least squares, and the SOAV model. The stated limitation of SOAV is that it is convex and cannot make points in the discrete constellation unique minimizers, so it lacks contrast to enforce discrete constellations strongly (Kume et al., 2024).

The proposed remedy is a polar-coordinate formulation. With

$\mu_n\searrow 0$28

the optimization model becomes

$\mu_n\searrow 0$29

The radius penalty encourages $\mu_n\searrow 0$30, while the angular penalty drives $\mu_n\searrow 0$31 toward $\mu_n\searrow 0$32. The model fits the general framework with $\mu_n\searrow 0$33, $\mu_n\searrow 0$34, and $\mu_n\searrow 0$35 the indicator of the box constraint (Kume et al., 2024).

The reported numerical setup uses random correlated MIMO channels, $\mu_n\searrow 0$36, typically $\mu_n\searrow 0$37, $\mu_n\searrow 0$38 or $\mu_n\searrow 0$39, smoothing sequence $\mu_n\searrow 0$40 with $\mu_n\searrow 0$41, and Armijo/backtracking parameters $\mu_n\searrow 0$42, $\mu_n\searrow 0$43, $\mu_n\searrow 0$44 (Kume et al., 2024). Against proximal subgradient baselines applied to the same model, the proposed method converges much faster in objective value versus CPU time. Against LMMSE, modulus-constrained least squares, and SOAV, all advanced nonlinear models outperform LMMSE, while the proposed model consistently achieves the lowest BER across SNRs, especially in the challenging $\mu_n\searrow 0$45 regime (Kume et al., 2024).

The 2025 follow-up (Kume et al., 6 Jun 2025) broadens the application portfolio to maxmin dispersion and MIMO signal detection while retaining the same single-loop proximal variable-smoothing architecture. Its experimental discussion again reports faster convergence than subgradient methods and strong detection performance. A broader contextual development appears in (Long et al., 31 Oct 2025), where the same variable-smoothing principle is extended to coupled composite optimization and evaluated on sparse signal recovery and image denoising, with explicit $\mu_n\searrow 0$46 iteration complexity.

Taken together, these works support a precise interpretation of Variable Smoothing Alternating Proximal Gradient: a time-varying forward-backward framework for nonsmooth nonconvex composite objectives in which Moreau-envelope smoothing is decreased over time, the forward operator is the gradient of the current smoothed surrogate, and the backward operator is the proximal map of the residual term. In the formulation of (Kume et al., 2024), its principal theoretical content is asymptotic stationarity via a smoothed surrogate residual, and its principal practical content is that a single-loop method based only on gradients and simple proximal mappings can be effective on structured signal-processing problems such as MIMO detection.