Projected Variable Smoothing Algorithms

• Projected variable smoothing-type algorithms are first-order methods that smooth nonsmooth functions using the Moreau envelope and enforce feasibility via explicit projection.
• They achieve provable complexity bounds (e.g., O(ε⁻³)) by integrating adaptive gradient steps with variable smoothing parameters and projection onto constraint sets.
• Widely used in signal processing, robust optimization, and large-scale learning, these methods offer practical improvements in convergence speed and computational efficiency.

A projected variable smoothing-type algorithm refers to a family of first-order optimization algorithms that combine variable smoothing of nonsmooth (often weakly convex) composite functions with explicit projection (typically onto a constraint set or subspace), and that are analyzed in the context of rigorous convergence and complexity guarantees. These methods exploit smooth surrogates constructed through the Moreau envelope, perform updates using gradient or forward-backward (proximal) steps restricted to a feasible set via projection, and are applicable to problems involving composite nonsmooth structure and potentially nonconvex constraints. This class of algorithms is now central in nonsmooth optimization, signal processing, and large-scale learning, incorporating advances in smooth approximation, efficient projection schemes, and robust convergence theory.

1. Mathematical Principles and Algorithmic Structure

Projected variable smoothing-type algorithms solve problems of the form

$\min_{x \in V} \ h(x) + g(Ax)$

where $V \subseteq H$ is a closed vector subspace or more generally a closed convex or nonconvex set, $h$ is smooth (with a Lipschitz continuous gradient), $g$ is a (possibly nonsmooth) weakly convex function, and $A$ is a (possibly nonlinear) mapping. The algorithm replaces $g$ with its Moreau envelope,

$$g_\mu(z) = \inf_{y} \left\{ g(y) + \frac{1}{2\mu}\|y - z\|^2 \right\}, \quad \text{with } \mu > 0,$$

yielding a smooth surrogate $h(x) + g_\mu(Ax)$. As $\mu \to 0$, $g_\mu$ approaches $g$ (in pointwise sense), while the gradient is computable as

$$\nabla g_\mu(z) = \frac{1}{\mu}(z - \mathrm{prox}_{\mu g}(z)).$$

The main iteration is then

$$x_{k+1} = P_V \left( x_k - \gamma_k \nabla F_k(x_k) \right), \qquad F_k(x) = h(x) + g_{\mu_k}(A x),$$

where $P_V$ is the projection onto $V$ (or another appropriate projection for the feasible set structure), and $\{\mu_k\}$ is a sequence of smoothing parameters decreasing to zero. The step-size $\gamma_k$ is adapted, often set as $1/L_k$ with $L_k$ the Lipschitz constant of $\nabla F_k$.

For more general models (involving additional regularization $\phi$, nonlinear mappings $S$, or sum/supremum structures in $g$), the update becomes

$$x_{k+1} = \mathrm{prox}_{\gamma_k \phi} \left( x_k - \gamma_k \nabla (h + g_{\mu_k} \circ S)(x_k) \right).$$

Convergence is typically analyzed in terms of the decay of a stationarity or criticality measure defined by the norm of the projected gradient or generalized fixed-point residual.

2. Theoretical Properties and Complexity

Projected variable smoothing-type algorithms achieve provable complexity bounds—most notably, an iteration complexity of $O(\epsilon^{-3})$ for obtaining an $\epsilon$-stationary solution in weakly convex minimization, interpolating between the $O(\epsilon^{-2})$ rate for smooth nonconvex problems and the $O(\epsilon^{-4})$ rate for subgradient methods (Böhm et al., 2020, López-Rivera et al., 1 Feb 2025). The results rely on:

• The Moreau envelope of a (weakly) convex function is continuously differentiable with a Lipschitz gradient for $\mu$ sufficiently small, even when $g$ itself is nonsmooth.
• The norm of the projected gradient of the smoothed surrogate is an upper bound on a distance to first-order stationarity in the original nonsmooth problem; this is formalized via the gradient consistency property,

$$\lim_{(y,\mu) \to (y^*,0)} \nabla (g^\mu \circ S)(y) \subseteq \partial (g \circ S)(y^*)$$

(Kume et al., 5 Dec 2024).

• Descent-type inequalities (using Armijo-type line search or fixed step-size) and summability conditions on the smoothing parameter sequence ensure that any cluster point of the generated sequence satisfies the necessary optimality conditions for the original nonsmooth, constrained problem.

In the presence of additional structure (e.g., supremum functions as regularizers, or parametric mappings for nonconvex constraint sets), proper selection of the projection operator and parametrization function ensures that stationarity for the lifted problem maps back to appropriate (first-order) stationarity in the original variable-constrained problem.

3. Smoothing, Proximity, and Projection Operations

The central tool enabling these algorithms is the Moreau envelope: $$g_\mu(z) = \min_y \{ g(y) + \tfrac{1}{2\mu} \|y-z\|^2 \}$$ with corresponding proximity operator

$$\mathrm{prox}_{\mu g}(z) = \arg\min_y \{ g(y) + \tfrac{1}{2\mu}\|y-z\|^2 \}$$

and gradient

$$\nabla g_\mu(z) = \frac{1}{\mu}(z - \mathrm{prox}_{\mu g}(z)).$$

For linear composition, $\nabla (g_\mu \circ A)(x) = A^* \nabla g_\mu(Ax)$.

Projection occurs onto a subspace $V$ or a nonconvex set parameterized by a smooth mapping $F$, i.e., $C = F(\mathcal{Y})$ for variable $\mathcal{Y}$ in an ambient Euclidean space (Kume et al., 2023, Kume et al., 5 Dec 2024). This approach allows explicit handling of complex constraints (e.g., Stiefel or Grassmannian structure in sparse PCA or clustering (Peng et al., 2022, Kume et al., 5 Dec 2024)).

The projection and proximity operations are also key in the full forward-backward splitting setting, where each iteration consists of:

1. Gradient descent on the smooth surrogate,
2. Application of the proximity operator for the nonsmooth constraint or penalty, and
3. (If needed) projection onto the feasible set.

4. Convergence, Stationarity Measures, and Asymptotic Guarantees

Progress and stationarity are measured using a generalized gradient mapping-type stationarity metric. For smooth $F$ and prox-friendly $\phi$, the measure is

$$\mathcal{M}_\gamma^{F, \phi}(x) = \min_{v \in \partial F(x)} \frac{1}{\gamma}\| x - \text{prox}_{\gamma \phi}(x - \gamma v) \|.$$

If $\mathcal{M}_\gamma^{F_n, \phi}(x_n) \to 0$ as $n \to \infty$ (with $F_n$ the $n$th smoothed surrogate), then any cluster point is a stationary point for $F+\phi$ (Kume et al., 17 Sep 2024, Kume et al., 6 Jun 2025).

In the unconstrained setting or when the constraint is parameterized by $F$, one considers the norm of the gradient of the smoothed surrogate; a vanishing gradient norm then suffices for asymptotic stationarity, assured by the gradient consistency property.

Convergence relies on:

• Proper rate of decrease for $\mu_n$ (e.g., $\mu_n = cn^{-\alpha}$ with $0 < \alpha < 1$), and
• Descent properties ensured by line search (e.g., Armijo rule)
• Summability and technical conditions on the smoothing parameter sequence.

5. Applications and Implementation Domains

Projected variable smoothing-type algorithms are widely deployed in the following areas:

• Signal recovery and imaging: Total variation denoising, deblurring, compressed sensing MRI using redundant frames, and robust phase retrieval (Liu et al., 2015, Bot et al., 2019, Yazawa et al., 18 Mar 2025).
• Robust and distributionally robust optimization: Problems with uncertainty in the objective (DRO) where the objective includes a supremum over a family of weakly convex functions (López-Rivera et al., 1 Feb 2025).
• Sparse learning and matrix factorization: Sparse principal component analysis, sparse spectral clustering, constrained LASSO (Peng et al., 2022, Kume et al., 5 Dec 2024, López-Rivera et al., 1 Feb 2025).
• MIMO signal detection: Formulations enforcing discrete algebraic constraints (e.g., phase-shift keying structure) via structure-promoting regularizers (Kume et al., 17 Sep 2024, Kume et al., 6 Jun 2025).
• Maxmin dispersion and location problems: Nonconvex location problems involving the minimum or maximum over a family of quadratic losses, often with additional linear subspace constraints (López-Rivera et al., 1 Feb 2025, Kume et al., 6 Jun 2025).
• Nonsmooth vector optimization under variable orderings: Inexact projected gradient methods generalize to smoothing-type strategies (Cruz et al., 2017).

A core advantage is the flexibility: provided the proximity operator for $g$ and the projection onto $V$ (or a suitable parametrization) is available, the algorithm is implementable without inner iterative loops for the nonsmooth term (unlike classical DCA or majorization algorithms (Yazawa et al., 18 Mar 2025)).

6. Numerical Performance and Empirical Insights

Empirical studies consistently show that projected variable smoothing-type algorithms attain favorable trade-offs between computational efficiency, solution accuracy, and robustness to nonsmoothness and nonconvexity. Results include:

• Faster convergence to lower objectives or reduced error rates compared to standard subgradient or primal-dual methods (Bot et al., 2012, Bot et al., 2019, Kume et al., 6 Jun 2025).
• Improved clustering metrics (NMI, ARI) in sparse spectral clustering due to the ability to handle weakly convex (nonconvex) regularizers and nonconvex constraints via parametrization (Kume et al., 5 Dec 2024, Kume et al., 2023).
• Superior bit error rates in large-scale MIMO detection under realistic SNR settings and underdetermined regimes (Kume et al., 17 Sep 2024, Kume et al., 6 Jun 2025).
• The ability to handle structured subspace constraints or supremum-type objectives efficiently for robust location and DRO problems (López-Rivera et al., 1 Feb 2025). Below is a summary table illustrating representative applications, problem structures, and algorithmic features:

Application Domain	Problem Structure	Smoothing/Projection Feature
Sparse Spectral Clustering	Minimize $h(x) + g(S(x))$ s.t. $x \in$ Grassmann	Parametrization $F$, Moreau envelope, gradient descent
Maxmin Dispersion	$\min_{x \in V} \max_{j} -w_j \|x-u_j\|^2$	Projection $P_V$, prox of max, variable smoothing
MIMO Detection	Signal detection w/ PSK penalties	Moreau smoothing for penalty, projection/constraint
Robust Phase Retrieval	Minimize DC function wrt phase/noise	Smoothing each DC term, single-loop gradient descent

7. Significance and Connections to Other Frameworks

Projected variable smoothing-type algorithms unify diverse research lines in nonsmooth and weakly convex optimization, bridging classical variational formulations (0802.0130) with modern variable projection (Leeuwen et al., 2016), stochastic splitting (Bot et al., 2019), and single-loop, forward-backward methodologies (Kume et al., 17 Sep 2024, Kume et al., 6 Jun 2025). The reliance on the Moreau envelope with variable smoothing, together with explicit projection, leads to robust algorithms for high-dimensional, composite-structured, and nonconvex problems.

The gradient consistency property and convergence complexity $O(\epsilon^{-3})$ frame these algorithms as state-of-the-art for their class, offering practical and theoretical advantages across a variety of challenging applications.