Inexact Variable Metric Proximal Linearization

• Inexact Variable Metric Proximal Linearization is a framework that builds a local first-order surrogate regularized by quadratic or Bregman proximal terms to stabilize and adapt updates.
• It leverages variable metrics and envelope techniques, including quasi-Newton and forward–backward methods, to optimize both smooth and nonsmooth objectives.
• Controlled inexact subproblem solutions balance model accuracy and computational cost while ensuring convergence across diverse problem classes.

Inexact Variable Metric Proximal Linearization denotes a family of optimization schemes in which a local first-order surrogate is regularized by a quadratic or Bregman proximal term whose metric may change with the iteration, and the resulting subproblem is solved only approximately. Across the literature, the term covers forward–backward schemes with variable SPD metrics and Armijo-like line search (Bonettini et al., 2015), envelope-based quasi-Newton acceleration via the Moreau envelope (Lin et al., 2016), general inexact model frameworks with Bregman geometry for optimization and variational inequalities (Stonyakin et al., 2019), fractional optimization with a variable metric proximal gradient-subgradient step (Yang et al., 15 Apr 2025), model-based proximal quasi-Newton methods for nonsmooth nonconvex problems (Jia et al., 24 Jul 2025), embedded-submanifold composite optimization (He et al., 16 Aug 2025), and modified proximal quasi-Newton methods with inexact evaluations of $f$, $\nabla f$, and proximal operators (Allaire et al., 16 Dec 2025). The unifying theme is that the “proximal” component stabilizes the local model, the “variable metric” component adapts local geometry, and the “inexact” component permits practical stopping rules while retaining convergence guarantees.

1. Core formulation and problem classes

A standard composite formulation is

$F(x) \equiv f_0(x) + f_1(x),$

where $f_0$ is continuously differentiable and $f_1$ is proper, convex, lower semicontinuous (Bonettini et al., 2015). Closely related formulations include

$F(x)=f(x)+h(x),$

with $f$ smooth convex and $h$ proper, closed, convex (Barré et al., 2020), and

$F(x)=f(x)=f_0(x)+\psi(x),$

where $f_0$ is smooth with $\nabla f$0-Lipschitz continuous gradient and $\nabla f$1 is a proper, closed, convex proximable function (Lin et al., 2016). The topic also includes block-structured constrained problems handled by inexact PALM-type updates (Hu et al., 2022), fractional models of the form

$\nabla f$2

with convex $\nabla f$3, $\nabla f$4 $\nabla f$5, and convex positive $\nabla f$6 (Yang et al., 15 Apr 2025), and manifold composite problems

$\nabla f$7

over a $\nabla f$8-smooth embedded closed submanifold $\nabla f$9 (He et al., 16 Aug 2025).

The central prox-linearized model is a local linearization of the smooth part plus a proximal regularization. In one canonical Euclidean form,

$F(x) \equiv f_0(x) + f_1(x),$0

with the scaled Euclidean instance

$F(x) \equiv f_0(x) + f_1(x),$1

(Bonettini et al., 2015). In a generic inexact-model formulation, the step is

$F(x) \equiv f_0(x) + f_1(x),$2

where $F(x) \equiv f_0(x) + f_1(x),$3 is either a Bregman divergence or a quadratic metric term (Stonyakin et al., 2019). In the variable-metric prox-linear update for fractional optimization,

$F(x) \equiv f_0(x) + f_1(x),$4

with $F(x) \equiv f_0(x) + f_1(x),$5 and $F(x) \equiv f_0(x) + f_1(x),$6 (Yang et al., 15 Apr 2025).

This suggests a broad common template: a surrogate is built from first-order information, a metric-dependent proximal term regularizes the step, and exact solution of the surrogate is replaced by a controlled approximation.

2. Variable metrics, Bregman geometry, and envelope viewpoints

The “variable metric” component is realized either by quadratic SPD operators or by a general Bregman divergence. For quadratic regularization,

$F(x) \equiv f_0(x) + f_1(x),$7

with $F(x) \equiv f_0(x) + f_1(x),$8 symmetric positive definite (Stonyakin et al., 2019). In the forward–backward line-search setting, the metric sequence satisfies

$F(x) \equiv f_0(x) + f_1(x),$9

which implies

$f_0$0

(Bonettini et al., 2015). In fractional optimization, the weighted norm is $f_0$1, and no explicit line search is required; one chooses a sequence of $f_0$2 satisfying the spectral bounds (Yang et al., 15 Apr 2025).

A Bregman formulation replaces the quadratic metric by

$f_0$3

and the inexact model inequality becomes

$f_0$4

for minimization, with a corresponding model notion for variational inequalities (Stonyakin et al., 2019). The paper explicitly states that this generalizes the $f_0$5-oracle of Devolder–Glineur–Nesterov and allows non-Euclidean setups (Stonyakin et al., 2019).

A distinct but closely related realization is the envelope-based variable-metric proximal point interpretation of QNing. For $f_0$6, the Moreau envelope is

$f_0$7

where

$f_0$8

(Lin et al., 2016). The paper applies quasi-Newton steps directly to the Moreau envelope of the full composite $f_0$9; it does not use the forward-backward envelope in its algorithmic core (Lin et al., 2016). The exact variable-metric proximal point step with metric $f_1$0 is

$f_1$1

whereas QNing performs a quasi-Newton step on $f_1$2,

$f_1$3

followed by a simple line search and a mixing with $f_1$4 under inexact gradients (Lin et al., 2016).

In the 2025 model-based proximal quasi-Newton formulation, the local model is any $f_1$5 obeying a local Taylor-like growth bound

$f_1$6

and the subproblem is

$f_1$7

(Jia et al., 24 Jul 2025). There, $f_1$8 comes from a continuous matrix generator $f_1$9, only a uniform lower bound $F(x)=f(x)+h(x),$0 is imposed a priori, and uniform boundedness of $F(x)=f(x)+h(x),$1 arises a posteriori under KŁ regularity (Jia et al., 24 Jul 2025).

3. Inexactness models and implementable stopping criteria

The “inexact” component appears in several non-equivalent but related ways. In the variable-metric line-search framework, an $F(x)=f(x)+h(x),$2-type approximation accepts $F(x)=f(x)+h(x),$3 if

$F(x)=f(x)+h(x),$4

with

$F(x)=f(x)+h(x),$5

which yields

$F(x)=f(x)+h(x),$6

(Bonettini et al., 2015). The same paper also provides an $F(x)=f(x)+h(x),$7-type approximation: $F(x)=f(x)+h(x),$8 and practical primal–dual-gap stopping rules for $F(x)=f(x)+h(x),$9 (Bonettini et al., 2015).

In the inexact model framework, subproblem inexactness $f$0 is encoded through Nemirovski’s inexact prox operator: for minimizing $f$1, an approximate solution $f$2 satisfies

$f$3

(Stonyakin et al., 2019). The paper treats model inexactness $f$4 and subproblem inexactness $f$5 separately, and both appear additively in the complexity bounds (Stonyakin et al., 2019).

QNing obtains inexact gradients through approximate proximal evaluations. At outer iterate $f$6, it approximately solves

$f$7

returns $f$8, and sets

$f$9

(Lin et al., 2016). The recommended stopping criterion is

$h$0

which implies

$h$1

when $h$2 (Lin et al., 2016). The paper also studies a fixed-budget heuristic, such as one pass over the data for SVRG (Lin et al., 2016).

In the fractional setting, the inexact optimality condition is

$h$3

with the practical error criterion

$h$4

(Yang et al., 15 Apr 2025). The paper describes this criterion as verifiable and flexible, allowing progressive inexactness (Yang et al., 15 Apr 2025).

Inexactness can also be controlled at the modeling level rather than by a residual in the subproblem solve. In the general proximal quasi-Newton method, the accepted step satisfies

$h$5

and the paper explicitly states that “inexactness” is handled at the level of the local model/acceptance criterion rather than via a residual $h$6 in the subproblem (Jia et al., 24 Jul 2025).

In iR2N, inexactness encompasses inaccurate values of $h$7, $h$8, and the proximal step. The proximal inexactness requirement is the relative-norm lower bound

$h$9

and the paper emphasizes that this is weaker than many classical inexactness criteria and is implementable (Allaire et al., 16 Dec 2025).

4. Descent mechanisms, line search, and global complexity

A recurrent feature is a sufficient-decrease mechanism. In the variable-metric forward–backward scheme, Armijo backtracking accepts $F(x)=f(x)=f_0(x)+\psi(x),$0 if

$F(x)=f(x)=f_0(x)+\psi(x),$1

and finite termination holds when $F(x)=f(x)=f_0(x)+\psi(x),$2 (Bonettini et al., 2015). Under Lipschitz gradient of $F(x)=f(x)=f_0(x)+\psi(x),$3 and the adaptive error condition, there exists $F(x)=f(x)=f_0(x)+\psi(x),$4 such that $F(x)=f(x)=f_0(x)+\psi(x),$5 for all $F(x)=f(x)=f_0(x)+\psi(x),$6 (Bonettini et al., 2015).

QNing uses a simple line search on $F(x)=f(x)=f_0(x)+\psi(x),$7 to enforce

$F(x)=f(x)=f_0(x)+\psi(x),$8

and the paper states that this always terminates, with $F(x)=f(x)=f_0(x)+\psi(x),$9 sufficing in the worst case (Lin et al., 2016). Under the adaptive inner accuracy criterion and sufficient decrease,

$f_0$0

(Lin et al., 2016).

In fractional optimization, the approximate descent inequality is

$f_0$1

and under $f_0$2 one obtains

$f_0$3

for some $f_0$4 (Yang et al., 15 Apr 2025). The notable aspect, stated explicitly, is that the KL-based rate analysis is proved without requiring a strict sufficient descent property (Yang et al., 15 Apr 2025).

For manifold composite optimization, the accepted step $f_0$5 satisfies

$f_0$6

and the resulting outer descent inequality is

$f_0$7

(He et al., 16 Aug 2025). Under restricted level boundedness, the method returns an $f_0$8-stationary point in at most

$f_0$9

outer iterations, and the paper further states $\nabla f$00 calls to the subproblem solver and $\nabla f$01 oracle calls under a dual first-order inner solver (He et al., 16 Aug 2025).

For iR2N, the stationarity surrogate is

$\nabla f$02

and the paper proves worst-case evaluation complexity $\nabla f$03 to reduce $\nabla f$04 below $\nabla f$05 (Allaire et al., 16 Dec 2025). The total number of iterations satisfies

$\nabla f$06

(Allaire et al., 16 Dec 2025).

5. Convergence guarantees and KŁ-based rates

The convergence theory depends on the underlying problem class. For convex objectives in the variable-metric forward–backward method, the whole sequence converges to a minimizer under summable $\nabla f$07 and controlled metric variation $\nabla f$08, and under Lipschitz gradient of $\nabla f$09 the paper gives the convergence rate estimate

$\nabla f$10

(Bonettini et al., 2015). For the more general nonconvex case, all limit points of the iterates sequence are stationary (Bonettini et al., 2015).

In QNing, if $\nabla f$11 is $\nabla f$12-strongly convex, then the Moreau envelope $\nabla f$13 is $\nabla f$14-strongly convex with $\nabla f$15, its condition number is $\nabla f$16, and the outer iterates satisfy the linear rate

$\nabla f$17

(Lin et al., 2016). In the convex but not strongly convex case, the rate is

$\nabla f$18

under bounded level sets (Lin et al., 2016).

In the inexact model framework, the non-accelerated method satisfies

$\nabla f$19

with $\nabla f$20, and the accelerated method satisfies

$\nabla f$21

(Stonyakin et al., 2019). For strongly convex objectives, restart schemes yield linear rates in terms of the Bregman divergence (Stonyakin et al., 2019).

In the fractional algorithm, any accumulation point is a critical point under A1–A5, spectral bounds on $\nabla f$22, and $\nabla f$23 (Yang et al., 15 Apr 2025). For the whole-sequence result, the paper introduces

$\nabla f$24

and an error accumulation sequence $\nabla f$25 (Yang et al., 15 Apr 2025). If $\nabla f$26 has the KL property with exponent $\nabla f$27, then the paper gives explicit convergence rates: finite convergence when $\nabla f$28, linear or geometric rates when $\nabla f$29 and $\nabla f$30 decays geometrically, and sublinear rates for $\nabla f$31 (Yang et al., 15 Apr 2025).

The 2025 general proximal quasi-Newton method proves that every accumulation point is stationary, and under the Kurdyka–Łojasiewicz property at a cluster point $\nabla f$32, the whole sequence converges to $\nabla f$33 with finite length

$\nabla f$34

(Jia et al., 24 Jul 2025). If $\nabla f$35, the rates are: finite termination for $\nabla f$36, superlinear convergence or finite termination for $\nabla f$37, Q-linear convergence of $\nabla f$38 and R-linear convergence of $\nabla f$39 for $\nabla f$40, and sublinear rates for $\nabla f$41 (Jia et al., 24 Jul 2025).

For manifold composite optimization, every cluster point is stationary under restricted level boundedness, and if the augmented potential $\nabla f$42 has the KL property on the cluster set, then $\nabla f$43 and the iterates converge globally to a stationary point (He et al., 16 Aug 2025). If the KL exponent is $\nabla f$44, the local rate is linear; if $\nabla f$45, the paper gives explicit sublinear rates for both $\nabla f$46 and $\nabla f$47 (He et al., 16 Aug 2025).

6. Relations to proximal gradient, prox-linear, quasi-Newton, and application domains

The literature repeatedly distinguishes inexact variable metric proximal linearization from classical prox-linear and proximal quasi-Newton methods. Classical proximal quasi-Newton minimizes at each outer step a local quadratic model

$\nabla f$48

where solving $\nabla f$49 can be nontrivial since $\nabla f$50 changes (Lin et al., 2016). QNing avoids repeatedly changing the proximal metric in inner loops by smoothing $\nabla f$51 via its Moreau envelope and applying quasi-Newton steps to the smooth envelope (Lin et al., 2016).

The general model-function approach broadens this viewpoint to nonconvex and nonsmooth objectives $\nabla f$52, additive composites $\nabla f$53, and composite structures $\nabla f$54 (Jia et al., 24 Jul 2025). The paper explicitly states four novel aspects relative to classic forward–backward, prox-linear, and quasi-Newton proximal methods: only local growth control near $\nabla f$55, an acceptance test based on model error, no a priori uniform boundedness of $\nabla f$56, and convergence to stationarity for nonconvex, nonsmooth problems (Jia et al., 24 Jul 2025).

The inexact-model framework of 2019 places proximal linearization within a larger class encompassing minimization, saddle-point problems, and variational inequalities (Stonyakin et al., 2019). It recovers proximal gradient, mirror descent, accelerated gradient methods, and generalized Mirror Prox as special cases, with variable metric/Bregman geometry entering through the prox function $\nabla f$57 and its divergence $\nabla f$58 (Stonyakin et al., 2019).

The manifold extension shows that the same ideas persist when Euclidean updates are replaced by tangent-space subproblems and retractions. RiVMPL extends Euclidean prox-linearization to embedded manifolds using retractions and tangent spaces, and introduces a variable metric $\nabla f$59 that can incorporate curvature of $\nabla f$60 via $\nabla f$61 (He et al., 16 Aug 2025). The paper compares this framework with Riemannian PG, manifold prox-QN, RiADMM, and RiALM (He et al., 16 Aug 2025).

Representative application domains in the cited papers include logistic regression, lasso, elastic-net, and large-scale empirical risk minimization (Lin et al., 2016); $\nabla f$62 Lasso and constrained $\nabla f$63 sparse optimization (Yang et al., 15 Apr 2025); polytope feasibility and sparse quadratic inverse problems (Jia et al., 24 Jul 2025); sparse spectral clustering, constrained group sparse PCA, and proper symplectic decomposition (He et al., 16 Aug 2025); total-variation image restoration (Bonettini et al., 2015); and nonsmooth regularized problems with $\nabla f$64 norms, $\nabla f$65 total variation, and the indicator of the nonconvex pseudo $\nabla f$66-norm ball (Allaire et al., 16 Dec 2025).

A plausible implication is that “Inexact Variable Metric Proximal Linearization” is best understood not as a single algorithm but as a design pattern: local first-order modeling, adaptive geometry through a variable metric or Bregman generator, and implementable inexactness control through residuals, primal–dual gaps, model-acceptance tests, or approximate optimality conditions. The cited papers differ in objective structure, globalization mechanism, and stationarity measure, but they consistently use that pattern to balance curvature exploitation, subproblem tractability, and rigorous convergence analysis (Bonettini et al., 2015, Lin et al., 2016, Stonyakin et al., 2019, Yang et al., 15 Apr 2025, Jia et al., 24 Jul 2025, He et al., 16 Aug 2025, Allaire et al., 16 Dec 2025).