---
title: Inexact Variable Metric Proximal Linearization
url: https://www.emergentmind.com/topics/inexact-variable-metric-proximal-linearization
type: topic
---

# Inexact Variable Metric Proximal Linearization

Inexact Variable Metric Proximal Linearization denotes a family of optimization schemes in which a local first-order surrogate is regularized by a quadratic or Bregman proximal term whose metric may change with the iteration, and the resulting subproblem is solved only approximately. Across the literature, the term covers forward–backward schemes with variable SPD metrics and Armijo-like line search [1506.00385], envelope-based quasi-Newton acceleration via the Moreau envelope [1610.00960], general inexact model frameworks with Bregman geometry for optimization and variational inequalities [1902.00990], fractional optimization with a variable metric proximal gradient-subgradient step [2504.11023], model-based proximal quasi-Newton methods for nonsmooth nonconvex problems [2507.18363], embedded-submanifold composite optimization [2508.12003], and modified proximal quasi-Newton methods with inexact evaluations of \(f\), \(\nabla f\), and proximal operators [2512.14507]. The unifying theme is that the “proximal” component stabilizes the local model, the “variable metric” component adapts local geometry, and the “inexact” component permits practical stopping rules while retaining convergence guarantees.

## 1. Core formulation and problem classes

A standard composite formulation is
\[
F(x) \equiv f_0(x) + f_1(x),
\]
where \(f_0\) is continuously differentiable and \(f_1\) is proper, convex, lower semicontinuous [1506.00385]. Closely related formulations include
\[
F(x)=f(x)+h(x),
\]
with \(f\) smooth convex and \(h\) proper, closed, convex [2006.06041], and
\[
F(x)=f(x)=f_0(x)+\psi(x),
\]
where \(f_0\) is smooth with \(L\)-Lipschitz continuous gradient and \(\psi\) is a proper, closed, convex proximable function [1610.00960]. The topic also includes block-structured constrained problems handled by inexact PALM-type updates [2204.06182], fractional models of the form
\[
\min_{\bm{x}\in\Omega}\; F(\bm{x}) \;:=\; \frac{f(\bm{x})+h(\bm{x})}{g(\bm{x})},
\]
with convex \(f\), \(C^1\) \(h\), and convex positive \(g\) [2504.11023], and manifold composite problems
\[
\min_{x\in\mathcal{M}} \Theta(x):= f(x)+\vartheta(F(x))
\]
over a \(\mathcal{C}^2\)-smooth embedded closed submanifold \(\mathcal{M}\) [2508.12003].

The central prox-linearized model is a local linearization of the smooth part plus a proximal regularization. In one canonical Euclidean form,
\[
h_\sigma(y,x) := \nabla f_0(x)^T (y - x) + d_\sigma(y,x) + f_1(y) - f_1(x),
\]
with the scaled Euclidean instance
\[
d_\sigma(y,x) = \frac{1}{2\alpha}\|y-x\|_{D}^2, \quad \sigma=(\alpha,D),\ \alpha>0,\ D\succ 0
\]
[1506.00385]. In a generic inexact-model formulation, the step is
\[
x_{k+1} \in \operatorname{Argmin}^{\widetilde{\delta}_k}_{x \in Q} \big\{ \langle\nabla g(x_k), x-x_k\rangle + h(x) + (1/\alpha_k) D(x,x_k) \big\},
\]
where \(D\) is either a Bregman divergence or a quadratic metric term [1902.00990]. In the variable-metric prox-linear update for fractional optimization,
\[
\bm{x}^{k+1}\;\approx\;\operatorname{prox}^{H_k}_{f}\Big(\bm{x}^k-H_k^{-1}\big(\nabla h(\bm{x}^k)-c_k\bm{y}^k\big)\Big),
\]
with \(H_k\succ0\) and \(\underline{\gamma}I_n \preceq H_k \preceq \overline{\gamma}I_n\) [2504.11023].

This suggests a broad common template: a surrogate is built from first-order information, a metric-dependent proximal term regularizes the step, and exact solution of the surrogate is replaced by a controlled approximation.

## 2. Variable metrics, Bregman geometry, and envelope viewpoints

The “variable metric” component is realized either by quadratic SPD operators or by a general Bregman divergence. For quadratic regularization,
\[
D_{H_k}(x, x_k) = \frac{1}{2} \|x-x_k\|_{H_k}^2,
\]
with \(H_k\) symmetric positive definite [1902.00990]. In the forward–backward line-search setting, the metric sequence satisfies
\[
\mathcal{M}_\mu := \left\{D \succ 0 : \lambda(D) \subset \left[\frac{1}{\mu}, \mu\right]\right\},
\]
which implies
\[
\frac{1}{\mu}\|u\|^2 \le \|u\|_{D}^2 \le \mu \|u\|^2
\]
[1506.00385]. In fractional optimization, the weighted norm is \(\|\bm{u}\|_{H_k}=\sqrt{\langle \bm{u},H_k\bm{u}\rangle}\), and no explicit line search is required; one chooses a sequence of \(H_k\) satisfying the spectral bounds [2504.11023].

A Bregman formulation replaces the quadratic metric by
\[
V[y](x) = d(x) - d(y) - \langle\nabla d(y), x-y\rangle,
\]
and the inexact model inequality becomes
\[
0 \le f(x) - [f_\delta(y) + \psi_\delta(x,y)] \le L V[y](x) + \delta
\]
for minimization, with a corresponding model notion for variational inequalities [1902.00990]. The paper explicitly states that this generalizes the \((\delta,L)\)-oracle of Devolder–Glineur–Nesterov and allows non-Euclidean setups [1902.00990].

A distinct but closely related realization is the envelope-based variable-metric proximal point interpretation of QNing. For \(\kappa>0\), the Moreau envelope is
\[
F(x) = \min_{z \in \mathbb{R}^d} \left\{ f(z) + \frac{\kappa}{2}\|z-x\|^2 \right\},
\qquad
\nabla F(x) = \kappa(x-p(x)),
\]
where
\[
p(x) = \arg\min_{z \in \mathbb{R}^d} \left\{ f(z) + \frac{\kappa}{2}\|z-x\|^2 \right\}
\]
[1610.00960]. The paper applies quasi-Newton steps directly to the Moreau envelope of the full composite \(f=f_0+\psi\); it does not use the forward-backward envelope in its algorithmic core [1610.00960]. The exact variable-metric proximal point step with metric \(B_k=H_k^{-1}\) is
\[
z_k = \arg\min_{z} \left\{ f(z) + \frac{1}{2\eta}\|z-x_k\|_{B_k}^2 \right\},
\]
whereas QNing performs a quasi-Newton step on \(F\),
\[
x_{k+1} = x_k - H_k \nabla F(x_k),
\]
followed by a simple line search and a mixing with \(H_0=(1/\kappa)I\) under inexact gradients [1610.00960].

In the 2025 model-based proximal quasi-Newton formulation, the local model is any \(f_{x^k}\) obeying a local Taylor-like growth bound
\[
|f_{x^k}(x)-f(x)| \le \omega_{x^k}(\|x-x^k\|),
\]
and the subproblem is
\[
x^{k+1}\in\arg\min_{x}\Big\{\,f_{x^k}(x)+\tfrac{\gamma_k}{2}\|x-x^k\|_{H_k}^2\,\Big\}
\]
[2507.18363]. There, \(H_k\) comes from a continuous matrix generator \(H(x^k)\), only a uniform lower bound \(H_k\succeq \mu I\) is imposed a priori, and uniform boundedness of \(\{H_k\}\) arises a posteriori under KŁ regularity [2507.18363].

## 3. Inexactness models and implementable stopping criteria

The “inexact” component appears in several non-equivalent but related ways. In the variable-metric line-search framework, an \(\epsilon\)-type approximation accepts \(y_k\) if
\[
\frac{1}{\alpha_k} D_k (z_k - y_k) \in \partial_{\epsilon_k} f_1(y_k),
\]
with
\[
z_k := x_k - D_k^{-1}\nabla f_0(x_k),
\]
which yields
\[
h_\sigma(y_k,x_k) - h_\sigma(y^*,x_k) \le \epsilon_k,
\qquad
\|y_k-y^*\|^2 \le \alpha_k \mu \epsilon_k
\]
[1506.00385]. The same paper also provides an \(\eta\)-type approximation:
\[
P_\eta(x;h_\sigma) := \{y \in \Omega : h_\sigma(y,x) \le \eta\, h_\sigma(p(x;h_\sigma),x)\}, \quad \eta \in (0,1],
\]
and practical primal–dual-gap stopping rules for \(f_1(x)=g(Ax)\) [1506.00385].

In the inexact model framework, subproblem inexactness \(\widetilde{\delta}_k\) is encoded through Nemirovski’s inexact prox operator: for minimizing \(\Psi(x)\), an approximate solution \(\widetilde{x} \in \operatorname{Argmin}^{\widetilde{\delta}} \Psi(x)\) satisfies
\[
\exists h \in \partial \Psi(\widetilde{x}) \text{ such that } \langle h, x-\widetilde{x}\rangle \ge -\widetilde{\delta} \quad \forall x \in Q
\]
[1902.00990]. The paper treats model inexactness \(\delta_k\) and subproblem inexactness \(\widetilde{\delta}_k\) separately, and both appear additively in the complexity bounds [1902.00990].

QNing obtains inexact gradients through approximate proximal evaluations. At outer iterate \(x\), it approximately solves
\[
h(w) = f(w) + (\kappa/2)\|w-x\|^2,
\]
returns \(z \approx p(x)\), and sets
\[
g = \kappa(x-z), \qquad F_a = h(z)
\]
[1610.00960]. The recommended stopping criterion is
\[
h(z) - h^* \le (\kappa/36)\|z-x\|^2,
\]
which implies
\[
\|g - \nabla F(x)\| \le \sqrt{2\kappa\epsilon}
\]
when \(\epsilon = h(z)-h^*\) [1610.00960]. The paper also studies a fixed-budget heuristic, such as one pass over the data for SVRG [1610.00960].

In the fractional setting, the inexact optimality condition is
\[
\Delta^k\in \partial_{\delta_k}f(\bm{x}^{k+1})+\nabla h(\bm{x}^k)-c_k\bm{y}^k+H_k(\bm{x}^{k+1}-\bm{x}^k),
\]
with the practical error criterion
\[
\|\Delta^k\|^2+\big|\langle \Delta^k,\bm{x}^{k+1}-\bm{x}^k\rangle\big|+\delta_k\;\le\;\varepsilon_k\,g(\bm{x}^{k+1})
\]
[2504.11023]. The paper describes this criterion as verifiable and flexible, allowing progressive inexactness [2504.11023].

Inexactness can also be controlled at the modeling level rather than by a residual in the subproblem solve. In the general proximal quasi-Newton method, the accepted step satisfies
\[
|f(x^{k+1})-f_{x^k}(x^{k+1})| \le \delta \,\tfrac{\gamma_k}{2}\|x^{k+1}-x^k\|_{H_k}^2,
\]
and the paper explicitly states that “inexactness” is handled at the level of the local model/acceptance criterion rather than via a residual \(r_k\) in the subproblem [2507.18363].

In iR2N, inexactness encompasses inaccurate values of \(f\), \(\nabla f\), and the proximal step. The proximal inexactness requirement is the relative-norm lower bound
\[
\|s\|\;\ge\;\kappa_s\;\min\big\{\|s_{\text{cp}}\| : s_{\text{cp}}\in\operatorname{prox}_{\nu_k\,\psi(\cdot;x_k)}\big(-\nu_k\,\tilde\nabla f(x_k)\big)\big\},\qquad \kappa_s\in(0,1],
\]
and the paper emphasizes that this is weaker than many classical inexactness criteria and is implementable [2512.14507].

## 4. Descent mechanisms, line search, and global complexity

A recurrent feature is a sufficient-decrease mechanism. In the variable-metric forward–backward scheme, Armijo backtracking accepts \(\lambda^{(k)}\) if
\[
F\bigl(x_k + \lambda^{(k)} d_k\bigr) \le F(x_k) + \beta\, \lambda^{(k)}\, \Delta^{(k)},
\qquad
\Delta^{(k)} := \tilde h_{\sigma_k,\gamma}(y_k,x_k),
\]
and finite termination holds when \(\tilde h_{\sigma_k,\gamma}(y_k,x_k) < 0\) [1506.00385]. Under Lipschitz gradient of \(f_0\) and the adaptive error condition, there exists \(\lambda_{\min}>0\) such that \(\lambda^{(k)} \ge \lambda_{\min}\) for all \(k\) [1506.00385].

QNing uses a simple line search on \(\eta_k \in \{1,1/2,1/4,1/8,0\}\) to enforce
\[
F_{k+1} \le F_k - \frac{1}{4\kappa}\|g_k\|^2,
\]
and the paper states that this always terminates, with \(\eta_k=0\) sufficing in the worst case [1610.00960]. Under the adaptive inner accuracy criterion and sufficient decrease,
\[
F(x_{k+1}) \le F(x_k) - \frac{1}{32\kappa}\|\nabla F(x_k)\|^2
\]
[1610.00960].

In fractional optimization, the approximate descent inequality is
\[
F(\bm{x}^{k+1})\;\le\;F(\bm{x}^k)-\frac{\|\bm{x}^{k+1}-\bm{x}^k\|_{2H_k-L_hI_n}^2}{2\,g(\bm{x}^{k+1})}+\varepsilon_k,
\]
and under \(\sum_k \varepsilon_k < \infty\) one obtains
\[
F(\bm{x}^{k+1})\;\le\;F(\bm{x}^k)-\frac{a}{2}\|\bm{x}^{k+1}-\bm{x}^k\|^2+\varepsilon_k
\]
for some \(a>0\) [2504.11023]. The notable aspect, stated explicitly, is that the KL-based rate analysis is proved *without* requiring a strict sufficient descent property [2504.11023].

For manifold composite optimization, the accepted step \(v^{k,j_k}\) satisfies
\[
\Theta(R_{x^k}(v^{k,j})) \le \Theta_{k,j}(v^{k,j}) - (\bar\gamma/2)\|v^{k,j}\|^2,
\]
and the resulting outer descent inequality is
\[
\Theta(x^{k+1}) \le \Theta(x^k) - (\bar\gamma/2)\|v^k\|^2
\]
[2508.12003]. Under restricted level boundedness, the method returns an \(\epsilon\)-stationary point in at most
\[
K := \left\lceil \frac{2(\Theta(x^0)-\vartheta^*)}{\bar\gamma \chi^2 \epsilon^2} \right\rceil
\]
outer iterations, and the paper further states \(O(\epsilon^{-2})\) calls to the subproblem solver and \(O(\epsilon^{-4})\) oracle calls under a dual first-order inner solver [2508.12003].

For iR2N, the stationarity surrogate is
\[
\mathcal{S}(x_k)\;\defeq\;\nu_k^{-1}\|s\|,
\]
and the paper proves worst-case evaluation complexity \(O(\varepsilon^{-2})\) to reduce \(\mathcal{S}(x_k)\) below \(\varepsilon\) [2512.14507]. The total number of iterations satisfies
\[
|\mathcal{S}(\varepsilon)|+|\mathcal{U}(\varepsilon)|=O(\varepsilon^{-2})
\]
[2512.14507].

## 5. Convergence guarantees and KŁ-based rates

The convergence theory depends on the underlying problem class. For convex objectives in the variable-metric forward–backward method, the whole sequence converges to a minimizer under summable \(\epsilon_k\) and controlled metric variation \(D_{k+1} \preceq (1+\zeta_k)D_k\), and under Lipschitz gradient of \(f_0\) the paper gives the convergence rate estimate
\[
F(x_{k+1}) - F^* \le \frac{C}{k}
\]
[1506.00385]. For the more general nonconvex case, all limit points of the iterates sequence are stationary [1506.00385].

In QNing, if \(f\) is \(\mu\)-strongly convex, then the Moreau envelope \(F\) is \(\mu_F\)-strongly convex with \(\mu_F = (\mu\kappa)/(\mu+\kappa)\), its condition number is \(q=(\mu+\kappa)/\mu\), and the outer iterates satisfy the linear rate
\[
F(x_k)-F^* \le \left(1-\frac{1}{16q}\right)^k (F(x_0)-F^*)
\]
[1610.00960]. In the convex but not strongly convex case, the rate is
\[
F(x_k)-F^* \le \frac{32\kappa R^2}{k}
\]
under bounded level sets [1610.00960].

In the inexact model framework, the non-accelerated method satisfies
\[
f(\bar{x}_N) - f(x^*) \le \frac{R^2}{A_N} + \frac{1}{A_N}\sum_{k=0}^{N-1}\alpha_{k+1}\widetilde{\delta}_k + \frac{2}{A_N}\sum_{k=0}^{N-1}\alpha_{k+1}\delta_k,
\]
with \(A_N \ge N/(2L)\), and the accelerated method satisfies
\[
f(x_N)-f(x^*) \le \frac{8LR^2}{(N+1)^2} + \text{controlled additive error}
\]
[1902.00990]. For strongly convex objectives, restart schemes yield linear rates in terms of the Bregman divergence [1902.00990].

In the fractional algorithm, any accumulation point is a critical point under A1–A5, spectral bounds on \(H_k\), and \(\sum_k\varepsilon_k<\infty\) [2504.11023]. For the whole-sequence result, the paper introduces
\[
\Phi_{\tau}(\bm{x},t):=F(\bm{x})+\frac{t^\tau}{\tau},\quad \tau>1,
\]
and an error accumulation sequence \(s_k:=\tau\cdot\sum_{i=k}^{\infty}\varepsilon_i\) [2504.11023]. If \(\Phi_\tau\) has the KL property with exponent \(\theta_\tau\), then the paper gives explicit convergence rates: finite convergence when \(\theta_\tau=0\), linear or geometric rates when \(\theta_\tau\in(0,1/2]\) and \(\varepsilon_k\) decays geometrically, and sublinear rates for \(\theta_\tau\in(1/2,1)\) [2504.11023].

The 2025 general proximal quasi-Newton method proves that every accumulation point is stationary, and under the Kurdyka–Łojasiewicz property at a cluster point \(x^*\), the whole sequence converges to \(x^*\) with finite length
\[
\sum_{k=0}^\infty \|x^{k+1}-x^k\|<\infty
\]
[2507.18363]. If \(\varphi(t)=ct^{1-\theta}\), the rates are: finite termination for \(\theta=0\), superlinear convergence or finite termination for \(\theta\in(0,1/2)\), Q-linear convergence of \(f(x^k)\) and R-linear convergence of \(x^k\) for \(\theta=1/2\), and sublinear rates for \(\theta\in(1/2,1)\) [2507.18363].

For manifold composite optimization, every cluster point is stationary under restricted level boundedness, and if the augmented potential \(\Xi_{\tilde c}\) has the KL property on the cluster set, then \(\sum_k \|x^{k+1}-x^k\|<\infty\) and the iterates converge globally to a stationary point [2508.12003]. If the KL exponent is \(q=1/2\), the local rate is linear; if \(q\in(1/2,1)\), the paper gives explicit sublinear rates for both \(\Theta(x^k)-\vartheta^*\) and \(\|x^k-x^*\|\) [2508.12003].

## 6. Relations to proximal gradient, prox-linear, quasi-Newton, and application domains

The literature repeatedly distinguishes inexact variable metric proximal linearization from classical prox-linear and proximal quasi-Newton methods. Classical proximal quasi-Newton minimizes at each outer step a local quadratic model
\[
q_k(x) = f_0(x_k) + \langle\nabla f_0(x_k), x-x_k\rangle + \frac{1}{2}(x-x_k)^\top B_k(x-x_k) + \psi(x),
\]
where solving \(q_k\) can be nontrivial since \(B_k\) changes [1610.00960]. QNing avoids repeatedly changing the proximal metric in inner loops by smoothing \(f\) via its Moreau envelope and applying quasi-Newton steps to the smooth envelope [1610.00960].

The general model-function approach broadens this viewpoint to nonconvex and nonsmooth objectives \(f\), additive composites \(F(x)=h(x)+q(x)\), and composite structures \(F(x)=q(x)+h(A(x))\) [2507.18363]. The paper explicitly states four novel aspects relative to classic forward–backward, prox-linear, and quasi-Newton proximal methods: only local growth control near \(x^k\), an acceptance test based on model error, no a priori uniform boundedness of \(\{H_k\}\), and convergence to stationarity for nonconvex, nonsmooth problems [2507.18363].

The inexact-model framework of 2019 places proximal linearization within a larger class encompassing minimization, saddle-point problems, and variational inequalities [1902.00990]. It recovers proximal gradient, mirror descent, accelerated gradient methods, and generalized Mirror Prox as special cases, with variable metric/Bregman geometry entering through the prox function \(d\) and its divergence \(V\) [1902.00990].

The manifold extension shows that the same ideas persist when Euclidean updates are replaced by tangent-space subproblems and retractions. RiVMPL extends Euclidean prox-linearization to embedded manifolds using retractions and tangent spaces, and introduces a variable metric \(\mathcal{Q}_k\) that can incorporate curvature of \(F\) via \(\nabla F F'\) [2508.12003]. The paper compares this framework with Riemannian PG, manifold prox-QN, RiADMM, and RiALM [2508.12003].

Representative application domains in the cited papers include logistic regression, lasso, elastic-net, and large-scale empirical risk minimization [1610.00960]; \(\ell_1/\ell_2\) Lasso and constrained \(\ell_1/\ell_2\) sparse optimization [2504.11023]; polytope feasibility and sparse quadratic inverse problems [2507.18363]; sparse spectral clustering, constrained group sparse PCA, and proper symplectic decomposition [2508.12003]; total-variation image restoration [1506.00385]; and nonsmooth regularized problems with \(\ell_p\) norms, \(\ell_p\) total variation, and the indicator of the nonconvex pseudo \(p\)-norm ball [2512.14507].

A plausible implication is that “Inexact Variable Metric Proximal Linearization” is best understood not as a single algorithm but as a design pattern: local first-order modeling, adaptive geometry through a variable metric or Bregman generator, and implementable inexactness control through residuals, primal–dual gaps, model-acceptance tests, or approximate optimality conditions. The cited papers differ in objective structure, globalization mechanism, and stationarity measure, but they consistently use that pattern to balance curvature exploitation, subproblem tractability, and rigorous convergence analysis [1506.00385, 1610.00960, 1902.00990, 2504.11023, 2507.18363, 2508.12003, 2512.14507].

Source: https://www.emergentmind.com/topics/inexact-variable-metric-proximal-linearization