Proximal Conjugate Gradient Method

• Proximal Conjugate Gradient (PCG) methods are composite optimization techniques that integrate a proximal treatment for nonsmooth terms with conjugate gradient search strategies.
• They address problems of the form f(x)=g(x)+h(x), using constructs like the forward-backward residual to replace the classical gradient in direction updates.
• Variants include Hessian-free implementations and active-set approaches, offering convergence guarantees and enhanced performance in large-scale or nonsmooth scenarios.

“Proximal Conjugate Gradient Method” denotes a family of composite-optimization algorithms that combine a proximal treatment of a nonsmooth term with conjugate-gradient structure in the search direction or in an inner Newton solve. The term is not used uniformly. In several papers, “PCG” refers instead to the classical preconditioned conjugate gradient method for symmetric positive definite linear systems, explicitly not to proximal methods (Bake et al., 13 Oct 2025). Within optimization, the proximal-CG label is used more narrowly for methods that minimize objectives of the form $f(x)=g(x)+h(x)$, where $g$ is smooth and $h$ is proximable, and that replace the gradient-based search mechanism of nonlinear CG by a proximal object such as the forward-backward residual, or else use CG to construct a Newton-like proximal update without explicit Hessian formation (Hamana et al., 13 Apr 2026).

1. Terminology and scope

The expression “PCG” is terminologically overloaded. In numerical linear algebra, multiple papers in the present corpus state that PCG means preconditioned conjugate gradient and does not mean proximal-gradient methods or proximal conjugate gradient (Bake et al., 13 Oct 2025). The same clarification appears in work on nonsymmetric multigrid preconditioning, where “PCG” denotes standard or flexible preconditioned conjugate gradient recurrences for $Ax=b$ with $A$ symmetric positive definite (Bouwmeester et al., 2012).

In optimization, by contrast, the proximal-CG viewpoint concerns composite minimization problems such as

$\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$

with $g$ continuously differentiable and $h$ proper, lower semicontinuous, and proximable (Hamana et al., 13 Apr 2026). A second line of work studies

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$

where $q\in C^2$ has $g$0-Lipschitz continuous gradient and both $g$1 and $g$2 may be nonconvex, while the proximal operator of $g$3 remains computable (Zhou et al., 19 Sep 2025). These formulations place proximal CG between proximal gradient, proximal Newton or quasi-Newton methods, and nonlinear conjugate gradient.

The literature also contains methods with only partial CG content. One paper titled “Distributed accelerated proximal conjugate gradient methods” is explicit that it does not actually propose a classical proximal conjugate gradient method in the standard optimization sense, but rather distributed accelerated proximal gradient methods with a momentum-like direction variable (Gebrie, 2023). Another work, “A Riemannian Proximal Newton-CG Method,” is described more accurately as a proximal Newton method whose inner Newton system is solved approximately by truncated CG, rather than as a standalone proximal nonlinear CG method (Huang et al., 2024). This suggests that “proximal conjugate gradient” is best understood as a family resemblance term rather than a single canonical algorithm.

2. Core composite-optimization framework

The clearest proximal nonlinear CG formulation in the corpus is the method for minimizing

$g$4

where $g$5 is continuously differentiable and $g$6 is proper and lower semicontinuous (Hamana et al., 13 Apr 2026). In the main convex case, $g$7 is convex and $g$8 satisfies

$g$9

for some $h$0. The same paper also extends the method to the case where $h$1 is $h$2-weakly convex, meaning

$h$3

with proximal parameter restricted by

$h$4

The central replacement for the gradient is the forward-backward residual. For $h$5, define

$h$6

and

$h$7

At iteration $h$8,

$h$9

This object plays the role of the gradient in three precise senses. First, if $Ax=b$0, then

$Ax=b$1

Second, in the convex case,

$Ax=b$2

and in the weakly convex case,

$Ax=b$3

Third, the residual satisfies

$Ax=b$4

in the convex case. This gives proximal CG a direct stationarity measure that reduces exactly to the smooth gradient in the limit $Ax=b$5 (Hamana et al., 13 Apr 2026).

A separate Hessian-free line of work keeps the same composite structure but allows both terms to be nonconvex: $Ax=b$6 with $Ax=b$7, $Ax=b$8-Lipschitz continuous gradient, proper lsc $Ax=b$9, and coercive full objective: $A$0 There the proximal gradient mapping

$A$1

is used as the first-order criticality device, with $A$2 iff $A$3 is first-order critical when the mapping is single-valued (Zhou et al., 19 Sep 2025).

3. Forward-backward-residual proximal nonlinear CG

The proximal nonlinear conjugate gradient method of (Hamana et al., 13 Apr 2026) defines the search direction by a three-term Hestenes–Stiefel-type recursion: $A$4 with

$A$5

and

$A$6

where

$A$7

The safeguard is

$A$8

and one concrete choice is

$A$9

A key exact identity is

$\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$0

This is the composite analogue of a sufficient-descent relation, built directly into the three-term formula. The proximal parameter $\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$1 is chosen as the largest value in

$\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$2

such that

$\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$3

The method uses a two-stage line search and a fallback proximal-gradient step. First, it searches over $\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$4, $\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$5, for the largest $\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$6 satisfying

$\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$7

with $\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$8, $\min_{x\in\mathbb{R}^n} f(x):=g(x)+h(x),$9. If $g$0, $g$1, it then backtracks on $g$2 until

$g$3

If no suitable trial satisfies the first condition with $g$4, the method switches to the proximal-gradient step

$g$5

This switching mechanism is part of the convergence proof, and the paper explicitly notes as future work the development of a version that guarantees convergence without this switching mechanism (Hamana et al., 13 Apr 2026).

The main convergence statement is global. Under Lipschitz gradient for $g$6, proper lsc convex $g$7, and bounded below $g$8, the generated sequence satisfies

$g$9

If $h$0 is bounded, every accumulation point is stationary: $h$1 The proof relies on the uniform decrease estimate

$h$2

for some $h$3, whether the step is a CG step or the fallback proximal-gradient step. In the weakly convex extension, the same residual convergence holds,

$h$4

and bounded accumulation points are Fréchet stationary: $h$5 Under additional strong convexity assumptions, the paper proves

$h$6

for some constant $h$7 (Hamana et al., 13 Apr 2026).

A defining structural property is exact reduction to classical nonlinear CG when $h$8. In that case,

$h$9

and under strong convexity with $$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$0, one may take $$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$1, so

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$2

The direction then becomes the standard three-term HS direction,

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$3

equivalent to the Zhang–Zhou–Li three-term HS formula (Hamana et al., 13 Apr 2026). This makes the method a genuine proximal analogue of a nonlinear CG scheme rather than merely a heuristic modification.

4. Hessian-free and second-order proximal CG variants

A distinct proximal-CG construction appears in the “Scalable Hessian-free Proximal Conjugate Gradient Method” (Zhou et al., 19 Sep 2025). Its starting point is the local second-order model

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$4

with

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$5

and the Newton system

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$6

Instead of solving this system exactly, the method applies CG with initialization

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$7

and recurrences

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$8

$$\min_{\bm{x}\in\mathbb{R}^n} f(\bm{x}) := q(\bm{x}) + h(\bm{x}),$$9

$q\in C^2$0

The inner loop terminates on negative curvature if

$q\in C^2$1

The method uses CG not only to form a Newton-like direction but also to extract local spectral information through the tridiagonal matrix

$q\in C^2$2

whose entries are defined from the CG coefficients and whose eigenvalues are the Ritz values of $q\in C^2$3 on the generated Krylov subspace: $q\in C^2$4 The largest Ritz value yields the step-size estimate

$q\in C^2$5

A proximal gradient candidate is then formed as

$q\in C^2$6

and the CG refinement continues until

$q\in C^2$7

holds.

The paper’s key innovation is a curvature-aware isotropic surrogate: $q\in C^2$8 where the Cauchy step length is

$q\in C^2$9

The ratio $g$00 scales the CG direction “to better preserve majorization after the proximal step and enable further approximation refinement” (Zhou et al., 19 Sep 2025). Because the surrogate is isotropic, the method keeps the subproblem in standard proximal form: $g$01 Acceptance is based on the majorization test

$g$02

The same paper proves monotone descent: $g$03 hence boundedness and vanishing steps,

$g$04

and concludes that every accumulation point of the generated sequence is a critical point of $g$05 (Zhou et al., 19 Sep 2025). This variant is explicitly Hessian-free, since curvature is accessed only through Hessian-vector products

$g$06

computed by automatic differentiation as

$g$07

A related but distinct construction is the “Preconditioned proximal gradient method with conjugate momentum,” which solves

$g$08

by combining a preconditioned proximal-gradient direction

$g$09

with a momentum direction $g$10 derived from the previous iterate, and then minimizing a proximal Newton model on the two-dimensional subspace

$g$11

after orthogonalization with respect to the Hessian-induced metric (Chen et al., 17 Mar 2026). That paper is explicit that the method is better described as a preconditioned proximal-gradient / subspace-proximal-Newton hybrid with a CG-like conjugate momentum mechanism, rather than as a classical proximal analogue of linear or nonlinear conjugate gradient. This suggests a broader proximal-CG ecology in which conjugacy may enter either through an explicit direction recursion or through low-dimensional Hessian-orthogonalized subspace updates.

5. Active-set and face-restricted CG for nonsmooth regularization

A third line, highly relevant to a proximal-CG interpretation, concerns generalized CG methods for

$g$12

with $g$13, possibly singular, and nonempty optimal solution set (Lu et al., 2015). The paper does not present a standard proximal-CG algorithm in the forward-backward-residual sense. Instead, it develops CG-based active-set / face-identification methods for composite nonsmooth optimization.

The pivotal geometric object is the minimum-norm subgradient $g$14, characterized componentwise by

$g$15

Optimality is

$g$16

On an orthant face, the $g$17-term becomes linear, so the composite objective reduces there to a smooth convex quadratic. The algorithms therefore alternate between releasing zero variables and running projected or truncated CG on the currently identified face.

The projected minimum-norm subgradient is

$g$18

and the decision between a release step and a face-CG subroutine is based on comparing the sizes of the zero and nonzero components of the subgradient: $g$19 If this condition holds, the algorithm takes the exact line-search step

$g$20

Otherwise it calls a truncated projected CG routine on the current face.

The CG subroutine solves face-restricted smooth quadratics of the form

$g$21

and either finds an approximate minimizer on the current face or stops when a CG iterate crosses the boundary, thereby identifying a smaller face. One variant, GCG4, introduces an explicit proximal step after the CG phase: $g$22 with closed form

$g$23

where

$g$24

This line of work is especially notable for finite-convergence and complexity claims. For the inexact face-CG subroutines, the number of PCG iterations needed is logarithmic in $g$25, and the paper states that the number of PCG iterations executed by GCG2, GCG2v, GCG3, and GCG4 depends on $g$26 as

$g$27

It then concludes that the arithmetic operation cost for computing an $g$28-optimal solution satisfies

$g$29

contrasted with accelerated proximal gradient methods, whose dependence is

$g$30

This suggests that, for $g$31-regularized convex quadratics, active-set CG can serve as a rigorous alternative to full-space proximal gradient when the orthant-face structure is exploited directly (Lu et al., 2015).

6. Related formulations, applications, and persistent ambiguities

Several neighboring methods illuminate the boundaries of the proximal-CG concept. The distributed multi-agent method of (Gebrie, 2023) studies

$g$32

with local proximal steps for $g$33, gradient steps for $g$34, Halpern fixed-point updates for $g$35, and inertial extrapolation. Its direction recursion,

$g$36

resembles nonlinear CG formally, but $g$37 is an externally prescribed scalar sequence rather than a Fletcher–Reeves, Polak–Ribière, or Hestenes–Stiefel coefficient. The paper states that this is better viewed as a proximal-gradient–type distributed algorithm with a momentum direction variable, not a full classical proximal nonlinear conjugate-gradient method (Gebrie, 2023).

On manifolds, “A Riemannian Proximal Newton-CG Method” addresses

$g$38

where $g$39 is a compact embedded submanifold. Its outer step is a Riemannian proximal Newton method, while the inner Newton correction is computed approximately by truncated CG on a tangent-space quadratic problem. The tangent-space proximal-gradient model is

$g$40

and the Newton correction is reduced to the quadratic problem

$g$41

The paper explicitly distinguishes this from a standalone proximal nonlinear CG method: the CG iteration is an inner solver for a Newton-related tangent-space problem, not the principal search-direction mechanism (Huang et al., 2024).

The same ambiguity appears in applications. A learned-preconditioner paper for PDE solvers studies preconditioned conjugate gradient for sparse SPD systems and treats the selection of the preconditioner as a graph-neural-network-based operator-learning problem, with learned sparse factorization

$g$42

embedded into standard PCG recurrences (Li et al., 2023). This is unrelated to proximal methods, but its use of the abbreviation “PCG” is typical of the dominant linear-algebra meaning.

Across the literature, one objective conclusion is therefore unavoidable: “Proximal Conjugate Gradient Method” does not name a single universally accepted algorithm. The most technically precise uses in the present corpus are the forward-backward-residual three-term HS method for composite optimization (Hamana et al., 13 Apr 2026), the Hessian-free CG-majorization method for nonconvex nonsmooth problems (Zhou et al., 19 Sep 2025), and the generalized face-restricted CG framework for $g$43-regularized quadratic programming (Lu et al., 2015). Other methods employ CG only as an inner Newton solver or use “conjugate” in a looser momentum or subspace sense. This suggests that the term is best reserved for methods in which proximal structure and conjugate-gradient structure are both essential, rather than merely adjacent.