ICG: Advances in Conjugate Gradient Methods

• ICG is a suite of methods that adjust the standard conjugate gradient algorithm to achieve faster convergence, improved accuracy, and robust performance.
• It encompasses both direct modifications—such as altered recurrences and search directions—and indirect enhancements via better preconditioning and parallel strategies.
• Empirical studies demonstrate significant speedups in distributed, high-performance, and nonlinear optimization settings, validating ICG’s practical benefits over classical CG.

Searching arXiv for recent and relevant papers on improved conjugate gradient methods. Improved Conjugate Gradient (ICG) is not a single universally named algorithm but a broad designation for methods that modify, extend, or operationally enhance conjugate gradient (CG) in order to improve convergence behavior, attainable accuracy, numerical stability, communication cost, preconditioner effectiveness, or practical robustness in a given problem class. Across the literature, the term can refer either to direct changes in CG recurrences—such as modified search directions, spectral scaling, recursive residual formulas, or safeguarded conjugacy parameters—or to indirect improvements that leave the core CG iteration unchanged while improving preconditioning, basis construction, or distributed and parallel execution. The most precise reading is therefore contextual: in distributed adaptive estimation, ICG may denote a modified conjugate-gradient recursion with one iteration per time instant; in nonlinear optimization, it may denote a new Dai–Liao- or spectral-type parameterization; and in high-performance computing, it may denote a communication-hiding or $s$-step CG variant with improved finite-precision behavior (Xu et al., 2013).

1. Term, scope, and conceptual boundaries

The literature represented here makes clear that “Improved Conjugate Gradient” is best understood as a family resemblance term rather than a canonical algorithm name. Some papers explicitly state that they do not introduce a universally named algorithm called ICG, yet still present methods that modify standard CG “in exactly the kinds of ways often associated with practical improvements,” including recursive gradient updates, modified step sizes, improved basis recurrences, or lower-complexity single-pass implementations (Xu et al., 2013). Other papers are equally explicit that their contribution is not a new CG recurrence, but an improvement to preconditioned CG through parameter tuning, learned preconditioners, or communication-aware implementation (Katrutsa et al., 2018).

A useful distinction runs through the literature. A first class consists of direct CG modifications, where the search direction, conjugacy parameter, spectral coefficient, or residual recursion is changed. This includes modified Dai–Liao spectral CG methods, modified secant-based spectral CG, Lipschitz-related restart rules, and adaptive distributed modified CG recursions (Sahu et al., 25 Jan 2025). A second class consists of indirect CG improvements, where standard CG or PCG remains intact but is enhanced through better preconditioners, improved pipelining, stochastic parameter selection, enlarged subspaces, or superiorization (Häusner et al., 2023).

This suggests that the defining feature of ICG is not a single formula but a design objective: preserve the low-memory and Krylov-subspace structure of CG while improving one or more of the following problem-dependent attributes: convergence speed, robustness under nonconvexity or nonsmoothness, numerical stability in finite precision, communication efficiency on distributed hardware, or application-level performance such as EMSE in sensor networks or reconstruction quality in imaging.

2. Distributed adaptive and recursive ICG forms

A clear example of ICG in the adaptive-filtering sense is the modified conjugate gradient (MCG) algorithm for distributed parameter estimation over sensor networks. The problem is cooperative estimation of an unknown vector $\boldsymbol{\omega}_0$ from local observations under the quadratic objective

$$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$

with normal equations $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$ and distributed operation under incremental or diffusion cooperation protocols (Xu et al., 2013).

In the baseline distributed conventional CG (CCG), each node performs $J$ inner iterations at each time instant. The modified CG (MCG) replaces this by a single iteration per time instant, which is the paper’s central practical improvement. The key changes are a recursive negative-gradient update,

$$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$

a modified step size

$$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$

and a Polak–Ribiere-type direction parameter

$$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$

The paper characterizes MCG as non-reset, lower complexity, recursively updated, and empirically better in EMSE (Xu et al., 2013).

The same framework supports both incremental and diffusion implementations. In the incremental mode, information is passed cyclically from one node to the next. In the diffusion mode, each node first combines neighbors’ estimates using the Metropolis rule and then adapts. The diffusion MCG inserts the combined estimate

$$\boldsymbol{\phi}_k^{(i-1)}=\sum_{l\in N_{k,i-1}} c_{kl}\boldsymbol{\psi}_l^{(i-1)}$$

into the update and gradient recursion. The resulting distributed algorithms are described as cooperative, real-time, and adaptive to nonstationary environments (Xu et al., 2013).

The improvement is also quantified in complexity terms. For the incremental case, IDCCG has $J$-dependent cost,

$\boldsymbol{\omega}_0$0

$\boldsymbol{\omega}_0$1

whereas IDMCG has fixed per-update complexity,

$\boldsymbol{\omega}_0$2

The simulation study on a 20-node sensor network reports that IDMCG and IDCCG outperform IDLMS, that IDMCG is very close to IDRLS, and that analogous improvements hold in diffusion mode as well (Xu et al., 2013).

3. Communication-avoiding, pipelined, and $\boldsymbol{\omega}_0$3-step improvements

In high-performance computing, ICG often means improving CG under the dominant cost model of global synchronization. Classic CG requires global reductions every iteration, and communication-hiding or pipelined variants attempt to overlap these reductions with matrix-vector products and local vector operations. A major practical difficulty is finite-precision instability, especially in deep pipelines (Cools et al., 2019).

A numerically stable pipelined $\boldsymbol{\omega}_0$4-CG variant addresses this by replacing the unstable multi-term basis recurrence with a chain of $\boldsymbol{\omega}_0$5 short recurrences using intermediate auxiliary bases $\boldsymbol{\omega}_0$6. The new method retains the same communication-hiding structure, one SpMV per iteration, and the same number of global reductions as earlier pipelined CG, but removes the error-amplification mechanism involving $\boldsymbol{\omega}_0$7. The central theoretical conclusion is that the basis gap in the stabilized method is only a sum of local rounding errors and no longer contains the transformation inverse that caused amplification in the original pipelined recurrence (Cools et al., 2019).

The paper reports that on a $\boldsymbol{\omega}_0$8 2D Laplace problem, stable $\boldsymbol{\omega}_0$9-CG achieved about $$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$0 speedup over single-node classic CG on 32 nodes, while attaining about $$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$1 maximal accuracy, comparable to classic CG’s $$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$2. In a time-to-accuracy view, maximal accuracy was reached in roughly 1.28 s for $$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$3-CG versus 13.9 s for classic CG. On a nonlinear ice-sheet flow problem, pipelined methods achieved about $$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$4 speedup over classic CG on 128 nodes, and stable $$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$5-CG reached a tighter inner tolerance that unstable pipelined CG could not (Cools et al., 2019).

A related line of work improves communication-hiding CG by predict-and-recompute recurrences. In PR-CG and Pipe-PR-CG, recursively updated quantities are first used as predictors and later recomputed exactly within the same iteration. For example, PR-CG predicts

$$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$6

uses it to form $$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$7, and then recomputes

$$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$8

exactly later in the iteration. In pipelined form, the same idea is applied to $$J(\boldsymbol{\omega}) = E\!\left[\|\boldsymbol{d} - \boldsymbol{X}\boldsymbol{\omega}\|^2\right],$$9, with $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$0 used as a predictor and $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$1 recomputed exactly. The paper’s rounding-error analysis shows that, with recomputation, the predictor gaps depend only on local rounding errors rather than accumulated ones, which explains why the new variants maintain convergence and attainable accuracy much closer to standard Hestenes–Stiefel CG than earlier pipelined alternatives (Chen et al., 2019).

A third communication-reduction strand is adaptive $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$2-step CG with dynamic basis updating. Here the improvement is achieved by using iteratively updated estimates of the largest and smallest Ritz values to set the parameters of Newton or Chebyshev polynomial bases used within each $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$3-step block. Those estimates are also used to replace a previously user-chosen heuristic parameter by an automatically computed quantity

$\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$4

where $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$5 and $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$6 are Ritz-based estimates. This improves both basis conditioning and the admissible block length. On the 494bus problem, for example, the improved adaptive method kept total iterations essentially at HSCG levels while reducing outer-loop iterations from 413 to as low as 32 for $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$7, which the paper interprets as over $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$8 fewer synchronizations (Carson, 2019).

4. Preconditioning-based and data-driven ICG

A large portion of the ICG literature improves CG indirectly by improving preconditioning rather than the CG recurrence itself. One example is stochastic optimization of preconditioner parameters for SPD systems $\boldsymbol{b}=\boldsymbol{R}\boldsymbol{\omega}_o$9. Instead of minimizing the condition-number-based functional

$J$0

the stochastic approach defines

$J$1

using trial CG runs from random initial guesses. The paper argues that condition-number minimization is a worst-case criterion, whereas $J$2 better captures average finite-iteration CG behavior. For $J$3, the optimized stochastic parameters are consistently smaller than the condition-number-based optima; for example, with $J$4 and constant coefficients, the reported optima are $J$5 versus $J$6. The method is derivative-free, uses Brent’s method, and is most useful when many systems share the same matrix (Katrutsa et al., 2018).

Another direction replaces hand-designed incomplete factorizations by learned ones. Neural incomplete factorization (NeuralIF) learns a sparse lower-triangular factor

$J$7

and forms the SPD preconditioner

$J$8

Training minimizes a stochastic approximation of the Frobenius factorization error rather than the condition number, using only matrix-vector products via a Hutchinson estimator. The output is SPD by construction because the diagonal is parameterized as

$J$9

On synthetic $$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$0 SPD systems, NeuralIF achieved 267.08 iterations and 1.19 s total time with 100% success, compared with IC(0) at 260.64 iterations, 1.40 s total time, and 96% success; the sparsified NeuralIF-sp variant reached 286.02 iterations and 1.03 s total time (Häusner et al., 2023).

These methods do not alter CG search directions, step sizes, or conjugacy formulas. Their relevance to ICG is therefore architectural rather than algorithmic in the narrow Krylov sense. They improve practical CG performance through better-conditioned spectra, lower setup cost, cheaper or more reliable preconditioner construction, and, in the learned case, avoidance of incomplete-Cholesky-style breakdown (Häusner et al., 2023).

5. Nonlinear optimization variants

In nonlinear unconstrained optimization, ICG typically refers to direct modifications of nonlinear CG formulas. A prominent recent example is the modified descent Dai–Liao spectral conjugate gradient method (MDDLSCG). For

$$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$1

the method replaces the usual secant vector $$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$2 by

$$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$3

with

$$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$4

so that

$$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$5

It then defines the modified Dai–Liao parameter

$$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$6

with

$$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$7

and uses the spectral direction

$$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$8

With safeguarded $$\boldsymbol{g}_k^{(i)}= \lambda_f \boldsymbol{g}_{k-1}^{(i)}-\alpha_k^{(i)} \boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}+\boldsymbol{x}_k^{(i)}\!\left[d_k^{(i)}-\boldsymbol{x}_k^{(i)H}\boldsymbol{\omega}_{k-1}^{(i)}\right],$$9, the direction satisfies

$$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$0

and the paper proves

$$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$1

under bounded level set, Lipschitz gradient, and strong Wolfe conditions (Sahu et al., 25 Jan 2025).

The paper reports that on the Beale function, MDDLSCG took 21 iterations and $$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$2 s, compared with 30 iterations for MSCG and 139 for ScCG. In compressed sensing tests with $$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$3, MDDLSCG required 272 iterations and $$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$4 s, versus 493 and $$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$5 s for MSCG and 3653 and 1.14 s for ScCG (Sahu et al., 25 Jan 2025).

A related spectral ICG is SCGMMWLS, which constructs modified secant vectors

$$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$6

with

$$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$7

and combines them with a modified Wolfe line search to retain useful negative $$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$8 values while preserving the curvature condition. The method uses

$$\alpha_k^{(i)}= \eta\, \frac{\boldsymbol{p}_k^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}{\boldsymbol{p}_k^{(i)H}\boldsymbol{R}_k^{(i)}\boldsymbol{p}_k^{(i)}}, \qquad (\lambda_f-0.5)\le \eta \le \lambda_f,$$9

with truncated spectral $$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$0 and safeguarded $$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$1, and proves

$$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$2

The numerical study reports that $$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$3 is best overall, winning about 85% of test problems among the secant-order choices, and that the full proposed method outperforms the DK and Jian baselines on Andrei’s test set (Wu et al., 2023).

Other nonlinear ICG interpretations include a modified Dai–Liao / HS hybrid parameter with restart property,

$$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$4

with

$$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$5

and hybrid cubic regularization of CGM, where poor directions are selectively repaired using explicit regularized memoryless-BFGS inverse formulas rather than reset outright (Alhawarat, 2023).

6. Applications, limitations, and interpretation

The application range of ICG is broad but segmented. Distributed adaptive MCG targets sensor-network estimation and system identification, where improvement is measured in EMSE, complexity, and real-time tracking (Xu et al., 2013). Pipelined, predict-and-recompute, enlarged, and adaptive $$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$6-step variants target large sparse SPD systems on distributed-memory hardware, where synchronization cost and attainable accuracy dominate (Cools et al., 2019). Spectral and Dai–Liao variants target large-scale smooth nonlinear optimization and have been applied to compressed sensing and signal processing (Sahu et al., 25 Jan 2025). Superiorized CG variants target image reconstruction, using TV as a secondary criterion while preserving least-squares CG structure (Zibetti et al., 2017). Learned or stochastic preconditioning methods are most useful when many related SPD systems are solved repeatedly (Katrutsa et al., 2018).

A recurring limitation is scope specificity. Distributed MCG is not a drop-in replacement for classical batch CG; it assumes streaming data, forgetting factors, and network cooperation (Xu et al., 2013). Pipelined and $$\beta_k^{(i)}= \frac{(\boldsymbol{g}_k^{(i)}-\boldsymbol{g}_{k-1}^{(i)})^H\boldsymbol{g}_k^{(i)}} {\boldsymbol{g}_{k-1}^{(i)H}\boldsymbol{g}_{k-1}^{(i)}}.$$7-step methods improve wall-clock scalability but may require extra storage, auxiliary bases, or basis-conditioning safeguards (Cools et al., 2019). Learned and stochastic preconditioning improve PCG performance without changing the Krylov recurrence, so they are only “improved CG” in an indirect sense (Häusner et al., 2023). Nonlinear ICG variants typically still rely on strong Wolfe line search for global convergence, even when sufficient descent of the direction itself is built into the update (Sahu et al., 25 Jan 2025).

One common misconception is that ICG must denote a single classical algorithm analogous to Fletcher–Reeves or Hestenes–Stiefel. The literature here does not support that reading. Another is that every CG improvement should be judged purely by iteration count. In parallel and communication-avoiding settings, the dominant metric may instead be outer-loop count, synchronization count, or maximal attainable accuracy (Carson, 2019). A third misconception is that preconditioning-based or superiorization-based work is external to ICG. The papers consistently treat such methods as legitimate CG improvements when the operational objective is end-to-end solver performance rather than alteration of the exact recurrence (Katrutsa et al., 2018).

Taken together, these works support a broad but technically coherent interpretation: Improved Conjugate Gradient denotes problem-tailored enhancements of the CG paradigm, ranging from direct recurrence modifications to indirect architectural improvements, whose common goal is to preserve the structural advantages of CG while correcting a specific bottleneck—complexity, stability, communication, curvature representation, preconditioning quality, or application-level reconstruction or estimation performance (Sahu et al., 25 Jan 2025).