Adaptive Nesterov Accelerated Proximal Gradient

• Adaptive Nesterov Accelerated Proximal Gradient (adaNAPG) is a family of methods that integrates adaptive control of stepsize, curvature surrogates, and gradient accuracy into the proximal gradient framework.
• The approach unifies multiple adaptive mechanisms—such as adaptive sampling and inexact gradient estimation—to achieve optimal iteration and sample complexities in both convex and strongly convex regimes.
• Experimental results in applications like logistic regression and inventory optimization show that adaNAPG can match or outperform traditional accelerated proximal methods while requiring fewer samples.

Adaptive Nesterov Accelerated Proximal Gradient (adaNAPG) denotes a family of composite first-order methods that preserve the extrapolated proximal structure of Nesterov accelerated proximal gradient while adapting some combination of stepsize, curvature surrogate, gradient accuracy, sampling budget, or restart logic. The name is explicit in "Boosting Accelerated Proximal Gradient Method with Adaptive Sampling for Stochastic Composite Optimization" (Zhu et al., 24 Jul 2025), whereas several closely related methods appear under other names, including the accelerated variant in "Universal Adaptive Proximal Gradient Methods via Gradient Mapping Accumulation" (Wang et al., 7 May 2026), adaptive inexact APG (Bollapragada et al., 19 Jul 2025), projected Nesterov proximal gradient (Gu et al., 2015), and adaptive FISTA (Ochs et al., 2017). In this literature, the unifying object is the composite proximal-gradient update and its associated gradient mapping, rather than a single universally fixed momentum or stepsize recursion.

1. Composite optimization setting and stationarity measure

The standard optimization model is the composite problem

$\min_{x\in\mathbb{R}^d}\ \phi(x)=f(x)+h(x),$

where $f$ is continuously differentiable and $L$-smooth, and $h$ is closed, convex, proper, and prox-friendly, with efficiently computable proximal operator $\prox_{\alpha,h}(y)$ (Bollapragada et al., 19 Jul 2025). In stochastic composite formulations one often has

$f(x)=\mathbb{E}[\tilde f(x;\xi)],$

and the smooth term may be convex or strongly convex, typically summarized as $f\in\mathcal S_{\mu,L}$ (Zhu et al., 24 Jul 2025). Other variants broaden the same template to nonconvex smooth $f$, convex nonsmooth $f$, multiobjective composite optimization, or composite models with nonconvex penalties (Wang et al., 7 May 2026, Huang, 9 Jul 2025, Yang et al., 2020).

The canonical stationarity object is the proximal gradient mapping

$$G_\alpha(x):=\frac{1}{\alpha}\Bigl(x-\operatorname{prox}_{\alpha h}(x-\alpha\nabla f(x))\Bigr),$$

or closely related residuals such as

$f$0

This choice is structurally important in composite optimization because, unlike the raw gradient, the gradient mapping vanishes at stationary or optimal points of the composite problem; the 2026 universal adaptive framework makes this point explicitly and builds its adaptive denominator from gradient-mapping-like proximal displacements rather than plain gradients (Wang et al., 7 May 2026).

2. Canonical accelerated proximal template and sources of adaptivity

The baseline Nesterov accelerated proximal gradient template uses an extrapolated point $f$1, a proximal update at that point, and a momentum step. In exact-gradient form this is written as

$f$2

which is the baseline NAPG structure adopted before adaptive modification in the stochastic composite adaNAPG paper (Zhu et al., 24 Jul 2025). In adaptive inexact APG, the same outer structure appears with a gradient estimate $f$3,

$f$4

with convex and strongly convex momentum schedules chosen in the standard Nesterov style (Bollapragada et al., 19 Jul 2025).

What changes across adaNAPG-type methods is the adaptive control law. In the universal adaptive accelerated scheme, the auxiliary proximal sequence satisfies

$f$5

and the effective stepsize is

$f$6

equivalently

$f$7

The paper contrasts this directly with standard APG/FISTA, which uses a fixed stepsize $f$8 and does not adapt to noise or unknown problem structure (Wang et al., 7 May 2026).

A different adaptive mechanism appears in adaptive gradient-estimation APG. There, the method does not primarily adapt the momentum; instead it imposes an iteration-dependent inexactness condition such as

$f$9

in the finite-sum case, or its conditional second-moment analogue in the stochastic expectation case. For unbiased estimators, sample sizes are then chosen to satisfy that bound, so the gradient accuracy increases as the iterates approach a solution (Bollapragada et al., 19 Jul 2025).

3. The explicitly named adaNAPG algorithm

The paper that explicitly introduces the name adaNAPG studies stochastic composite optimization

$L$0

with $L$1, $L$2 proper, closed, convex, and unbiased stochastic gradients satisfying finite variance assumptions. Its baseline NAPG iteration is retained, but the exact gradient at the extrapolated point $L$3 is replaced by a mini-batch estimator

$L$4

where the batch size $L$5 is increased until two adaptive tests are satisfied: a projection/inner-product-type test and an orthogonality/residual-type test. Conceptually these tests enforce that the stochastic gradient error is small relative to the current stationarity measure $L$6. The method uses the constant stepsize

$L$7

updates the proximal point by

$L$8

and, in the strongly convex case, simplifies the extrapolation to

$L$9

The central claim is that this adaptive-sampling Nesterov proximal scheme achieves the optimal iteration complexity of accelerated first-order methods while also attaining optimal sample complexity in the strongly convex regime (Zhu et al., 24 Jul 2025).

Its iteration guarantees are the classical accelerated ones. In the convex case $h$0,

$h$1

equivalently $h$2 iterations for $h$3-accuracy. In the strongly convex case $h$4, the paper proves linear convergence,

$h$5

with

$h$6

For total sample usage $h$7, it further shows under uniformly bounded variance that

$h$8

and derives a central limit theorem of the form

$h$9

The experimental section uses regularized logistic regression and a large inventory optimization problem, and reports that adaNAPG matches or beats geometric batch growth with fewer samples in strongly convex logistic regression and matches polynomial-growth accelerated methods while using fewer samples in convex problems (Zhu et al., 24 Jul 2025).

4. Major design variants in the adaNAPG family

The recent literature shows that "adaptive" in accelerated proximal gradient methods is not tied to a single mechanism. The following variants are representative.

Variant	Adaptive ingredient	Stated guarantee
Universal accelerated proximal method (Wang et al., 7 May 2026)	Denominator $\prox_{\alpha,h}(y)$0 built from proximal displacements	Smooth convex stochastic rate $\prox_{\alpha,h}(y)$1
Adaptive inexact APG (Bollapragada et al., 19 Jul 2025)	Gradient-estimation accuracy controlled relative to $\prox_{\alpha,h}(y)$2	Optimal proximal-iteration complexity; with unbiased estimates, optimal stochastic-gradient complexity
Multiobjective accelerated proximal method (Huang, 9 Jul 2025)	Deterministic Lipschitz estimate recursion $\prox_{\alpha,h}(y)$3 with $\prox_{\alpha,h}(y)$4 and no line search	$\prox_{\alpha,h}(y)$5
AAPG/AAPG-SPIDER (Yuan, 28 Feb 2025)	Adaptive diagonal metric $\prox_{\alpha,h}(y)$6 and Nesterov extrapolation	$\prox_{\alpha,h}(y)$7 full batch; $\prox_{\alpha,h}(y)$8 with SPIDER

The universal accelerated composite method is especially close to classical adaNAPG in form. It combines an $\prox_{\alpha,h}(y)$9 Nesterov/AC-SA extrapolation scheme with an AdaGrad-like denominator built from accumulated norms of a gradient-mapping-like proximal displacement. The paper emphasizes that implementation requires no prior knowledge of $f(x)=\mathbb{E}[\tilde f(x;\xi)],$0, although the accelerated convex analysis assumes bounded auxiliary iterates through a known diameter bound $f(x)=\mathbb{E}[\tilde f(x;\xi)],$1 (Wang et al., 7 May 2026).

The adaptive inexact APG work modifies a different axis of the algorithmic stack. There the outer APG recursion is standard, but the gradient estimate can be biased or unbiased and is made more accurate as the residual shrinks. The paper analyzes nonconvex, convex, and strongly convex regimes, proving $f(x)=\mathbb{E}[\tilde f(x;\xi)],$2, $f(x)=\mathbb{E}[\tilde f(x;\xi)],$3, and $f(x)=\mathbb{E}[\tilde f(x;\xi)],$4 proximal-iteration complexities for the corresponding settings, with matching optimal stochastic-gradient complexity under unbiased sampling (Bollapragada et al., 19 Jul 2025).

The multiobjective extension shows that analogous ideas survive outside scalar-valued objectives. Its key adaptive quantity is not stochastic variance control but a predesigned Lipschitz estimate sequence $f(x)=\mathbb{E}[\tilde f(x;\xi)],$5 that remains below $f(x)=\mathbb{E}[\tilde f(x;\xi)],$6 without backtracking, enabling a Nesterov-style coefficient

$f(x)=\mathbb{E}[\tilde f(x;\xi)],$7

and an $f(x)=\mathbb{E}[\tilde f(x;\xi)],$8 rate in the merit function $f(x)=\mathbb{E}[\tilde f(x;\xi)],$9 for weak Pareto optimality (Huang, 9 Jul 2025).

5. Nonconvex, constrained, and hybrid extensions

Several earlier and adjacent lines of work expand the adaNAPG design space beyond the standard smooth-convex composite regime. Projected Nesterov’s proximal-gradient algorithm (PNPG) treats sparse signal reconstruction with a convex set constraint and possibly non-Lipschitz-gradient negative log-likelihood terms. Its extrapolated point is explicitly projected onto the feasible set,

$f\in\mathcal S_{\mu,L}$0

followed by the proximal-gradient step

$f\in\mathcal S_{\mu,L}$1

Its step size is chosen by a local majorization condition rather than a global Lipschitz constant, and the method includes restart and inexact proximal computation. Under the stated assumptions it attains an $f\in\mathcal S_{\mu,L}$2 objective-rate bound and weak iterate convergence (Gu et al., 2015). Adaptive FISTA (aFISTA) moves further toward local model optimization: it chooses the extrapolation parameter by solving an inner minimization over

$f\in\mathcal S_{\mu,L}$3

and in some cases is equivalent to an SR1-type proximal quasi-Newton method. Its main nonconvex guarantee is monotone decrease and stationarity of limit points, not a general $f\in\mathcal S_{\mu,L}$4 rate, although hybrid convex variants in the appendix recover accelerated convex rates (Ochs et al., 2017).

Nonconvex sparse statistical learning provides another distinct extension. For objectives with convex smooth loss and nonconvex penalties such as SCAD or MCP, the accelerated gradient method of Ghadimi–Lan is modified by choosing the damping sequence from a complexity upper bound. The optimized convex-case parameters satisfy

$f\in\mathcal S_{\mu,L}$5

and the resulting upper bound remains $f\in\mathcal S_{\mu,L}$6; the contribution is improved constants and practical convergence speed rather than a better worst-case order (Yang et al., 2020).

Related hybrid and restart-based constructions show how adaptive acceleration can also be mediated by residual certification or phase switching. Restart ACG formulates restarted Nesterov-type acceleration as an instance of the accelerated inexact proximal point framework and preserves the optimal $f\in\mathcal S_{\mu,L}$7 outer rate for convex smooth composite optimization (Liang, 7 Jan 2025). NIDAAREM uses a two-phase procedure in which standard Nesterov/FISTA initialization is followed by damped Anderson acceleration with monotonicity-based switching, illustrating a broader algorithmic neighborhood around adaNAPG rather than a literal Nesterov-proximal variant (Henderson et al., 16 Aug 2025).

6. Terminology, common misconceptions, and current limits

A frequent source of confusion is terminological. Only one of the cited papers explicitly names its method adaNAPG (Zhu et al., 24 Jul 2025). Other works are described as adaptive proximal gradient, accelerated variant, adaptive inexact APG, projected Nesterov proximal gradient, adaptive FISTA, or adaptive extrapolated proximal gradient, even when they are functionally in the same family. This suggests that adaNAPG is better regarded as a design space built around accelerated proximal-gradient structure plus adaptive control, rather than as a single standardized algorithm.

A second misconception is that "adaptive" always refers to the momentum coefficient. In fact, the literature uses the term for at least five different mechanisms: accumulated gradient-mapping norms in the stepsize denominator (Wang et al., 7 May 2026), adaptive sample sizes or gradient-accuracy rules (Zhu et al., 24 Jul 2025, Bollapragada et al., 19 Jul 2025), deterministic line-search-free Lipschitz estimate sequences (Huang, 9 Jul 2025), locally optimized extrapolation parameters (Ochs et al., 2017), and adaptive local majorization with restart (Gu et al., 2015). Some methods change the extrapolation law itself; others keep the Nesterov law fixed and adapt only the gradient oracle or effective metric.

The main theoretical limitations are equally heterogeneous. The 2026 universal adaptive framework is universal across nonconvex smooth, convex nonsmooth, and convex smooth classes in its non-accelerated form, but its accelerated variant is analyzed only in the convex regime, and the paper explicitly states that it is not yet known whether the accelerated variant can also handle nonconvex problems optimally (Wang et al., 7 May 2026). The explicit stochastic adaNAPG paper warns that simply inserting a noisy gradient estimator into NAPG without controlling its error may fail to converge, which is why its adaptive sampling tests are central (Zhu et al., 24 Jul 2025). Adaptive FISTA proves stationarity in a general nonconvex setting, but its main adaptive method does not guarantee the classical $f\in\mathcal S_{\mu,L}$8 accelerated rate (Ochs et al., 2017).

Taken together, these results position adaNAPG as a technically diverse family of accelerated composite methods whose common core is a proximal step at an extrapolated point and whose main innovation lies in how the algorithm measures local difficulty—through gradient mapping magnitude, curvature surrogates, sample accuracy, or certified restart conditions—and feeds that information back into the update rule.