--- title: Adaptive Nesterov Accelerated Proximal Gradient url: https://www.emergentmind.com/topics/adaptive-nesterov-accelerated-proximal-gradient-adanapg type: topic --- # Adaptive Nesterov Accelerated Proximal Gradient Adaptive Nesterov Accelerated Proximal Gradient (adaNAPG) denotes a family of composite first-order methods that preserve the extrapolated proximal structure of Nesterov accelerated proximal gradient while adapting some combination of stepsize, curvature surrogate, gradient accuracy, sampling budget, or restart logic. The name is explicit in "Boosting Accelerated Proximal Gradient Method with Adaptive Sampling for Stochastic Composite Optimization" [2507.18277], whereas several closely related methods appear under other names, including the accelerated variant in "Universal Adaptive Proximal Gradient Methods via Gradient Mapping Accumulation" [2605.05944], adaptive inexact APG [2507.14479], projected Nesterov proximal gradient [1502.02613], and adaptive FISTA [1711.04343]. In this literature, the unifying object is the composite proximal-gradient update and its associated gradient mapping, rather than a single universally fixed momentum or stepsize recursion. ## 1. Composite optimization setting and stationarity measure The standard optimization model is the composite problem \[ \min_{x\in\mathbb{R}^d}\ \phi(x)=f(x)+h(x), \] where \(f\) is continuously differentiable and \(L\)-smooth, and \(h\) is closed, convex, proper, and prox-friendly, with efficiently computable proximal operator \(\prox_{\alpha,h}(y)\) [2507.14479]. In stochastic composite formulations one often has \[ f(x)=\mathbb{E}[\tilde f(x;\xi)], \] and the smooth term may be convex or strongly convex, typically summarized as \(f\in\mathcal S_{\mu,L}\) [2507.18277]. Other variants broaden the same template to nonconvex smooth \(f\), convex nonsmooth \(f\), multiobjective composite optimization, or composite models with nonconvex penalties [2605.05944], [2507.06737], [2009.10629]. The canonical stationarity object is the proximal gradient mapping \[ G_\alpha(x):=\frac{1}{\alpha}\Bigl(x-\operatorname{prox}_{\alpha h}(x-\alpha\nabla f(x))\Bigr), \] or closely related residuals such as \[ R_{\alpha_k}(y_k)=\frac{1}{\alpha_k}(y_k-x_{k+1}). \] This choice is structurally important in composite optimization because, unlike the raw gradient, the gradient mapping vanishes at stationary or optimal points of the composite problem; the 2026 universal adaptive framework makes this point explicitly and builds its adaptive denominator from gradient-mapping-like proximal displacements rather than plain gradients [2605.05944]. ## 2. Canonical accelerated proximal template and sources of adaptivity The baseline Nesterov accelerated proximal gradient template uses an extrapolated point \(y_k\), a proximal update at that point, and a momentum step. In exact-gradient form this is written as \[ x_{n+1}=\operatorname{prox}_{\alpha_n h}\!\bigl(y_n-\alpha_n \nabla f(y_n)\bigr),\qquad y_{n+1}=x_{n+1}+\beta_n(x_{n+1}-x_n), \] which is the baseline NAPG structure adopted before adaptive modification in the stochastic composite adaNAPG paper [2507.18277]. In adaptive inexact APG, the same outer structure appears with a gradient estimate \(g_k\approx \nabla f(y_k)\), \[ x_{k+1}=\prox_{\alpha_k,h}(y_k-\alpha_k g_k),\qquad y_{k+1}=x_{k+1}+\beta_{k+1}(x_{k+1}-x_k), \] with convex and strongly convex momentum schedules chosen in the standard Nesterov style [2507.14479]. What changes across adaNAPG-type methods is the adaptive control law. In the universal adaptive accelerated scheme, the auxiliary proximal sequence satisfies \[ z_{k+1}=\operatorname{prox}_{\frac{\eta}{\theta_kS_k}h}\left(z_k-\frac{\eta}{\theta_kS_k}g_k\right),\qquad y_{k+1}=x_k+\theta_k(z_{k+1}-z_k), \] and the effective stepsize is \[ \eta_k=\frac{\eta}{S_k},\qquad S_{k+1}^2=S_k^2\left(1+\frac{\|z_{k+1}-z_k\|^2}{\eta^2}\right), \] equivalently \[ S_k^2=\gamma^2+\sum_{j=1}^{k-1}\|\widetilde G_{z,j}\|^2,\qquad \widetilde G_{z,k}=\frac{1}{\eta_k}(z_k-z_{k+1}). \] The paper contrasts this directly with standard APG/FISTA, which uses a fixed stepsize \(1/L\) and does not adapt to noise or unknown problem structure [2605.05944]. A different adaptive mechanism appears in adaptive gradient-estimation APG. There, the method does not primarily adapt the momentum; instead it imposes an iteration-dependent inexactness condition such as \[ \|g_k-\nabla f(y_k)\|\le \frac{\eta_k}{2}\|R_{\alpha_k}(y_k)\|+\iota_0\delta_k \] in the finite-sum case, or its conditional second-moment analogue in the stochastic expectation case. For unbiased estimators, sample sizes are then chosen to satisfy that bound, so the gradient accuracy increases as the iterates approach a solution [2507.14479]. ## 3. The explicitly named adaNAPG algorithm The paper that explicitly introduces the name adaNAPG studies stochastic composite optimization \[ \min_{x\in\mathbb{R}^d}F(x):=f(x)+h(x), \] with \(f\in\mathcal S_{\mu,L}\), \(h\) proper, closed, convex, and unbiased stochastic gradients satisfying finite variance assumptions. Its baseline NAPG iteration is retained, but the exact gradient at the extrapolated point \(y_n\) is replaced by a mini-batch estimator \[ \hat\nabla f(y_n)=\frac1{K_n}\sum_{k=1}^{K_n}g(y_n,\xi_n^k), \] where the batch size \(K_n\) is increased until two adaptive tests are satisfied: a projection/inner-product-type test and an orthogonality/residual-type test. Conceptually these tests enforce that the stochastic gradient error is small relative to the current stationarity measure \(\|G_{\alpha_n}(y_n)\|\). The method uses the constant stepsize \[ \alpha=\frac{1}{L(\theta^2+\nu^2+1)}, \] updates the proximal point by \[ x_{n+1}=\operatorname{prox}_{\alpha h}\!\bigl(y_n-\alpha \hat\nabla f(y_n)\bigr), \] and, in the strongly convex case, simplifies the extrapolation to \[ y_{n+1}=x_{n+1}+\beta(x_{n+1}-x_n),\qquad \beta=\frac{1-\sqrt{\alpha\mu}}{1+\sqrt{\alpha\mu}}. \] The central claim is that this adaptive-sampling Nesterov proximal scheme achieves the optimal iteration complexity of accelerated first-order methods while also attaining optimal sample complexity in the strongly convex regime [2507.18277]. Its iteration guarantees are the classical accelerated ones. In the convex case \((\mu=0)\), \[ \mathbb E[F(x_n)]-F(x^\star)=O(n^{-2}), \] equivalently \(n=O(\varepsilon^{-1/2})\) iterations for \(\varepsilon\)-accuracy. In the strongly convex case \((\mu>0)\), the paper proves linear convergence, \[ \mathbb E[F(x_n)]-F(x^\star)\le C_2\rho^n,\qquad \mathbb E\|x_n-x^\star\|^2\le C_2\rho^n, \] with \[ \rho=1-\sqrt{\frac{\mu}{L(\theta^2+\nu^2+1)}}<1. \] For total sample usage \(\Gamma_n\), it further shows under uniformly bounded variance that \[ \|x_n-x^\star\|^2 = O_{\mathbb P}(\Gamma_n^{-1}), \] and derives a central limit theorem of the form \[ \alpha^{-1}\rho^{-n/2} \begin{pmatrix} x_n-x^\star\\ x_{n-1}-x^\star \end{pmatrix} \Rightarrow \mathrm{Normal}(0,\Sigma). \] The experimental section uses regularized logistic regression and a large inventory optimization problem, and reports that adaNAPG matches or beats geometric batch growth with fewer samples in strongly convex logistic regression and matches polynomial-growth accelerated methods while using fewer samples in convex problems [2507.18277]. ## 4. Major design variants in the adaNAPG family The recent literature shows that "adaptive" in accelerated proximal gradient methods is not tied to a single mechanism. The following variants are representative. | Variant | Adaptive ingredient | Stated guarantee | |---|---|---| | Universal accelerated proximal method [2605.05944] | Denominator \(S_k^2=\gamma^2+\sum_{j