PASTA: Proximally Anchored Stochastic Approximation

• PASTA is a unified framework for stochastic nonconvex optimization that combines Halpern anchoring with Tikhonov regularization to control unbounded gradient noise.
• It instantiates techniques like dynamic batching, PAGE-type variance reduction, or epoch-based proximal updates to meet minimax-optimal oracle complexities (e.g., Ω(ε⁻⁶) and Ω(ε⁻⁴)).
• The framework uses proximal anchoring both as a stabilizing mechanism and an information-theoretic tool to secure optimal convergence across smooth, weakly convex, PL, and star-convex regimes.

Proximally Anchored STochastic Approximation (PASTA) is a unified framework for stochastic nonconvex optimization under unbounded gradient noise satisfying the BG-0 condition. It couples Halpern anchoring with Tikhonov regularization, and in different regimes can be instantiated with dynamic batching, PAGE-type variance reduction, or epoch-based inexact proximal-point structure. In the formulation introduced in “Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance,” PASTA is both an algorithmic template and an information-theoretic response to a weakened oracle model: the same work proves that under BG-0, smooth nonconvex minimization requires $\Omega(\varepsilon^{-6})$ stochastic oracle queries and mean-square smooth nonconvex minimization requires $\Omega(\varepsilon^{-4})$ queries, and then shows that suitable PASTA instantiations attain these rates on unbounded domains and with unbounded stochastic gradients (Fazla et al., 17 Apr 2026).

1. Problem class and motivating oracle model

PASTA is formulated for unconstrained stochastic optimization of the form

$$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$

The target is not global minimization in the worst-case nonconvex sense, but the computation of an $\varepsilon$-stationary point, typically measured through either $\mathbb E\|\nabla f(x)\| \le \varepsilon$ or $\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$ depending on the theorem (Fazla et al., 17 Apr 2026).

The framework is organized around the BG-0 stochastic first-order oracle. For a closed convex set $\mathcal X \subset \mathbb R^p$ and reference point $x_0 \in \mathcal X$, the oracle is unbiased in the sense that

$$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$

and there exist constants $B_v \ge 0$ and $\Omega(\varepsilon^{-4})$0 such that

$\Omega(\varepsilon^{-4})$1

When $\Omega(\varepsilon^{-4})$2 is differentiable, this becomes

$\Omega(\varepsilon^{-4})$3

This model differs fundamentally from classical bounded-variance assumptions. The baseline term $\Omega(\varepsilon^{-4})$4 acts as a noise floor, while the term $\Omega(\varepsilon^{-4})$5 permits variance to grow quadratically with distance from the reference point. The paper emphasizes that uniformly bounded variance often fails even in unconstrained least squares and deep neural network training, and presents BG-0 as the weakest viable variance condition in this setting (Fazla et al., 17 Apr 2026).

PASTA is therefore motivated by a specific failure mode of standard SGD: if iterates drift far from the reference point, the oracle itself becomes increasingly noisy. The framework’s anchoring and proximal regularization are designed to counteract exactly that distance-driven variance explosion.

2. Lower bounds and the information-theoretic role of anchoring

A central contribution of the PASTA paper is that under BG-0, the familiar bounded-variance complexity landscape is no longer attainable. For $\Omega(\varepsilon^{-4})$6-smooth nonconvex objectives satisfying

$\Omega(\varepsilon^{-4})$7

the paper proves the lower bound

$\Omega(\varepsilon^{-4})$8

for producing a point $\Omega(\varepsilon^{-4})$9 with $$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$0, where $$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$1. The headline implication is the rate

$$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$2

Under mean-square smoothness,

$$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$3

the lower bound becomes

$$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$4

with headline rate

$$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$5

These bounds are strictly worse than the classical bounded-variance lower bounds $$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$6 in the smooth case and $$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$7 in the mean-square-smooth case (Fazla et al., 17 Apr 2026).

The paper’s intuition is geometric and statistical. An adversarial function can force the algorithm to travel a distance

$$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$8

before entering a nearly stationary region. Under BG-0, variance at that scale becomes

$$\min_{x \in \mathbb{R}^p} f(x) = \mathbb{E}_{\xi}[\tilde f(x,\xi)].$$9

The difficult optimization phase therefore occurs precisely where the oracle is most noisy. In the smooth case this adds an $\varepsilon$0 penalty to the usual bounded-variance complexity, and under mean-square smoothness variance reduction helps only partially.

In this sense, anchoring is not introduced merely as a stabilizing heuristic. Within the PASTA analysis it is a mechanism for meeting a sharp lower-bound barrier created by the BG-0 model.

3. Core construction of the PASTA framework

PASTA is a template rather than a single immutable update rule. Its generic inner iteration is

$\varepsilon$1

where $\varepsilon$2 is the epoch anchor, $\varepsilon$3 is an anchoring coefficient, $\varepsilon$4 is a stepsize, and $\varepsilon$5 is a stochastic gradient estimator (Fazla et al., 17 Apr 2026).

When mean-square smoothness is available, the paper uses a PAGE-style estimator: $\varepsilon$6

The defining structural move is the coupling

$\varepsilon$7

With this choice, the update becomes

$\varepsilon$8

which is exactly a stochastic gradient step on the Tikhonov-regularized surrogate

$\varepsilon$9

This identity gives PASTA its name and its main interpretation. The iterate is not only pulled toward an anchor in the Halpern sense; it is also optimized against a proximal quadratic centered at that anchor. The anchor therefore defines the center of a local regularized problem, while the quadratic term adds curvature that suppresses the BG-0 variance growth associated with drifting too far from the reference.

The framework removes the bias introduced by optimizing $\mathbb E\|\nabla f(x)\| \le \varepsilon$0 rather than $\mathbb E\|\nabla f(x)\| \le \varepsilon$1 through epoch resetting. At epoch $\mathbb E\|\nabla f(x)\| \le \varepsilon$2, one approximately minimizes

$\mathbb E\|\nabla f(x)\| \le \varepsilon$3

for $\mathbb E\|\nabla f(x)\| \le \varepsilon$4 inner iterations, and then sets

$\mathbb E\|\nabla f(x)\| \le \varepsilon$5

The resulting method is an inexact stochastic proximal-point scheme whose proximal center is updated only at epoch boundaries.

A common misconception is that PASTA always uses active anchoring in every regime. The paper is explicit that this is not the case. Different parameter choices recover different algorithmic behaviors:

• Smooth nonconvex: $\mathbb E\|\nabla f(x)\| \le \varepsilon$6, $\mathbb E\|\nabla f(x)\| \le \varepsilon$7, $\mathbb E\|\nabla f(x)\| \le \varepsilon$8, yielding an unanchored SGD-like method with dynamic batching.
• Mean-square smooth nonconvex: $\mathbb E\|\nabla f(x)\| \le \varepsilon$9, $\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$0, with PAGE-style variance reduction.
• Weakly convex: epoch-based PASTA with $\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$1 and $\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$2.
• PL and star-convex: again $\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$3, $\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$4, $\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$5 suffices.

Thus PASTA is a unified framework whose “proximal anchoring” component is essential in the regimes where curvature must be manufactured to overcome BG-0, but unnecessary where global structure already controls drift.

4. Complexity guarantees across optimization regimes

The theoretical guarantees vary by geometry, smoothness, and stationarity notion. In each case the paper states explicit parameter choices and obtains rates on unbounded domains without bounded stochastic gradients (Fazla et al., 17 Apr 2026).

For standard smooth nonconvex minimization, the paper uses dynamic batching with

$\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$6

$\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$7

and proves

$\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$8

with expected total oracle complexity

$\mathbb E\|\nabla f(x)\|^2 \le \varepsilon^2$9

For mean-square smooth nonconvex minimization, the PAGE-type instantiation uses

$\mathcal X \subset \mathbb R^p$0

$\mathcal X \subset \mathbb R^p$1

and

$\mathcal X \subset \mathbb R^p$2

It then shows

$\mathcal X \subset \mathbb R^p$3

with total stochastic oracle complexity

$\mathcal X \subset \mathbb R^p$4

For weakly convex objectives, the paper assumes $\mathcal X \subset \mathbb R^p$5 is $\mathcal X \subset \mathbb R^p$6-weakly convex and $\mathcal X \subset \mathbb R^p$7-Lipschitz, takes $\mathcal X \subset \mathbb R^p$8, defines $\mathcal X \subset \mathbb R^p$9, and uses $x_0 \in \mathcal X$0. The analysis is phrased via the Moreau envelope: $x_0 \in \mathcal X$1

$x_0 \in \mathcal X$2

$x_0 \in \mathcal X$3

With suitable choices of $x_0 \in \mathcal X$4, the paper proves

$x_0 \in \mathcal X$5

and total oracle complexity

$x_0 \in \mathcal X$6

For Polyak–Łojasiewicz objectives,

$x_0 \in \mathcal X$7

with $x_0 \in \mathcal X$8, $x_0 \in \mathcal X$9, $$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$0, and $$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$1, the paper chooses

$$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$2

$$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$3

and proves

$$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$4

with complexity

$$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$5

For $$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$6-star-convex objectives satisfying

$$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$7

the paper obtains the same headline complexity

$$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$8

again without anchoring, because the global geometry already controls distance growth.

Regime	Guarantee	Complexity
Smooth nonconvex	$$\mathbb E[\widetilde{\nabla}f(x)] = \bar g \quad\text{for some } \bar g \in \partial f(x),$$9	$B_v \ge 0$0
Mean-square smooth nonconvex	$B_v \ge 0$1	$B_v \ge 0$2
Weakly convex	$B_v \ge 0$3	$B_v \ge 0$4
PL	$B_v \ge 0$5	$B_v \ge 0$6
Star-convex	$B_v \ge 0$7	$B_v \ge 0$8

The paper presents these bounds as minimax-optimal for smooth nonconvex and mean-square smooth nonconvex BG-0 optimization, and as optimal in the weakly convex case via smooth lower-bound inheritance.

5. Relation to earlier proximal stochastic methods

PASTA is explicit as a named framework only in the 2026 paper, but its structure sits within a broader line of proximal and anchored stochastic methods. Several earlier works do not use the term “anchored,” yet they instantiate closely related update geometries.

The clearest ancestor in classical stochastic approximation is the proximal Robbins–Monro method, which replaces the explicit Robbins–Monro step by an implicit proximal update centered at the previous iterate: $B_v \ge 0$9 That work emphasizes numerical stability, resolvent contraction, and approximate proximal implementations, and provides one of the clearest formal precedents for proximally anchored stochastic approximation in the Robbins–Monro sense (Toulis et al., 2015).

A second line appears in model-based stochastic convex optimization. The aProx family updates

$\Omega(\varepsilon^{-4})$00

using a local stochastic model plus a quadratic anchor at the current iterate. This unifies stochastic subgradient, stochastic proximal point, truncated models, and bundle-like constructions, and shows that model fidelity plus proximal anchoring can yield stronger convergence and robustness than bare stochastic subgradients (Asi et al., 2018).

For weakly convex nonsmooth nonconvex objectives, the proximally guided stochastic subgradient method organizes computation as an outer proximal-point anchor sequence

$\Omega(\varepsilon^{-4})$01

combined with an inner projected stochastic subgradient solver. Its stationarity certificate is the proximal residual $\Omega(\varepsilon^{-4})$02, equivalently the Moreau-envelope gradient norm up to $\Omega(\varepsilon^{-4})$03, and its total oracle complexity is

$\Omega(\varepsilon^{-4})$04

for weakly convex problems (Davis et al., 2017).

Constraint-handling variants produce different anchor geometries. The stochastic proximal distance algorithm uses the projection of the previous iterate onto the constraint set as the proximal center: $\Omega(\varepsilon^{-4})$05 so the anchor is explicitly tied to feasibility rather than merely to the previous iterate itself (Jiang et al., 2022). Related model-based methods over proximally smooth sets similarly combine local objective models, quadratic centering at the current iterate, and retraction or projection-like feasibility restoration (Davis et al., 2020).

Relative to these precedents, PASTA is specialized in a different direction. Its distinctive contribution is not the mere existence of a proximal anchor, but the coupling of Halpern anchoring and Tikhonov regularization to neutralize the distance-dependent variance term of the BG-0 oracle. In that sense it extends the proximal-stochastic tradition from stability under bounded or weakly controlled noise to minimax-optimal complexity under explicitly unbounded variance growth.

6. Interpretation, scope, and limitations

The geometric function of proximal anchoring in PASTA is precise: when oracle variance grows like

$\Omega(\varepsilon^{-4})$06

the quadratic term

$\Omega(\varepsilon^{-4})$07

acts as a variance shield by making long excursions away from the current anchor expensive. In the weakly convex regime this manufactured curvature turns

$\Omega(\varepsilon^{-4})$08

into a strongly convex surrogate whenever $\Omega(\varepsilon^{-4})$09, because then $\Omega(\varepsilon^{-4})$10.

The paper also isolates a necessary tradeoff between anchoring strength, batch size, and stepsize. For the one-epoch anchored or dynamic-batch update

$\Omega(\varepsilon^{-4})$11

it states that eliminating an unbounded $\Omega(\varepsilon^{-4})$12 residual in the drift recursion requires

$\Omega(\varepsilon^{-4})$13

This “geometric-statistical uncertainty principle” formalizes why ordinary one-sample unanchored SGD is generally not viable under BG-0 (Fazla et al., 17 Apr 2026).

Several boundary conditions follow from the same analysis. First, PASTA is not a generic proximal-optimization formalism over arbitrary geometries. The 2026 paper studies Euclidean stochastic first-order methods for unconstrained problems on $\Omega(\varepsilon^{-4})$14, with theory tailored to BG-0. Second, the framework is primarily theoretical. The mechanisms—dynamic batch sizes, epoch resetting, and recursive variance-reduced estimators—are implementable, but the prescribed parameter choices are theorem-driven and especially elaborate in the weakly convex case. Third, “proximal” does not mean “constraint-based” in the PASTA paper: unlike stochastic proximal distance or retracted model-based methods, the role of the proximal term here is variance control rather than feasibility restoration.

A further misconception is that PASTA’s contribution is only algorithmic. The paper’s significance is equally lower-bound based. Its headline claim is that it provides the first minimax-optimal framework for stochastic nonconvex optimization under the weakest meaningful unbounded-variance assumption, BG-0, across smooth, mean-square smooth, weakly convex, PL, and star-convex regimes (Fazla et al., 17 Apr 2026). Within the proximal stochastic-approximation literature, that places PASTA at the point where anchoring becomes not just stabilizing or convenient, but information-theoretically necessary.