Polyak-Style Learning Rates

• Polyak-style learning rates are adaptive step-size rules that compute the update as the ratio between the objective gap and the squared gradient norm.
• They dynamically adjust the step size—taking larger steps when far from the optimum and smaller ones as the solution is approached—with variants for stochastic, momentum, and preconditioned methods.
• These methods reduce manual tuning by incorporating objective progress directly into the learning rate, though challenges include denominator instability and estimating the optimal objective value.

Searching arXiv for recent and foundational papers on Polyak-style learning rates, SPS, and related PL-condition context. Polyak-style learning rate denotes a family of adaptive step-size rules in which the update magnitude is chosen from a ratio of an objective-gap term to a squared gradient norm term, rather than prescribed by a fixed schedule. In its classical deterministic form, for minimizing $f$, the step size is

$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$

and the same basic template reappears in stochastic optimization, reinforcement learning, momentum methods, preconditioned methods, and sharpness-aware training, usually with modified numerators, denominators, or clipping mechanisms (Oberman et al., 2019, Loizou et al., 2020). The defining feature is a self-tuning dependence on current optimization state: the step becomes larger when the estimated gap is large and smaller when either the gap shrinks or the relevant gradient norm grows (Li et al., 2024).

1. Classical rule and geometric motivation

The classical Polyak step size arises from minimizing a quadratic upper bound on the next-step distance to an optimizer. For convex minimization, one writes

$$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$

and minimizing $Q(\gamma)$ gives

$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$

This derivation appears explicitly in optimization-oriented expositions of Polyak-style rules and is the basic template later reused in stochastic and structured settings (Li et al., 2024, Abdukhakimov et al., 2023).

The interpretation is equally standard. If the current objective is far above optimum, the numerator is large and the method takes a larger step; if the gradient norm is large, the denominator is large and the step is reduced; as the iterate approaches optimality, the gap term tends to $0$, so the step size automatically shrinks (Li et al., 2024). In the stochastic-gradient setting studied by Loizou et al., the same idea is written as a learning rate proportional to the current loss gap divided by a stochastic gradient second moment, making the rule state-dependent rather than schedule-dependent (Oberman et al., 2019).

This classical ratio is the reason Polyak-style learning rates are often described as objective-aware. Unlike fixed or externally decayed learning rates, they encode progress information directly through $f(x^k)-f^*$, and unlike purely gradient-statistics-based methods, they do not rely only on gradient magnitude or moving averages (Loizou et al., 2020).

2. Stochastic Polyak step-size and empirical risk minimization

In finite-sum empirical risk minimization, the dominant stochastic formulation is the stochastic Polyak step-size (SPS), which replaces full-objective quantities by per-sample or minibatch quantities. A standard form is

$$\gamma_k=\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},$$

with clipped variant

$$\gamma_k=\min\left\{\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},\gamma_b\right\}.$$

The update is then $x^{k+1}=x^k-\gamma_k\nabla f_i(x^k)$ (Loizou et al., 2020). The practical attraction of SPS is that in many machine-learning losses one has $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$0, so the rule can be implemented without estimating the global optimum value $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$1 (Loizou et al., 2020, Abdukhakimov et al., 2023).

Theoretical analyses in this literature are unusually broad. SPS has been analyzed for strongly convex, convex, Polyak–Łojasiewicz, and general smooth nonconvex objectives, with interpolation playing a special role because the residual quantity

$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$2

vanishes when every sample loss is minimized at the same solution (Loizou et al., 2020). In that regime, SPS yields exact linear convergence for strongly convex and PL settings and exact $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$3 convergence in convex settings without requiring manually designed decay schedules (Loizou et al., 2020).

A closely related refinement is $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$4, which replaces the numerator by its positive part,

$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$5

thereby avoiding negative stochastic steps when $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$6. This modification yields a clean Fejér-type inequality and a near-best $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$7 rate in the Lipschitz nonsmooth setting without requiring interpolation (Garrigos et al., 2023). By contrast, FUVAL attempts to learn the unknown per-sample optimal values through auxiliary slack variables $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$8; the paper presents projection, prox-linear, and online-SGD interpretations, but explicitly concludes that the stochastic version shows no clear practical advantage over SGD and that the convergence analysis shows no advantage over SGD (Garrigos et al., 2023).

The literature also contains a corrected stochastic Polyak construction, StoPS, in which both numerator and denominator are adjusted by sample-optimum gradient terms: $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2},$9 This rule is theoretically exact but generally impractical because it depends on $$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$0, $$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$1, and $$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$2; its practical surrogate StoP removes these quantities and converges linearly only to a neighborhood (Horváth et al., 2022).

3. Momentum, preconditioning, and metric generalizations

Once momentum or non-Euclidean geometry is introduced, Polyak-style adaptation ceases to be a simple substitution into the classical ratio. One line of work derives the learning rate by minimizing the next-step squared distance to the optimizer for a momentum update

$$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$3

which gives the idealized rule

$$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$4

Replacing the inaccessible $$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$5 term by convexity-based lower bounds yields adaptive heavy-ball and moving-averaged-gradient rules. For the moving-averaged-gradient method $$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$6, the resulting step size is particularly simple: $$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$7 Its stochastic counterpart is

$$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$8

and is analyzed under convex smoothness, semi-strong convexity, and interpolation-type conditions (Wang et al., 2023).

A second momentum-based route is model-based. MoMo constructs a momentum model from exponentially averaged sampled losses and gradients and then approximately minimizes a truncated local model. If

$$Q(\gamma)=\|x^k-x^*\|^2-2\gamma\bigl[f(x^k)-f^*\bigr]+\gamma^2\|\nabla f(x^k)\|^2,$$9

the update is

$Q(\gamma)$0

This reduces to SPS when momentum is removed, extends to preconditioned variants such as MoMo-Adam, and empirically improves robustness to hyperparameter tuning on image classification, recommendation, translation, and diffusion workloads (Schaipp et al., 2023).

A third generalization reinterprets Polyak in an arbitrary SPD metric. Under a linear-Gaussian inference model, the update becomes

$Q(\gamma)$1

which reduces to a generalized Polyak rule when $Q(\gamma)$2: $Q(\gamma)$3 Here the denominator is the squared gradient norm in the geometry induced by $Q(\gamma)$4, not the Euclidean norm (Roos et al., 2021). Preconditioned SPS makes the same point in finite-sum form by replacing $Q(\gamma)$5 with a preconditioned norm and the search direction with $Q(\gamma)$6, yielding

$Q(\gamma)$7

with Hutchinson, AdaGrad-style, and Adam-style preconditioners studied experimentally (Abdukhakimov et al., 2023).

Variant	Representative rule	Distinctive feature
SPS	$Q(\gamma)$8	Per-sample Polyak ratio
Generalized momentum Polyak	$Q(\gamma)$9	Denominator uses momentum direction
MoMo	$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$0	Momentum model of loss and gradient
Metric-generalized Polyak	$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$1	Non-Euclidean geometry and noise term
PSPS	$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$2	Preconditioned denominator and direction

These generalizations preserve the core Polyak logic—objective-aware scaling by a gradient-energy term—but alter what counts as the relevant “gap,” the relevant “gradient,” and the relevant geometry.

4. Reinforcement learning and other structured extensions

In reinforcement learning, transplanting Polyak-style learning rates is nontrivial because policy-gradient updates are stochastic, the objective is a maximization problem, and the optimal return $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$3 is unknown. The direct analogue is a clipped ascent rule of the form

$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$4

but the paper on policy-gradient adaptation argues that naive use is unsafe: under softmax parameterization, the gradient norm can become arbitrarily small while the policy remains suboptimal, producing explosive updates (Li et al., 2024).

The proposed remedy has two RL-specific components. First, the objective is augmented with entropy regularization,

$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$5

specifically to prevent denominator pathologies. Second, the unknown optimum $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$6 is replaced by a conservative twin-model proxy: two policies of the same architecture are maintained, the better one provides a pessimistic target value, and only the worse one is updated. The resulting Polyak-style step size is the clipped objective difference between the two twins divided by the squared norm of the worse model’s GPOMDP gradient (Li et al., 2024). The paper reports faster and more stable convergence than Adam with fixed learning rates on Acrobot, CartPole, and LunarLander (Li et al., 2024).

Structured Polyak rules also appear in SAM-style optimization. For unnormalized SAM, the deterministic scheduler derived from the SAM-specific distance recursion is

$\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$7

with stochastic clipped version obtained by replacing $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$8 by a lower bound $\gamma_k=\frac{f(x^k)-f^*}{\|\nabla f(x^k)\|^2}.$9 and capping by $0$0 (Oikonomou et al., 1 Jun 2026). The essential novelty is the correction term $0$1, which accounts for the fact that the gradient is evaluated at the perturbed point $0$2 while the update is applied at $0$3. When the sharpness radius is zero, the rule reduces to classical Polyak/SPS (Oikonomou et al., 1 Jun 2026).

A decentralized extension derives a graph-aware Polyak rule for ATC-Diffusion. There, step sizes are coupled across agents through the two-hop matrix $0$4, and the method reduces to the classical Polyak formula only when $0$5. The resulting update is not a closed-form ratio but a network-coupled fixed-point iteration whose numerator and denominator involve graph-weighted local objective differences and cross-agent gradient inner products (Fainman et al., 18 Sep 2025). This suggests that, in networked settings, “gap over squared gradient norm” becomes “graph-weighted local gap over graph-weighted gradient interaction.”

5. Relation to the Polyak–Łojasiewicz condition and to heavy-ball methods

A persistent source of ambiguity is that “Polyak-style” may refer either to the adaptive step-size rule or to the Polyak–Łojasiewicz inequality. These are distinct. The PL inequality,

$0$6

is a geometric condition implying that every stationary point is globally optimal and that fixed-step gradient descent with step $0$7 converges linearly under $0$8-smoothness (Karimi et al., 2016). It is not itself a learning-rate rule.

This distinction matters because several papers central to modern “Polyak” theory are not step-size papers. Karimi, Nutini, and Schmidt analyze global linear convergence of gradient, coordinate, stochastic, and proximal-gradient methods under the PL condition, largely with fixed or standard steps such as $0$9, not with the classical Polyak ratio $f(x^k)-f^*$0 (Karimi et al., 2016). Likewise, the deep-network work on locally Polyak–Łojasiewicz regions shows that a local PL inequality can emerge near initialization under NTK stability, yielding linear convergence of fixed-step gradient descent with $f(x^k)-f^*$1, but does not analyze a Polyak adaptive rule (Aich et al., 29 Jul 2025).

The same clarification applies to momentum. Polyak’s heavy-ball method is a fixed-parameter momentum method, not a Polyak step-size method. Under the PL inequality, heavy ball with parameters

$f(x^k)-f^*$2

achieves accelerated local linear rates in discrete time and accelerated global rates in continuous time, but these are curvature-tuned constants, not adaptive Polyak ratios (Kassing et al., 2024). A plausible implication is that the phrase “Polyak-style learning rate” is best reserved for objective-gap-based step-size rules, while PL-condition and heavy-ball results provide geometry and momentum theory that often explain why such rules may be well scaled.

6. Empirical behavior, limitations, and recurring issues

Across the literature, Polyak-style learning rates are repeatedly presented as a way to reduce manual tuning of a global learning-rate schedule. In over-parameterized and interpolation-like settings, SPS is reported to perform strongly against state-of-the-art optimizers on synthetic problems, kernel classifiers, and deep networks, with the strongest theoretical and empirical behavior occurring precisely when $f(x^k)-f^*$3 or can be treated as known (Loizou et al., 2020). Preconditioned SPS shows its clearest gains on badly scaled datasets, where Euclidean normalization is mismatched to the problem geometry and preconditioned norms materially improve robustness (Abdukhakimov et al., 2023). MoMo and MoMo-Adam are reported to widen the range of good learning rates relative to SGD-M and Adam across image classification, recommendation, translation, and diffusion workloads (Schaipp et al., 2023). In policy-gradient RL, the twin-model entropy-regularized construction is reported to achieve faster convergence and more stable policies than Adam with fixed learning rates on the tested Gym tasks (Li et al., 2024). For SAM-style methods, Polyak schedulers are reported to match or outperform tuned constant or cosine schedules while reducing learning-rate sensitivity and improving robustness to larger sharpness radii (Oikonomou et al., 1 Jun 2026).

At the same time, the limitations are consistent across papers. The most fundamental is the need for $f(x^k)-f^*$4, $f(x^k)-f^*$5, $f(x^k)-f^*$6, or a lower bound $f(x^k)-f^*$7. This is easy in some regimes and awkward in others (Oberman et al., 2019, Horváth et al., 2022). Clipping is therefore common: SPS$f(x^k)-f^*$8, ALI-G, stochastic SAM Polyak schedulers, and many RL variants all impose an upper bound to prevent excessively large steps (Loizou et al., 2020, Berrada et al., 2021, Oikonomou et al., 1 Jun 2026, Li et al., 2024). A second recurring issue is denominator instability. In Euclidean stochastic settings this motivates clipping and sometimes additive stabilization; in RL it motivates entropy regularization because softmax policies can make the denominator effectively singular before the policy is optimal (Berrada et al., 2021, Li et al., 2024). A third issue is that removing the optimum-value oracle without sacrificing the attractive theoretical behavior remains difficult: StoP, FUVAL, and other practical surrogates generally converge only to a neighborhood or show no clear practical gain over tuned baselines (Horváth et al., 2022, Garrigos et al., 2023).

The evaluation of Polyak-style methods has itself been contested. A note on ALI-G argues that prior SPS comparisons understated ALI-G’s performance by omitting momentum and meaningful tuning; with momentum $f(x^k)-f^*$9, learning rate $$\gamma_k=\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},$$0, and max $$\gamma_k=\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},$$1-norm regularization $$\gamma_k=\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},$$2, the note reports $$\gamma_k=\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},$$3 on CIFAR-10 and $$\gamma_k=\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},$$4 on CIFAR-100 for ResNet-34, versus $$\gamma_k=\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},$$5 and $$\gamma_k=\frac{f_i(x^k)-f_i^*}{c\,\|\nabla f_i(x^k)\|^2},$$6 as previously reported (Berrada et al., 2021). This does not refute SPS, but it does show that empirical conclusions about Polyak-style rules can depend strongly on the full training protocol.

In encyclopedic terms, the modern concept is therefore best understood as a family rather than a single formula. The common invariant is objective-gap normalization by a squared gradient quantity. What changes from paper to paper is which objective gap is accessible, which gradient quantity is geometrically appropriate, how stochasticity is stabilized, and whether the problem structure—finite sums, interpolation, momentum, entropy regularization, perturbation-based training, or graph coupling—permits a faithful Polyak analogue.