---
title: Zoobot Foundation Model Overview
url: https://www.emergentmind.com/topics/zoobot-foundation-model
type: topic
---

# Zoobot Foundation Model Overview

Polyak–Ruppert Averaging

Polyak–Ruppert averaging (also called Ruppert–Polyak averaging or simply PR averaging) is a variance-reduction and efficiency-enhancing technique for stochastic approximation algorithms, especially stochastic gradient descent (SGD) and linear stochastic approximation. Instead of returning the last parameter iterate, the method outputs the average of the entire trajectory of parameter estimates. Developed independently by Polyak and Juditsky (1992) and Ruppert (1988), PR averaging transforms asymptotic and finite-sample properties, yielding optimal mean-squared error (MSE), minimal asymptotic variance, and robustness under model and noise conditions. This technique is widely used in both first-order stochastic optimization and black-box/search-based schemes, including noisy zeroth-order and order-oracle methods, as well as in reinforcement learning, distributed estimation, and extremum seeking.

## 1. Core Algorithmic Principle

The essential principle of Polyak–Ruppert averaging is to form the output as the average of all iterates from a stochastic approximation process. For a generic recursion
\[
x_{k+1} = x_k - \alpha_k \, g(x_k, \xi_k),
\]
the averaged iterate is defined as
\[
\bar{x}_n = \frac{1}{n} \sum_{k=1}^n x_k.
\]
Here, $g(\cdot, \xi_k)$ is an unbiased (or possibly biased) estimate of the gradient or update direction, and $\alpha_k$ is a decaying step size, usually $\alpha_k = \alpha / k^\beta$, with $\beta \in (\frac{1}{2}, 1)$ [1709.03342].

This averaging can be trivially implemented in a running sum and provides substantial improvements in both the stability and statistical efficiency of stochastic optimization [1709.04073, 2004.04719]. For time-varying mini-batch or streaming settings, a weighted average relative to the batch sizes maintains optimality [2109.07117].

## 2. Asymptotic Normality and Covariance

Averaged iterates $\bar{x}_n$ admit a central limit theorem (CLT) of the form
\[
\sqrt{n}\,( \bar{x}_n - x^* ) \xrightarrow{d} \mathcal{N}(0,\, \Sigma^*).
\]
For classical stochastic gradient schemes under strong convexity and smoothness assumptions, the limiting covariance is
\[
\Sigma^* = [\nabla^2 f(x^*)]^{-1} S^* [\nabla^2 f(x^*)]^{-1},
\]
where $S^*$ is the asymptotic covariance of the noise martingale increments [1709.03342].

In linear stochastic approximation (LSA), the optimal covariance is similarly
\[
\Sigma^* = A^{-1} \Sigma_{\text{noise}} A^{-T},
\]
with $A$ the mean dynamics matrix and $\Sigma_{\text{noise}}$ the noise innovation covariance [2004.04719, 2207.04475].

A salient property is that this covariance attains the Cramér–Rao lower bound for unbiased estimators, making PR averaging semiparametrically efficient [2112.14582, 2505.21796]. Furthermore, this minimal covariance law persists even under Markovian or dependent noise, manifold attractors, and order- or value-oracle settings [1912.09187, 2411.15866, 2405.17834].

In black-box comparison-based optimization, PR averaging combined with stochastic order oracles yields an explicit, Hessian-determined asymptotic covariance
\[
V_{\mathrm{new}} = \frac{d}{(d-1)^2\alpha^2} \left[ \nabla^2 f(x^*) \right]^{-2}
\]
without unknown factors, and strictly tighter dispersion relative to non-averaged schemes [2411.15866].

## 3. Non-Asymptotic Error Rates and High-Probability Bounds

Polyak–Ruppert averaging provides optimal non-asymptotic performance: for $n$ samples, the mean-squared error satisfies
\[
\mathbb{E}\left[ \|\bar{x}_n - x^*\|^2 \right] \leq \frac{\operatorname{Tr} \Sigma^*}{n} + O(n^{-r}),
\]
where the main $1/n$ rate matches the minimax optimality, and higher-order terms decay as $n^{-r}$ varying with precise step-size schedule [1709.03342]. For optimal decay, exponents near $0.75$ in the step-size schedule minimize constants in the remainder [1709.03342].

Recent advances provide fully non-asymptotic high-probability concentration:
\[
\mathbb{P}\left( \| \bar{x}_n - x^* \| > \varepsilon \right) \leq \exp\left( -c\,n \varepsilon^2 \right) + o(n^{-1}) ,
\]
with explicit dependence on confidence level in both finite and infinite-horizon regimes [2505.21796]. For linear SA (including LSA and temporal-difference (TD) learning), sharp moment and deviation bounds are available that respect both dimension and mixing parameters [2207.04475, 2405.16644].

In complex or infinite-dimensional settings (e.g., functional CLTs for RL), PR averages satisfy process-level invariance principles, supporting pathwise inference [1911.01483, 2112.14582].

## 4. Applications and Specialized Regimes

Polyak–Ruppert averaging is robustly beneficial across numerous contexts:

- **Classical SGD and streaming mini-batch learning:** Achieves the Cramér–Rao lower bound under quasi-strong convexity and with non-i.i.d. dependent samples, provided batch-size and step-size schedules are adapted [2109.07117, 2205.12549].
- **Linear Stochastic Approximation and Reinforcement Learning:** Provides $O(1/n)$ MSE rates for policy evaluation (TD learning), Q-learning, and two-time-scale actor-critic via instance-optimal bounds even with constant step-size [2207.04475, 1709.04073, 2502.09884, 2508.07928].
- **Zeroth-order (gradient-free) and order-oracle optimization:** Empowers stochastic approximation when only function comparisons or noisy rankings are available, as in pairwise bandit feedback or black-box optimization [2411.15866, 2102.05198].
- **Extremum seeking and quasi-stochastic approximation:** Accelerates deterministic or quasi-random optimization schemes (e.g., with sinusoidal probing), doubling decay rates and yielding subquartic mean-squared error under suitable spectral conditions [2206.00814].
- **Distributed and federated estimation:** In decentralized consensus and policy evaluation problems, dual-accelerated PR algorithms achieve network-optimal error, often outperforming previous distributed stochastic optimization methods [2207.11425].
- **Regularized learning:** Weighted (e.g., geometric) PR averages mimic ridge regularization effects when applied to linear models, providing an explicit and computationally efficient bias-variance tradeoff [1802.08009].

## 5. Bias–Variance Decomposition and Step-Size Considerations

The decomposition of PR-averaged estimation error is
\[
\mathbb{E} \| \bar{x}_n - x^* \|^2 = \underbrace{ \| b_n \|^2 }_{\text{bias}^2}
+ \underbrace{ \frac{\operatorname{Tr} \Sigma^*}{n} }_{\text{variance}}
+ o( \max \{ \|b_n\|^2, 1/n \} )
\]
where the bias $b_n$ decays as $a_n = n^{-\rho}$ under a step-size $\alpha_n = a_0 n^{-\rho}$, with the variance term always scaling $O(1/n)$ [2405.17834, 2508.05570]. For $\rho > 1/2$, variance dominates and MSE achieves optimal order; for $\rho\leq 1/2$, bias dominates for long.

For constant step-size, PR averaging cannot eliminate the $O(\alpha)$ bias induced by persistent components, especially under Markov or multiplicative noise. Richardson–Romberg extrapolation can eliminate leading bias, restoring minimax optimality [2508.05570].

Geometric or weighted PR averaging tunes regularization implicitly, controlling the shrinkage of the estimator and allowing efficient hyperparameter selection [1802.08009].

## 6. Extensions and Advanced Regimes

The mathematical framework of PR averaging extends to:

- **Stable manifolds and submanifold attractors:** Central limit theorems hold for SGD on manifolds, with fluctuations in normal directions as in the isolated minimum case, and tangential errors decaying faster [1912.09187].
- **Two-timescale stochastic approximation:** PR averaging on both fast and slow variables allows simultaneous attainment of $O(1/\sqrt{n})$ convergence on both, provided timescale separation is not excessive [2502.09884, 2508.07928].
- **High-order non-asymptotic regimes:** Moment and Berry–Esseen bounds for PR averages are available, with rates matching optimal CLT constants and full explicit finite-sample error expansion [2405.16644, 2207.04475].
- **Batch-means and multi-batch confidence:** Functional CLTs for PR averages underpin batch-means techniques for confidence region construction, providing rigorous joint inference in high dimensions [1911.01483].

## 7. Numerical and Practical Implications

Empirical evaluations routinely confirm that PR averaging reduces the spread of iterates relative to non-averaged schemes, with error histograms sharply contracting to theoretical variance predictions [2411.15866]. In streaming and dependent-data scenarios, combining PR averaging with time-varying batch sizes selectively mitigates long-range dependence and structural biases [2109.07117, 2205.12549].

Practitioner guidelines are explicit:
- Use step-sizes $\alpha_n \sim n^{-0.75}$ for optimal non-asymptotic constants [1709.03342].
- In Markov settings, choose step-size exponents $\rho > 1/2$ if bias decay is essential [2405.17834].
- For constant step-size, employ PR averaging with a properly tuned value below stability thresholds [1709.04073, 2207.04475].
- In distributed contexts, PR averaging ensures both bias contraction and minimax variance rates, independent of network topology, provided communication rates scale with network condition numbers [2207.11425].

The universality, optimality, and robustness of PR averaging make it a core component of modern stochastic approximation. Its effects extend through finite-sample, non-i.i.d., model-misspecified, nonconvex, or streaming regimes, substantiated by rigorous central limit theorems, non-asymptotic analysis, and empirical validation.

Source: https://www.emergentmind.com/topics/zoobot-foundation-model