Stochastic Approximate Gradient Descent

• Stochastic Approximate Gradient Descent is a method that employs underdamped Langevin dynamics to construct biased yet asymptotically accurate gradient estimates for expectation-based models.
• It integrates sampling with optimization by replacing unavailable unbiased gradients with controlled approximations via tunable Langevin step sizes and chain lengths.
• SAGD has practical applications in expectation-maximization and variational autoencoders, achieving convergence rates comparable to classical SGD under both convex and nonconvex conditions.

Searching arXiv for papers directly relevant to “stochastic approximate gradient descent” and closely related stochastic approximation/SGD analyses. Stochastic approximate gradient descent (SAGD) is a stochastic optimization method for expectation-based objectives in which unbiased stochastic gradients cannot be trivially obtained. In the formulation introduced in "Stochastic Approximate Gradient Descent via the Langevin Algorithm" (Qiu et al., 2020), the unavailable gradient estimator is replaced by a stochastic approximate gradient generated from an underdamped Langevin chain targeting an auxiliary distribution $\pi(\xi)$. The resulting estimator is biased at finite chain length but asymptotically accurate, and its bias and mean-squared error can be controlled analytically through the Langevin step size and chain length. In this sense, SAGD occupies an intermediate position between stochastic approximation, Markov chain Monte Carlo, and latent-variable optimization, with direct applications to expectation-maximization and variational autoencoders (Qiu et al., 2020).

1. Problem setting and definition

The canonical setting is an objective of the form

$F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$

with gradient

$g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$

For many models, especially those based on data averaging, an unbiased stochastic gradient can be formed by sampling a data point. SAGD addresses the complementary regime: $\xi$ follows a complex or intractable distribution $\pi(\xi)$, known only up to a normalizing constant, so direct unbiased stochastic gradients are often infeasible (Qiu et al., 2020).

This distinction is fundamental. Classical stochastic gradient descent assumes that the noise arises from inexpensive subsampling of an already accessible gradient expectation. SAGD instead treats gradient computation itself as an inference subproblem. The stochasticity in the update therefore reflects both optimization noise and Monte Carlo approximation error. The method is explicitly proposed as an alternative to traditional reliance on general-purpose sampling techniques such as Markov chain Monte Carlo in cases where unbiased stochastic gradients cannot be trivially obtained (Qiu et al., 2020).

A common misconception is to identify SAGD with ordinary minibatch SGD. The two methods share an outer descent recursion, but their estimator models differ. In SAGD, the gradient surrogate is constructed from samples of an auxiliary latent-variable chain, so the estimator is biased in finite computation. The central analytical claim is not unbiasedness but asymptotic accuracy together with convergence guarantees under explicit schedules for the sampler and optimizer (Qiu et al., 2020).

2. Langevin construction of the stochastic approximate gradient

The key mechanism in SAGD is the use of the underdamped Langevin algorithm to sample approximately from $\pi(\xi)$, where

$V(\xi) = -\log \pi(\xi) + C.$

The continuous-time underdamped Langevin diffusion is

$$\begin{aligned} d\xi(t) & = \rho(t)dt \ d\rho(t) & = -\gamma\rho(t)dt - \nabla V(\xi(t))dt + \sqrt{2\gamma}dB(t), \end{aligned}$$

with $B(t)$ a Brownian motion and $\gamma>0$. The paper discretizes this SDE by the explicit Euler scheme,

$F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$0

Averages over this chain provide estimators of expectations under the target distribution (Qiu et al., 2020).

For a test function $F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$1, the estimator

$F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$2

satisfies

$F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$3

where $F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$4 is the true expectation under the augmented target distribution. These bounds formalize the phrase “biased in finite steps but accurate asymptotically”: both bias and MSE can be made arbitrarily small by proper choice of $F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$5 and $F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$6 (Qiu et al., 2020).

The construction is notable because it replaces direct sampling from $F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$7 with a controllable discretized diffusion. This makes the approximate-gradient mechanism analyzable by combining Langevin error bounds with stochastic optimization arguments. A plausible implication is that the “approximate” qualifier in SAGD refers not to heuristic gradient corruption, but to a deliberate estimator design whose approximation error is itself parameterized and schedulable.

3. Algorithmic recursion and convergence guarantees

At outer iteration $F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$8, SAGD runs a Langevin chain of length $F(\theta) = \mathbb{E}_{\xi}[f(\theta; \xi)],$9 with step size $g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$0, constructs

$g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$1

and updates the parameter by

$g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$2

where $g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$3 is projection onto the feasible set. The algorithm takes as input the number of steps $g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$4, learning rates $g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$5, Langevin step sizes $g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$6, chain lengths $g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$7, and initial values $g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$8, and returns the average iterate

$g(\theta) = \mathbb{E}[\nabla f(\theta;\xi)].$9

(Qiu et al., 2020).

In the convex case, the paper proves that if

$\xi$0

then

$\xi$1

The stated significance is that SAGD achieves the same convergence rate as classic SGD despite the use of biased stochastic gradients (Qiu et al., 2020).

In the nonconvex case, under Lipschitz conditions on the approximate gradient and similar parameter scheduling,

$\xi$2

Accordingly, the iterates approach first-order critical points in expectation, paralleling standard stochastic first-order results. The theory emphasizes that the number of Langevin steps per gradient estimate must grow with time, although the same source notes that this requirement can be loosened in practice (Qiu et al., 2020).

These guarantees clarify a second misconception: bias in the per-iteration gradient estimator does not by itself preclude SGD-type asymptotics. What matters in this framework is the coupled schedule governing optimizer step sizes, sampler step sizes, and chain lengths. The convergence proof therefore treats optimization and sampling error as a single controlled dynamical system rather than as separable modules.

4. Relation to stochastic approximation and other approximate-gradient frameworks

SAGD belongs to a broader family of stochastic approximation methods in which the driving signal is noisy, partial, or only indirectly observed. A recent convergence framework based on the Generalized Strong Law of Large Numbers (GSLLN) provides sufficient conditions for almost sure convergence of stochastic approximation and SGD while decoupling the properties of the mean field from those of the noise sequence. In that analysis, the noise need not have a finite second moment, and under suitable conditions not even a finite mean; the same approach also yields convergence results for zero-order SGD using $\xi$3 function evaluations but no gradient computations (Karandikar et al., 16 May 2025).

From that perspective, SAGD occupies a distinct regime. Its error source is not only heavy-tailed or dependent observation noise; it is also the controlled finite-time bias produced by an internal Markov chain. This suggests that SAGD extends stochastic approximation in a direction complementary to GSLLN-based theory: instead of weakening moment assumptions on exogenous noise, it engineers an approximate gradient through asymptotically accurate sampling.

A related but different line of work appears in constrained stochastic compositional optimization. There, unbiased gradients are unavailable because the objective and constraints are compositions of stochastic functions, and approximate gradients are constructed by tracking inner function evaluations. The resulting quasi-gradient saddle point algorithm is proved to converge almost surely and requires $\xi$4 data samples to obtain an $\xi$5-approximate optimal point while ensuring zero constraint violation (Thomdapu et al., 2020). In that comparison, SAGD is one specific realization of a general principle: when exact stochastic gradients are inaccessible, one may recover optimization guarantees by introducing auxiliary state variables that approximate the missing quantities.

5. Sampling, Bayesian, and continuous-time interpretations

Although SAGD uses Langevin dynamics to build gradient estimators, a neighboring literature reinterprets stochastic gradient methods themselves as approximate samplers. Constant-step-size SGD has been analyzed as a Markov chain with a stationary distribution; near a local optimum and under a quadratic approximation, the dynamics are described by a multivariate Ornstein-Uhlenbeck process, and the stationary law can be tuned to approximate a Bayesian posterior by minimizing a Kullback-Leibler divergence (Mandt et al., 2017). In a related non-asymptotic framework, properly noised SGD is shown to be essentially Langevin Monte Carlo, with explicit Wasserstein-$\xi$6 guarantees for approximate sampling under strong convexity and smoothness assumptions (Karagulyan, 2018).

Natural-gradient variants extend this sampling viewpoint. For minibatch NGD, the stationary distribution near local minima approaches a Bayesian posterior with temperature

$\xi$7

and a corrected “stochastic NGD” adds metric terms required for parameterisation invariance away from local minima (Smith et al., 2018). These results do not define SAGD, but they sharpen its conceptual placement: both SAGD and sampling-by-SGD exploit stochasticity as a computational resource rather than treating it purely as nuisance.

Continuous-time approximations supply a further interpretive layer. In nonconvex optimization, diffusion approximations suggest that SGD escapes local minimizers in a number of iterations exponentially dependent on the inverse stepsize and escapes saddle points almost linearly dependent on the inverse stepsize; the same analysis associates small batch size with easier escape from unstable stationary points and sharp minimizers (Hu et al., 2017). Piecewise-deterministic continuous-time models similarly represent SGD as a gradient flow coupled to an index-switching Markov process, yielding weak convergence to full gradient flow as learning rate vanishes and exponential ergodicity in Wasserstein distance for constant learning rate (Latz, 2020). For continuously indexed data, the stochastic gradient process extends this construction by coupling gradient flow to reflected diffusions, pure jump processes, or other Lévy processes on compact spaces (Jin et al., 2021).

Taken together, these works show that approximate-gradient methods are naturally studied through stochastic-process formalisms. SAGD fits squarely within this program, but with a distinctive architecture: the continuous-time process is used to approximate the latent sampling problem that underlies the gradient, rather than merely to approximate the outer parameter recursion.

6. Practical implementation, applications, and limitations

The implementation guidance given for SAGD is unusually explicit. The paper recommends warm-starting the Langevin chain with an educated initial value, using persistent chains rather than resetting at each outer iteration, and replacing vanilla SGD updates by accelerated gradient steps such as Adam or momentum-based methods. Although the theory suggests increasing $\xi$8, the same source states that in practice moderate, possibly fixed, chain lengths after warm start are often sufficient (Qiu et al., 2020).

Two application domains are emphasized. In expectation-maximization, SAGD uses Langevin sampling to approximate the conditional expectation in the E-step, yielding what the paper describes as a truly automated EM procedure. In synthetic simulations, the path of parameters under SAGD closely tracks exact gradient descent with access to the true gradient (Qiu et al., 2020). In variational autoencoders, SAGD is used to fine-tune a pretrained model by optimizing the true marginal likelihood via Langevin-based E-steps. In simulations recovering latent distributions, SAGD is reported to produce estimated latent distributions closest to truth as measured by Kolmogorov-Smirnov and Wasserstein distances, and on high-dimensional synthetic data and MNIST it further improves the quality of generated samples (Qiu et al., 2020).

The principal limitations follow directly from the method’s defining mechanism. The estimator is biased at finite chain length; convergence theory therefore depends on schedules for $\xi$9 and $\pi(\xi)$0, and practical performance depends on the quality of the Langevin approximation. The method also presupposes access to the log-density of $\pi(\xi)$1, since the Langevin dynamics require $\pi(\xi)$2. These are not incidental details but structural requirements. A concise way to state the method’s trade-off is that SAGD exchanges exact stochastic gradients for a controlled inner sampling problem.

In that form, stochastic approximate gradient descent is best understood not as a minor variant of SGD, but as a general optimization strategy for expectation-based models with intractable latent expectations. Its defining contribution is to turn “gradient estimation by inference” into a schedulable, analyzable outer-loop descent method, thereby connecting stochastic approximation to Langevin simulation in a way that retains both optimization guarantees and practical scalability (Qiu et al., 2020).