---
title: Kinetic Langevin Monte Carlo
url: https://www.emergentmind.com/topics/kinetic-langevin-monte-carlo-klmc
type: topic
---

# Kinetic Langevin Monte Carlo

Kinetic Langevin Monte Carlo (KLMC), also called underdamped or kinetic Langevin Monte Carlo, is a class of Monte Carlo algorithms obtained by discretizing the kinetic Langevin diffusion, an underdamped Langevin system on position–velocity space whose invariant law has Gibbs form and whose position marginal is the target distribution [1807.09382]. In the standard Euclidean formulation, the target density is \(\pi(x)\propto e^{-f(x)}\), the method augments \(x\) with a velocity variable \(v\), and the dynamics combine inertial transport, linear friction, and Gaussian noise injected into the velocity equation rather than directly into position space [2509.00941]. Subsequent work has developed KLMC in several directions: nonasymptotic \(W_2\) theory for smooth strongly log-concave targets, convexification-based extensions to merely log-concave targets, higher-order and splitting discretizations, structure-preserving formulations on Lie groups, and specialized variants for latent-variable inference, random regime switching, and unbiased estimation with inexact gradients [1906.08530].

## 1. Continuous-time formulation and invariant measure

In its standard form, the kinetic Langevin diffusion evolves according to
\[
dV_t=-\gamma V_t\,dt-\nabla f(X_t)\,dt+\sqrt{2\gamma}\,dB_t,\qquad dX_t=V_t\,dt,
\]
with friction coefficient \(\gamma>0\) and Brownian motion acting only on the velocity variable [2509.00941]. The corresponding invariant law on phase space is
\[
\pi(v,x)\propto e^{-f(x)-\frac12\|v\|^2},
\]
more precisely \(\mathcal N(0,I_d)\otimes \pi\), where \(\pi(x)\propto e^{-f(x)}\) is the position marginal [2509.00941].

This formulation is the defining structural distinction between KLMC and overdamped Langevin Monte Carlo. Overdamped Langevin evolves directly in position space, whereas KLMC introduces momentum and therefore a second-order dynamics. In the Euclidean setting, the underdamped structure is described in the literature as adding momentum and making the dynamics nonreversible; the friction damps momentum while calibrating the Gaussian noise so that the Gibbs law remains invariant [2509.00941]. A recurring interpretation is that underdamped dynamics can transport mass ballistically in the position variable instead of relying only on local diffusion [2509.00941].

A central early result is the geometric mixing analysis of the continuous-time diffusion for smooth strongly log-concave targets. Under
\[
mI_p\preceq \nabla^2 f(x)\preceq MI_p,
\]
the position marginal contracts exponentially in \(W_2\); in particular, if \(\gamma\ge \sqrt{M+m}\), then
\[
W_2(\mu_t^L,\mu_t^{\prime\,L}) \le \sqrt{2}\exp\!\left(-\frac{m}{\gamma}t\right)W_2(\mu,\mu')
\]
[1807.09382]. That contraction estimate became a basic ingredient in subsequent KLMC discretization analyses.

## 2. Discretization principles and core algorithms

A standard KLMC discretization is not the naive Euler discretization of the second-order SDE. The classical construction freezes the gradient over a time step and integrates the resulting linear Ornstein–Uhlenbeck system exactly. With
\[
\psi_0(t)=e^{-\gamma t},\qquad \psi_{k+1}(t)=\int_0^t \psi_k(s)\,ds,
\]
the update takes the form
\[
\begin{bmatrix} V_{k+1}\\ L_{k+1} \end{bmatrix}
=
\begin{bmatrix}
\psi_0(h)V_k-\psi_1(h)\nabla f(L_k)\\
L_k+\psi_1(h)V_k-\psi_2(h)\nabla f(L_k)
\end{bmatrix}
+\sqrt{2\gamma}
\begin{bmatrix}
\xi_{k+1}\\ \xi'_{k+1}
\end{bmatrix},
\]
where the Gaussian perturbations have covariance determined by the exact integrated Ornstein–Uhlenbeck part [1807.09382]. This matrix-exponential structure is one of the defining algorithmic signatures of KLMC.

A second line of development replaces first-order freezing of the force by higher-order local models. In the strongly convex Euclidean setting, KLMC2 uses Hessian information through
\[
H_k=\nabla^2 f(L_k)
\]
and auxiliary functions \(\varphi_2,\varphi_3\), yielding a second-order kinetic discretization with a smaller discretization floor in Wasserstein distance [1807.09382]. In the convex but not strongly convex setting, the same first-order and second-order distinction reappears after strong convexification of the potential [1906.08530].

A complementary viewpoint treats KLMC as a splitting method. One paper writes the kinetic Langevin diffusion as a Lie–Trotter splitting between a deterministic Hamiltonian flow and an Ornstein–Uhlenbeck velocity refreshment step
\[
O_h(\mathsf b)(x,v)=\left(x,\ e^{-\gamma h}v+\bigl(1-e^{-2\gamma h}\bigr)^{1/2}\mathsf b\right),
\]
and then improves the Hamiltonian subflow discretization using randomized Runge–Kutta–Nyström integrators [2310.07399]. That work introduces \(5/2\)- and \(7/2\)-order \(L^2\)-accurate randomized Runge–Kutta–Nyström methods, establishes quantitative \(5/2\)-order \(L^2\)-accuracy upper bounds, and uses the resulting chain as an unadjusted KLMC sampler [2310.07399].

Another important discretization family is the stochastic exponential Euler discretization, or exponential integrator. In this scheme the velocity damping is treated exactly and the nonlinear force is frozen over each step. A refined analysis later showed that this exponential integrator remains well behaved not only in the usual underdamped regime but also all the way to the overdamped limit, provided one applies the correct time acceleration [2510.03949].

## 3. Nonasymptotic convergence theory

The basic nonasymptotic KLMC guarantee for smooth strongly log-concave targets is a two-term Wasserstein bound combining geometric decay from initialization and a discretization floor. Under
\[
mI_p\preceq \nabla^2f(x)\preceq MI_p,\qquad \gamma\ge \sqrt{m+M},\qquad h\le \frac{m}{4\gamma M},
\]
the law \(\nu_k\) of the KLMC position iterate satisfies
\[
W_2(\nu_k,\pi) \le \sqrt{2}\Big(1-\frac{0.75\,mh}{\gamma}\Big)^k W_2(\nu_0,\pi) + \frac{Mh\sqrt{2p}}{m}
\]
[1807.09382]. With \(\gamma=\sqrt{m+M}\), the resulting iteration complexity improves prior underdamped analyses in its dependence on the condition number \(\kappa=M/m\) [1807.09382].

When Hessian Lipschitzness is available,
\[
\|\nabla^2 f(x)-\nabla^2 f(y)\| \le M_2\|x-y\|_2,
\]
KLMC2 replaces the first-order discretization floor by an \(h^2\)-scale floor. Under explicit step-size restrictions, the law \(\nu_k^{\rm KLMC2}\) satisfies
\[
W_2(\nu_k^{\rm KLMC2},\pi) \le \sqrt{2}\Big(1-\frac{mh}{4\gamma}\Big)^k W_2(\nu_0,\pi)
+\frac{2h^2M_2p}{m}
+\frac{h^2M\sqrt{2Mp}}{m}
+\frac{8M}{m}h e^{-p/2}
\]
[1807.09382]. In the regime where \(M\) and \(M_2\) are bounded and \(m\) is bounded away from \(0\), the paper states that KLMC2 achieves precision \(\varepsilon\) in \(K_\varepsilon\asymp \sqrt{p/\varepsilon}\) iterations, up to logarithmic factors [1807.09382].

The later analysis of the exponential integrator sharpened this picture rather than changing the algorithmic object. For the phase-space weighted norm
\[
\|z\|_{a,b}^2=\|x\|^2+2b\langle x,v\rangle+a\|v\|^2,\qquad a=\frac{4}{\gamma^2},\quad b=\frac{1}{\gamma},
\]
it established a contraction
\[
W_{a,b}^2(\mu K^n,\nu K^n)\le (1-c(h,\gamma,\eta))^n W_{a,b}^2(\mu,\nu)
\]
under weaker parameter restrictions than previous work [2510.03949]. The same paper also derived asymptotic bias bounds for the invariant measure \(\pi_h\) of the discretized chain, and showed that in the overdamped scaling \(h=h_{\mathrm{LMC}}\gamma\), \(\eta=1\), \(\gamma\to\infty\), both the contraction rate and the bias converge to the corresponding overdamped LMC behavior [2510.03949]. This directly counters the earlier impression that the exponential integrator degenerates in the overdamped regime.

One persistent point in the literature is that KLMC does not uniformly dominate overdamped LMC. The strongly log-concave analysis explicitly states that KLMC is preferable when \(p/(m\varepsilon^2)\) is large relative to \(\kappa\), whereas LMC can be preferable when \(\kappa\) is very large [1807.09382]. This is a methodological comparison, not a defect of the theory: the gain from momentum is accompanied by its own parameter sensitivity.

## 4. Extensions beyond the classical strongly log-concave Euclidean setting

A substantial extension concerns smooth log-concave targets that are not necessarily strongly log-concave. One paper analyzes KLMC in this regime by replacing the original potential \(f\) with the strongly convex surrogate
\[
f_\alpha(\theta)=f(\theta)+\frac{\alpha}{2}\|\theta\|_2^2,
\]
with corresponding density \(\pi_\alpha\), and then decomposing the error into a strongly convex KLMC term plus the surrogate bias between \(\pi_\alpha\) and \(\pi\) [1906.08530]. A basic bound is
\[
W_q^q(\pi,\pi_\alpha)\le C_q\,\alpha\,\mu_2(\pi)^{q+2},
\]
for \(q\in[1,\infty)\), with \(C_1\le 11\) and \(C_2\le 111\) [1906.08530]. This strong-convexification trick is the backbone of the non-strongly log-concave theory for both KLMC and KLMC2.

The resulting \(\alpha\)-KLMC bound has the standard three-term form: finite-time error, discretization error, and convexification bias. For \(q\in[1,2]\), under explicit restrictions on \(\alpha\), \(\gamma\), and \(h\),
\[
W_q(\nu_K^{\alpha\text{-KLMC}},\pi)
\le
\sqrt{2}\,\mu_2\Big(1-\frac{3\alpha h}{4\gamma}\Big)^K
+1.5\,M\,p^{1/2}\frac{h}{\alpha}
+\big(C_q\alpha\mu_2^{q+2}\big)^{1/q}
\]
[1906.08530]. Under the paper’s scale-adapted Wasserstein complexity notion, this yields polynomial dependence on the dimension.

A more geometric extension places KLMC on compact Lie groups. There the target on the group is
\[
\pi(dg)\propto e^{-U(g)}\,dg,
\]
and left trivialization identifies \(TG\) with \(G\times\mathfrak g\), so the momentum variable lives in a fixed Euclidean Lie algebra \(\mathfrak g\) [2403.12012]. The resulting invariant Gibbs law is
\[
\nu_*(dg,d\xi)=\frac1Z \exp\!\left(-U(g)-\frac12\langle \xi,\xi\rangle\right)\,dg\,d\xi
\]
[2403.12012]. This framework gives a structure-preserving discretization that updates the group element by
\[
g_{k+1}=g_k\exp(h\xi_{k+1})
\]
on matrix Lie groups, preserves the manifold exactly, and proves exponential convergence in Wasserstein distance under compactness of the Lie group and geodesically \(L\)-smoothness of the potential function, without assuming convexity, geodesic convexity, log-concavity, or explicit isoperimetric inequalities [2403.12012].

These two extensions show that the phrase “KLMC theory” no longer refers only to Euclidean smooth strongly log-concave sampling. In the convex but not strongly convex Euclidean setting the key device is surrogate strong convexity; on compact Lie groups the key device is left trivialization and a custom semimetric rather than convexity.

## 5. Variant families and problem-specific generalizations

Several recent variants enlarge the KLMC design space while preserving the underdamped Gibbs target.

| Variant | Defining feature | Representative source |
|---|---|---|
| KLMC2 | Second-order kinetic scheme using Hessian information | [1807.09382] |
| RS-KLMC | KLMC with random step sizes \(h_k=\beta_k\eta\) | [2509.00941] |
| FRS-KLMC | KLMC with random friction coefficients \(\gamma_k\) | [2509.00941] |
| KIPLMC | Latent-variable/interacting-particle extension of KLMC | [2407.05790] |
| Lie-group KLMC | Structure-preserving kinetic Langevin sampler on compact Lie groups | [2403.12012] |

The regime-switching variants randomize either the effective step scale or the friction through a finite-state continuous-time Markov chain. In RS-KLMC, the regime process multiplies the whole physical time scale, so the only algorithmic change relative to vanilla KLMC is that the effective step becomes random:
\[
h_k=\beta_k\eta
\]
[2509.00941]. In FRS-KLMC, the physical step is fixed and the friction coefficient becomes a finite-state Markov chain. Both preserve the same Gibbs target, and both come with non-asymptotic \(W_2\) guarantees whose rates depend on spectral quantities such as \(\mathbf Q-\frac{m}{\gamma}\Lambda\) and \(\mathbf Q-2m\Lambda_\gamma^{-1}\) [2509.00941].

A conceptually different generalization is Kinetic Interacting Particle Langevin Monte Carlo for maximum marginal likelihood estimation in latent variable models. Its central idea is to turn MMLE into sampling from an interacting underdamped diffusion whose stationary \(\theta\)-marginal concentrates around the MMLE as the number of particles \(N\) grows [2407.05790]. The paper is explicit that it is best viewed as a latent-variable/interacting-particle extension of KLMC, with the number of particles playing the role of an inverse temperature [2407.05790]. After rescaling, the interacting diffusion becomes a standard underdamped Langevin SDE with potential \(N\bar U_N\), so classical KLMC contraction and discretization theory can be imported into the latent-variable setting [2407.05790]. A crucial clarification is that KIPLD does not target the posterior of \(\theta\); rather, it targets a temperature-controlled surrogate
\[
\pi_\Theta(\theta)\propto \exp(-N\kappa(\theta)),
\qquad \kappa(\theta)=-\log p_\theta(y),
\]
whose mode is the MMLE [2407.05790].

Another direction concerns unbiased estimation. One paper presents an unbiased method for Bayesian posterior means based on kinetic Langevin dynamics that combines advanced splitting methods with enhanced gradient approximations, avoids Metropolis correction by coupling Markov chains at different discretization levels in a multilevel Monte Carlo approach, and proves that the estimator is unbiased, attains finite variance, and satisfies a central limit theorem [2311.05025]. Its expected complexity for estimating expectations of Lipschitz functions in \(d\) dimensions is \(\mathcal O(d^{1/4}\epsilon^{-2})\) expected gradient evaluations, with similar bounds using both approximate and stochastic gradients, and the computational cost is shown to scale independently of the size of the dataset [2311.05025]. In the supplied description of that work, the UBU splitting scheme is emphasized because its strong second order property and generally high accuracy make it suitable for unbiased estimation [2311.05025].

## 6. Empirical behavior, applications, and limitations

The empirical literature presents a consistent but qualified picture. In strongly log-concave theory, the main advertised advantage of KLMC over LMC is improved dependence on precision and dimension in regimes where the underdamped dynamics can exploit smoother trajectories and smaller discretization error [1807.09382]. In splitting-based unadjusted KLMC, randomized high-order Hamiltonian integrators substantially improve simulation efficiency relative to Verlet and an earlier \(3/2\)-order randomized method; on a logistic regression posterior, the reported bias at fixed gradient budget is much smaller for the new randomized Runge–Kutta–Nyström schemes [2310.07399].

For regime-switching KLMC, the practical message is that switching can exploit beneficial regimes while preserving the target and keeping the per-iteration cost essentially the same as KLMC plus a cheap finite-state Markov chain update [2509.00941]. The experiments most directly relevant to KLMC show that RS-KLMC with a generator matrix having large spectral gap accelerates convergence in MSE relative to standard KLMC in Bayesian linear regression, while FRS-KLMC can accelerate convergence if the friction regime values are sufficiently large and well chosen [2509.00941].

For KIPLMC, the experiments compare MPGD, KIPLMC1, and KIPLMC2. On a synthetic Bayesian logistic regression latent-variable model, all methods concentrate around the true \(\bar\theta_\star\) as \(N\) increases and the variance decreases approximately as \(\mathcal O(1/N)\); KIPLMC2 has variance comparable to MPGD and far smaller than KIPLMC1 [2407.05790]. On the Wisconsin breast cancer data, the most relevant message is numerical stability: as the step size grows, MPGD becomes unstable and explodes around \(\eta=0.05\), KIPLMC1 also eventually becomes unstable, and KIPLMC2 remains stable over a much wider step-size range [2407.05790].

The limitations are equally clear and vary by setting. The classical Euclidean KLMC guarantees are for smooth strongly log-concave targets, and KLMC2 requires additional Hessian Lipschitzness [1807.09382]. The convex-but-not-strongly-convex extension relies on strong convexification rather than a direct convex-only contraction analysis, and the numerical constants in the bounds are explicitly described as too large for direct practical use [1906.08530]. The regime-switching theory is only for smooth, strongly convex targets and does not solve the design problem of choosing optimal regime values or generator matrix [2509.00941]. The Lie-group theory is restricted to compact connected Lie groups, and the convergence rate may be exponentially small in geometric scale under such weak assumptions [2403.12012]. KIPLMC has substantial limitations as well: the theory requires global strong convexity and global Lipschitz gradients, the computational cost scales linearly in the number of particles \(N\), and no nonconvex theory is proved there [2407.05790].

Two misconceptions are repeatedly corrected by the literature. The first is that KLMC is automatically superior to overdamped LMC; the strongly log-concave comparison does not support that conclusion uniformly [1807.09382]. The second is that the exponential-integrator version of KLMC degenerates in the overdamped regime; the later refined analysis shows that, with proper time acceleration, it converges to the expected overdamped LMC behavior instead of breaking down [2510.03949].

Taken together, these results define KLMC as more than a single algorithm. It is a family of underdamped Langevin methods centered on phase-space Gibbs sampling, exact treatment of linear velocity damping, and nonasymptotic Wasserstein analysis. The common structure is stable across Euclidean sampling, convexification-based theory, high-order splitting, Lie-group geometry, regime switching, latent-variable interacting-particle inference, and unbiased multilevel estimators with inexact gradients.

Source: https://www.emergentmind.com/topics/kinetic-langevin-monte-carlo-klmc