KL-Accelerated Tamed ULA (kTULA)

• The paper introduces kTULA, a tamed Langevin sampler that uses split drift taming to control super-linear log-gradient growth and preserve dissipativity.
• It achieves a nearly second-order bias in KL divergence while providing explicit Wasserstein-2 and optimization guarantees under LSI and local polynomial regularity conditions.
• The method combines explicit drift damping, strict step-size constraints, and rigorous entropy-based analysis to efficiently sample non-convex, non-globally-smooth target distributions.

KL-Accelerated Tamed Unadjusted Langevin Algorithm (kTULA) is a tamed explicit Langevin sampler for Gibbs measures whose log-gradients grow super-linearly, a regime in which the global Lipschitz assumptions used in standard analyses of the unadjusted Langevin algorithm (ULA) are not available and Euler-type discretizations may become unstable. In its original formulation, kTULA is designed for targets of the form

$$\pi_\beta(A) \coloneqq \frac{\int_A e^{-\beta u(\theta)}\, d\theta}{\int_{\mathbb{R}^d} e^{-\beta u(\theta)}\, d\theta}, \qquad A\in\mathcal B(\mathbb R^d),$$

with $u\in C^3$, $h=\nabla u$, and $H=\nabla^2u$, under dissipativity, polynomial local regularity, and a log-Sobolev inequality (LSI). Its distinguishing feature is a split taming of the drift that preserves dissipativity while yielding a non-asymptotic Kullback-Leibler bound with bias exponent arbitrarily close to $2$, together with corresponding Wasserstein-$2$ and optimization guarantees (Lytras et al., 5 Jun 2025).

1. Problem class and motivation

kTULA addresses sampling from non-log-concave and non-globally-smooth targets for which the gradient of the log-density may grow faster than linearly at infinity. In the Gibbs formulation, the reference continuous-time process is the overdamped Langevin SDE

$dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$

whose invariant law is $\pi_\beta$. The standard ULA discretization,

$$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$

is a natural Euler–Maruyama approximation in globally Lipschitz settings, but under super-linear $\nabla U$ it can blow up (Lytras et al., 5 Jun 2025).

This instability is central rather than peripheral. Earlier work on tamed Langevin methods was motivated by the observation that deep-learning objectives and other non-convex energies often fail standard Lipschitz smoothness hypotheses, while still satisfying weaker dissipativity and local polynomial regularity conditions. In that regime, taming is introduced to control numerical explosion without changing the target measure being approximated (Lytras et al., 2024).

The phrase “KL-accelerated” has a specific meaning in the kTULA literature. In the 2025 formulation, it refers to the fact that the discretization error in KL divergence has bias order $u\in C^3$0, with $u\in C^3$1 arbitrarily small, improving the $u\in C^3$2 KL bias typical of earlier tamed Langevin analyses under super-linear drifts. This usage differs from the older underdamped-Langevin literature, where “acceleration” in KL geometry refers to a hypocoercive, momentum-based analogue of accelerated gradient descent on the space of measures. This suggests that “KL acceleration” is not a single standardized notion across sampling theory (Lytras et al., 5 Jun 2025, Ma et al., 2019).

2. Structural assumptions

The original kTULA analysis imposes three principal classes of conditions on the potential and the target law. First, the Hessian is required to satisfy a polynomial Lipschitz condition: there exist $u\in C^3$3 and $u\in C^3$4 such that for all $u\in C^3$5,

$u\in C^3$6

In addition, there exist $u\in C^3$7 such that

$u\in C^3$8

These bounds imply a local polynomial Lipschitz condition on $u\in C^3$9 (Lytras et al., 5 Jun 2025).

Second, kTULA assumes dissipativity or coercivity: $h=\nabla u$0 for some $h=\nabla u$1 and all $h=\nabla u$2. This condition enforces a restoring drift at large radius and is the structural mechanism that allows the tamed drift to inherit a usable Lyapunov structure (Lytras et al., 5 Jun 2025).

Third, the target law $h=\nabla u$3 is assumed to satisfy an LSI with constant $h=\nabla u$4. In the paper’s formulation, this inequality is used to convert Fisher-information dissipation into exponential KL contraction and, through Talagrand’s $h=\nabla u$5 inequality,

$h=\nabla u$6

to obtain Wasserstein-$h=\nabla u$7 error bounds from KL estimates (Lytras et al., 5 Jun 2025).

An initialization condition is also imposed: the initial law has exponential tails, and $h=\nabla u$8 and $h=\nabla u$9 have polynomial growth. The role of this assumption is technical but consequential: it supports the regularity and moment estimates required by the entropy method (Lytras et al., 5 Jun 2025).

These hypotheses are stronger than the weakest known assumptions under which taming can stabilize Langevin sampling. Earlier work developed tamed schemes under local polynomial Lipschitz continuity, weak dissipativity, and either a Poincaré or a log-Sobolev inequality, with separate regimes depending on whether weak convexity or regularization is available. kTULA specializes to the LSI-driven regime and uses stronger second-order regularity to obtain sharper KL bias exponents (Lytras et al., 2024).

3. Algorithmic construction

The key construction in kTULA is a split-tamed drift

$H=\nabla^2u$0

The linear term $H=\nabla^2u$1 is left untouched, while only the super-linear remainder $H=\nabla^2u$2 is damped. This preserves dissipativity and enforces at-most-linear growth in the tamed drift (Lytras et al., 5 Jun 2025).

The resulting Markov chain is

$H=\nabla^2u$3

The paper proves several basic properties of $H=\nabla^2u$4: inherited dissipativity,

$H=\nabla^2u$5

linear-growth control,

$H=\nabla^2u$6

a global Lipschitz bound

$H=\nabla^2u$7

and a taming bias estimate

$H=\nabla^2u$8

(Lytras et al., 5 Jun 2025).

The admissible step size is constrained by

$H=\nabla^2u$9

Within this range, the scheme is stable enough for the non-asymptotic entropy analysis. The recommended default in the original implementation guidance is $2$0, with $2$1 matched to the dissipativity constant and $2$2 matched to the degree of polynomial growth in the Hessian (Lytras et al., 5 Jun 2025).

A later paper studies the same sampler through a local-error framework, using the special-case taming

$2$3

and emphasizes that kTULA has one drift evaluation per step, in contrast with a tamed randomized midpoint scheme that requires two evaluations (Lytras et al., 24 May 2026).

4. Non-asymptotic KL, Wasserstein, and optimization guarantees

The central theorem for kTULA is a non-asymptotic KL bound. Under Assumptions 2.1–2.5 of the paper, for any $2$4, $2$5, and $2$6, there exist explicit positive constants $2$7 such that, for all $2$8 and $2$9,

$2$0

Here $2$1, while $2$2 is explicit and depends polynomially on the dimension $2$3, on $2$4, and on the growth and dissipativity parameters. The bias exponent can be written as $2$5, where

$2$6

so the rate can be made arbitrarily close to second order, at the price of worsening constants (Lytras et al., 5 Jun 2025).

Via Talagrand’s $2$7 inequality, the KL estimate implies a Wasserstein-$2$8 bound: $2$9 with $dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$0. Consequently, the $dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$1 bias scales as

$dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$2

which is arbitrarily close to first order (Lytras et al., 5 Jun 2025).

The same paper derives an optimization corollary by working in the low-temperature regime. For large $dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$3, $dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$4 concentrates near global minimizers of $dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$5, and kTULA satisfies

$dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$6

The final term is the low-temperature bias; the first two are the finite-time sampling and discretization contributions (Lytras et al., 5 Jun 2025).

Later work sharpened the iteration-complexity picture. Using a shifted-composition local-error analysis, a 2026 paper obtained a finite-time KL bound against the diffusion of the form

$dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$7

and, in the sampling specialization under LSI, deduced a near-optimal $dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$8 iteration complexity to reach $dZ_t = -h(Z_t)\,dt + \sqrt{2\beta^{-1}}\,dB_t,$9. The same paper then transferred this bound to total variation and Wasserstein distance via Pinsker and Talagrand inequalities (Lytras et al., 24 May 2026).

5. Analytical mechanism

The proof architecture of kTULA is entropy-based. The starting point is the derivative of KL divergence along a continuous-time interpolation of the chain: $\pi_\beta$0 The first term is dissipative; the second is the discretization/taming defect that must be controlled (Lytras et al., 5 Jun 2025).

A key step is a Bayes or integration-by-parts decomposition of the conditional drift mismatch,

$\pi_\beta$1

The term $\pi_\beta$2 is the principal stochastic contribution and admits the representation

$\pi_\beta$3

Through Hölder and Young inequalities, one obtains

$\pi_\beta$4

so the quality of the final KL rate depends on a uniform control of the discrete Fisher information $\pi_\beta$5 (Lytras et al., 5 Jun 2025).

That control is obtained using a convolution inequality due to Rioul. Writing $\pi_\beta$6, with $\pi_\beta$7 the density of $\pi_\beta$8 for $\pi_\beta$9 and $$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$0 the Gaussian kernel $$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$1, the paper proves

$$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$2

which leads to

$$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$3

Substituting this into the $$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$4 estimate produces the exponent $$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$5, while the remaining terms $$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$6 are controlled at order $$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$7 by moment bounds, global Lipschitz regularity of $$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$8, and the $$X_{k+1} = X_k - h \,\nabla U(X_k) + \sqrt{2h}\,\xi_k, \qquad \xi_k\sim\mathcal N(0,I_d),$$9 taming error (Lytras et al., 5 Jun 2025).

This analytical route differs from the earlier sTULA analysis, which also used an LSI-driven entropy differential inequality but produced an $\nabla U$0 KL bias and an $\nabla U$1 Wasserstein-$\nabla U$2 bias. The later local-error approach of 2026 recasts the analysis through shifted composition and state-dependent coupling increments, again leveraging dissipativity for uniform moment control, but now at the level of finite-time comparison between the discrete kernel and the diffusion semigroup (Lytras et al., 2023, Lytras et al., 24 May 2026).

6. Canonical examples, comparative context, and limitations

The first canonical application in the original paper is the high-dimensional double-well potential

$\nabla U$3

For this model, the paper verifies Assumption 2.2 with $\nabla U$4, $\nabla U$5, $\nabla U$6, $\nabla U$7, and the dissipativity inequality

$\nabla U$8

so $\nabla U$9 and $u\in C^3$00. It also states that LSI holds by convexity at infinity and decomposition arguments. With $u\in C^3$01, the corresponding constant is

$u\in C^3$02

and the step-size cap is governed by the term $u\in C^3$03; the implementation template therefore recommends $u\in C^3$04 (Lytras et al., 5 Jun 2025).

The second application is a non-convex neural-network objective,

$u\in C^3$05

for a one-hidden-layer network with fixed pretrained $u\in C^3$06. In this example, the paper verifies Assumptions 2.2–2.5, takes $u\in C^3$07, $u\in C^3$08, and $u\in C^3$09, and interprets kTULA as a low-temperature optimizer through the excess-risk bound above (Lytras et al., 5 Jun 2025).

As practical guidance, the paper recommends Gaussian or deterministic initialization satisfying the exponential-tail condition, monitoring empirical moments $u\in C^3$10 via batch estimates, enforcing $u\in C^3$11, and, when needed, using decreasing step-size schedules to reduce bias over long runs. For diagnostics, it mentions empirical KL via variational estimators when the normalizing constant is unknown, sliced-Wasserstein proxies for $u\in C^3$12, autocorrelation times, and effective sample size. It also explicitly notes that the main focus is theory (Lytras et al., 5 Jun 2025).

In comparative terms, kTULA advances a line of tamed Langevin methods developed for super-linear drifts. The 2024 paper “Tamed Langevin sampling under weaker conditions” analyzes wd-TULA and reg-TULA under local polynomial Lipschitz continuity, weak dissipativity, and either LSI or Poincaré-type assumptions, obtaining $u\in C^3$13 KL bias in the LSI regime and weaker complexity under PI-based regimes (Lytras et al., 2024). The 2023 sTULA paper likewise derives an LSI-based entropy inequality and obtains exponential KL decay with an $u\in C^3$14 bias floor (Lytras et al., 2023). Against this backdrop, the main technical novelty of kTULA is the improvement from first-order KL bias to $u\in C^3$15, together with the induced near-first-order Wasserstein bias (Lytras et al., 5 Jun 2025).

The limitations are also explicit. The LSI assumption is stronger than a Poincaré inequality and may fail in highly multimodal landscapes without convexity-at-infinity or decomposition arguments. The constants in the bounds depend polynomially on $u\in C^3$16 and $u\in C^3$17, and pushing the exponent arbitrarily close to $u\in C^3$18 worsens this dependence. Moreover, $u\in C^3$19 can be very small when the polynomial-growth constants are large, so the method remains stable but may require tiny steps. The paper identifies as open directions the removal of LSI in favor of weaker functional inequalities, adaptive taming parameters, kinetic variants under non-Lipschitz drift, and Metropolis-adjusted tamed schemes with explicit non-asymptotic KL rates (Lytras et al., 5 Jun 2025).

A later development extends the perspective further: under locally Lipschitz super-linear drift and LSI, kTULA can be analyzed through a KL local-error framework yielding $u\in C^3$20 complexity in KL, while a separate tamed randomized midpoint method, tRLMC, achieves $u\in C^3$21 complexity in total variation and Wasserstein distance. This suggests that kTULA has become a reference point for a broader program of high-order or near-high-order non-Lipschitz Langevin discretization theory (Lytras et al., 24 May 2026).