No-U-Turn Sampler (NUTS): Adaptive MCMC

Updated 17 December 2025

No-U-Turn Sampler (NUTS) is an adaptive MCMC method that automatically determines trajectory lengths via a U-turn criterion to explore high-dimensional, complex distributions effectively.
It uses reversible, volume-preserving integrators combined with slice sampling to achieve rapid mixing and maintain the target measure without manual tuning.
Recent innovations such as local step-size adaptation, parallel implementations, and surrogate-gradient strategies have significantly enhanced NUTS performance in challenging statistical models.

The No-U-Turn Sampler (NUTS) is an adaptive Markov chain Monte Carlo (MCMC) method that eliminates the need to tune the trajectory length in Hamiltonian Monte Carlo (HMC), enabling robust, scalable inference in high-dimensional and geometrically complex target distributions. By implementing a geometric stopping rule that detects when a trajectory has begun to retrace its steps (“U-turn”), NUTS achieves rapid, non-random-walk exploration, sidesteps detailed manual tuning, and preserves the target measure via reversible, volume-preserving integrator dynamics with slice-augmented selection mechanisms. Recent theoretical and empirical studies establish its invariance, ergodicity, and even accelerated mixing rates under appropriate conditions, while a growing range of research extends NUTS to locally adaptive step-size regimes, parallel and surrogate-gradient settings, and nonstandard mixing diagnostics.

1. Theoretical Foundations and Algorithmic Structure

NUTS is built upon the HMC framework, in which one samples an auxiliary momentum $p$ (typically Gaussian) and simulates Hamiltonian dynamics under a Hamiltonian $H(\theta, p) = U(\theta) + K(p) = -\log \pi(\theta) + ½ p^\top M^{-1} p$ . Proposals are generated by integrating Hamilton's equations via a volume-preserving, reversible symplectic integrator, usually the leapfrog method, and accepting according to a Metropolis criterion that compensates for integration error (Hoffman et al., 2011, Durmus et al., 2023).

NUTS replaces HMC’s fixed trajectory length $L$ (equivalently, integration time $T$ ) with an adaptive, binary-tree–based exploration:

At each iteration, NUTS grows a trajectory both forward and backward, recursively doubling its length.
At each doubling, the No-U-Turn Criterion is checked: write the two ends of the current tree as $(\theta^-, p^-)$ and $(\theta^+, p^+)$ , then halt further extension if

$(\theta^+ - \theta^-)\cdot p^- < 0 \quad\text{or}\quad (\theta^+ - \theta^-)\cdot p^+ < 0,$

which geometrically detects when further movement will begin to reduce the separation between the ends—i.e., a U-turn (Hoffman et al., 2011, Modi, 28 Oct 2024).

Along the built “orbit” (set of all leapfrog states explored before stopping), a candidate next state is selected among those that pass a slice test: a slice variable $u$ is sampled uniformly over $[0, \exp(-H(\theta_0, p_0))]$ , and only states with $\exp(-H(\theta, p)) \geq u$ are eligible; typically, one is sampled either uniformly or with weights $\propto \exp(-H(\theta, p))$ .
Volume preservation and reversibility of leapfrog ensure that the Markov kernel is π-invariant.

Dual-averaging adaptation of the step size $\epsilon$ is performed during “warmup” to achieve a target acceptance probability, and the mass matrix is often adapted as a full or diagonal covariance of previous samples, then kept fixed (Hoffman et al., 2011, Grumitt et al., 2019).

2. Invariance, Ergodicity, and Mixing Properties

The theoretical justification and convergence properties of NUTS have been analyzed extensively in the context of so-called dynamic HMC, a general class of MCMC methods in which the proposal orbit and index-selection steps are random and adapted on the fly (Durmus et al., 2023). NUTS fits precisely into this framework:

Invariance: Under mild regularity conditions (Lipschitz or analytic potential), the NUTS state-selection and tree-growing rules satisfy microscopic reversibility, ensuring that the stationary distribution in position $\theta$ is exactly $\pi(\theta)$ (Durmus et al., 2023).
Ergodicity: Under further mild tail and regularity conditions, NUTS is $\pi$ -irreducible and aperiodic, and hence Harris recurrent and ergodic (Durmus et al., 2023).
Geometric ergodicity: For targets satisfying standard Lyapunov drift and minorization conditions (including Gaussian mixtures and smooth perturbations), NUTS is $V$ -uniformly geometrically ergodic, implying existence of CLTs for averages of functions along the chain and quantitative bounds on autocorrelation decay (Durmus et al., 2023).

Several rigorous analyses demonstrate accelerated mixing properties (“diffusive-to-ballistic” $\sqrt{\cdot}$ speedup) for NUTS when targeting high-dimensional Gaussian measures. For example, for canonical Gaussians, the total iteration complexity to mix to accuracy $\varepsilon$ is $O(d^{1/4} \log^2 d \log^{7/4} \varepsilon^{-1})$ , demonstrating the same scaling as critically tuned randomized HMC, provided the step size is chosen outside resonance bands to avoid catastrophic “looping” (Bou-Rabee et al., 9 Oct 2024, Oberdörster, 17 Jul 2025). In two-scale Gaussians, a sharp phase transition appears: NUTS achieves the optimal mixing rate when the U-turn rule reliably picks a trajectory length on the slowest timescale (Oberdörster, 17 Jul 2025).

3. Adaptive Tuning and Extensions for Complex Target Geometry

While NUTS adaptively selects path length, it assumes a fixed global step size during trajectory building. However, complex targets often exhibit severe inhomogeneity in curvature, leading to the need for fine local step size. Several innovations address this shortcoming:

Within‐Orbit Adaptive Leapfrog NUTS (WALNUTS) (Bou-Rabee et al., 23 Jun 2025): Introduces local step-size adaptation within each segment (“macro step”) of a NUTS trajectory. The algorithm selects the largest step size from a dyadic schedule (e.g., $h/2^k$ ) that keeps the energy error $\Delta H$ in that macro step below a user-defined threshold $\delta$ , thereby stabilizing integration in “stiff” regions (e.g., funnel necks) without globally reducing $h$ . Time reversibility and detailed balance are exactly preserved via a randomized pairing of micro-step schedules in forward and reverse, requiring no Metropolis correction on $\theta$ . Empirically, WALNUTS achieves orders-of-magnitude improvements in tail exploration and divergence control on multiscale targets.
Locally adaptive step size via Gibbs self-tuning (GIST) (Bou-Rabee et al., 15 Aug 2024): Poses the leapfrog step size as a latent variable in an augmented Metropolis-within-Gibbs construction—sampling a step-size adaptation index $k$ per trajectory and metropolizing only on the ratio of local conditional probability of $k$ at the forward and reverse endpoints. This treats step-size selection as a reversible, probabilistically principled variable, rigorously preserving invariance and enabling efficient adaptation to local energy error structure.

Further, ATLAS (Modi, 28 Oct 2024) extends this approach by estimating local curvature via a low-rank Hessian approximation and power iteration, proposing step-size and trajectory length adaptively. A delayed-rejection framework ensures detailed balance, and in highly anisotropic or funnel-shaped targets, this yields superior effective sample size per gradient evaluation by adapting integration scale at each step.

4. Efficient Implementation, Parallelization, and Surrogate-Gradient Variants

Implementing NUTS efficiently in large-scale or high-dimensional settings raises both algorithmic and systems challenges:

Parallel and distributed NUTS: Vectorized NUTS implementations in TensorFlow (Edward2), with parallel gradient evaluation and tree construction across multi-GPU/TPU settings, maintain correctness while achieving nearly linear speedup and outperforming Stan and PyMC3 by one to two orders of magnitude in leapfrog time per sample (Tran et al., 2018). This is achieved by having each device build tree sub-branches independently, with minimal synchronization at key no-U-turn-check points.
NUTS as SMC kernels: NUTS proposals can be incorporated into population-based Sequential Monte Carlo samplers, with a carefully constructed “L-kernel” to minimize weight variance and exploit parallel hardware resources (Devlin et al., 2021). Empirical studies demonstrate competitive or superior performance to traditional SMC moves in high-dimensional settings.
Surrogate-gradient NUTS: Hamiltonian Neural Networks (HNNs) and latent-variable HNNs (L-HNNs) can learn the system Hamiltonian or its components from data, replacing expensive numerical gradients by differentiable neural surrogates that are symplectic and reversible by design (Dhulipala et al., 2022, Dhulipala et al., 2022). Embedding L-HNNs into NUTS, with an online error-monitoring and fallback switch to true gradients when network error exceeds a threshold, yields orders-of-magnitude reductions in total gradient evaluations and boosts ESS per gradient call by an order of magnitude.

5. Practical Performance, Diagnostics, and Limitations

NUTS has established itself as a default MCMC backend for complex Bayesian models across cosmology, hierarchical regression, and machine learning (Grumitt et al., 2019, Mootoovaloo et al., 7 Jun 2024). Typical diagnostic metrics include effective sample size (ESS), autocorrelation times, and the potential scale reduction $\hat{R}$ statistic, with NUTS typically achieving high acceptance rates and sharply improved mixing relative to random-walk and even optimally tuned Metropolis–Hastings algorithms.

In cosmology and large-scale structure, NUTS yields an order-of-magnitude improvement in effective independent samples per likelihood evaluation compared with Metropolis–Hastings (Mootoovaloo et al., 7 Jun 2024), though the wall-clock speedup is moderated by expensive gradient computations unless highly optimized auto-diff and GPU/TPU code is used. Hierarchical models with strong “funnels” or multiscale structure reveal the main practical limitation of vanilla NUTS: fixed global step size leads to integration instability in “narrow necks” and frequent divergent transitions. Adaptive variants such as WALNUTS or step-size–adaptive GIST–NUTS address this, delivering robust sampling in challenging regimes (Bou-Rabee et al., 23 Jun 2025, Bou-Rabee et al., 15 Aug 2024, Modi, 28 Oct 2024).

NUTS has some limitations:

It is not natively compatible with SIMD/GPU-style parallelization due to complex tree recursion (Millard et al., 3 Apr 2025).
It can encounter rare “looping” pathology where, for certain step sizes and orbit lengths, the U-turn condition is never triggered due to resonance, causing pathological non-mixing. Avoidance requires jittering step size or randomizing time grids (Bou-Rabee et al., 9 Oct 2024).
Without step-size adaptation, NUTS underexplores tails in stiff or hierarchical targets due to energy error blow-up in narrow regions.

6. Recent Algorithmic Innovations and Alternative Proposals

Recent work introduces several refinements:

SpreadNUTS grows trajectories via moderately-aried (k-ary) rather than binary trees, reducing the number of U-turn checks, and partitions the trajectory to bias sample selection toward underexplored regions using nearest-neighbor statistics. These maintain stationarity and—especially in moderate or high dimensions—improve mixing as measured by discretized total variation between empirical and true densities (Sheriff, 2023).
ATLAS algorithm jointly adapts trajectory length (via the U-turn condition) and step size (via Hessian-based curvature estimation and stochastic proposal) within a delayed-rejection framework, achieving robust high-ESS sampling on complex, highly curved posteriors with only modest overhead on well-conditioned tasks relative to vanilla NUTS (Modi, 28 Oct 2024).

Alternative proposals, such as ChEES-HMC, offer more efficient GPU parallelization and can supersede NUTS in massively parallel settings (Millard et al., 3 Apr 2025).

7. Empirical Validation, Diagnostics, and Benchmarks

Empirical benchmarks across mixture models, high-dimensional Gaussians, banana functions, funnel densities, cosmological likelihoods, and hierarchical Bayesian regression exhibit the following:

In moderate dimensions ( $d=25$ –$50$), relative to Metropolis–Hastings, NUTS delivers an order-of-magnitude speedup in effective sample size per likelihood or gradient evaluation (Mootoovaloo et al., 7 Jun 2024).
Local step-size adaptation within orbit (WALNUTS, GIST-NUTS, ATLAS) removes bottlenecks and recovers proper marginal and tail structure for stiff hierarchical models (e.g., Neal’s funnel), empirically verified by quantile recovery, ESS, and trace plots (Bou-Rabee et al., 23 Jun 2025, Bou-Rabee et al., 15 Aug 2024, Modi, 28 Oct 2024).
Surrogate-gradient NUTS (L-HNN-NUTS) achieves up to 100× reduction in gradient evaluations and correspondingly higher ESS per gradient; careful error monitoring is necessary to avoid degeneracy in untrained regions (Dhulipala et al., 2022, Dhulipala et al., 2022).
Distributed implementations realize $100\times$ wall-clock speedup over traditional CPU-based NUTS (Tran et al., 2018).
Stationarity diagnostics (potential scale reduction $\hat{R}$ , batch means, and Kolmogorov–Smirnov scores) are routinely satisfied across these studies, including in hierarchical CMB applications (Grumitt et al., 2019).

Empirical studies recommend careful tuning of step size, mass matrix estimation, and, for locally adaptive variants, parameterization of local energy error thresholds or Hessian-rank, acknowledging the trade-off between per-iteration cost and overall exploration efficiency.

Key References: