Zig-Zag Process (ZZP) Overview

• Zig-Zag Process (ZZP) is a nonreversible piecewise deterministic Markov process that alternates linear flows with velocity sign flips to target exact invariant distributions.
• It employs state-dependent switching rates and Poisson thinning to simulate event times without discretisation error for scalable Bayesian computations.
• Extensions like the Speed Up Zig-Zag process and hybrid state space formulations enhance mixing properties and adapt to heavy-tailed targets and big-data applications.

The Zig-Zag Process (ZZP) is a nonreversible piecewise deterministic Markov process (PDMP) used as a continuous-time Monte Carlo sampler on an extended position–velocity state space. In its standard form, the position moves linearly between random event times, while one velocity coordinate flips sign at each event; the switching rates are chosen so that the position marginal has a prescribed target density $\pi(x)\propto e^{-U(x)}$. This construction yields an exact PDMP sampler with invariant law $\pi$ in position and a uniform law on velocities, and it has been studied from the viewpoints of ergodicity, limit theorems, large deviations, scalable Bayesian computation, heavy-tailed targets, numerical implementation, and hybrid discrete–continuous state spaces (Bierkens et al., 2016, Bierkens et al., 2016, Vasdekis et al., 2021, Bierkens et al., 2019).

1. Canonical construction and dynamics

In the multidimensional formulation, the state space is

$E=\mathbb{R}^d\times\{-1,+1\}^d,$

with state $(\xi,\theta)$, where $\xi\in\mathbb R^d$ is position and $\theta\in\{-1,+1\}^d$ is a velocity/sign vector. Between switching events, the dynamics are deterministic: $\frac{d}{dt}\Xi(t)=\Theta(t),$ while $\Theta(t)$ remains constant; at an event of type $i$, only the $i$-th component of $\pi$0 flips (Bierkens et al., 2016). The process is therefore a continuous-time Markov process with deterministic linear flow and stochastic sign flips in individual velocity coordinates.

The target distribution is written

$\pi$1

and the structural condition guaranteeing stationarity is

$\pi$2

Equivalently, the admissible switching rates are

$\pi$3

with $\pi$4. The canonical Zig-Zag process corresponds to $\pi$5 (Bierkens et al., 2016).

The generator takes the PDMP form

$\pi$6

combining deterministic transport with coordinate-wise jump terms (Bierkens et al., 2016).

In one dimension, the state space reduces to

$\pi$7

with state $\pi$8. The target density is

$\pi$9

and the switching intensity is

$E=\mathbb{R}^d\times\{-1,+1\}^d,$0

Between switching times,

$E=\mathbb{R}^d\times\{-1,+1\}^d,$1

and at a switch the direction flips, $E=\mathbb{R}^d\times\{-1,+1\}^d,$2. In this form, ZZP is a piecewise linear motion with random velocity flips; the term $E=\mathbb{R}^d\times\{-1,+1\}^d,$3 produces systematic bounces against the gradient, while $E=\mathbb{R}^d\times\{-1,+1\}^d,$4 adds refreshments (Vasdekis et al., 2021).

2. Invariant distribution and event-time simulation

Under the rate identity above, the extended invariant law is

$E=\mathbb{R}^d\times\{-1,+1\}^d,$5

with $E=\mathbb{R}^d\times\{-1,+1\}^d,$6 uniform on $E=\mathbb{R}^d\times\{-1,+1\}^d,$7; in one dimension,

$E=\mathbb{R}^d\times\{-1,+1\}^d,$8

Thus the position marginal is exactly the target $E=\mathbb{R}^d\times\{-1,+1\}^d,$9, which is the basic reason ZZP is useful for Monte Carlo simulation (Bierkens et al., 2016, Vasdekis et al., 2021).

A major practical feature of the classical construction is that ZZP can be simulated without discretisation error when one can sample its event times exactly. If

$(\xi,\theta)$0

and one has tractable dominating rates $(\xi,\theta)$1, then Poisson thinning can be used: candidate event times are generated from the easier inhomogeneous Poisson processes with rates $(\xi,\theta)$2, and accepted with probability $(\xi,\theta)$3. Two classes of bounds highlighted in the literature are globally bounded gradients, for which $(\xi,\theta)$4, and dominated-Hessian settings, for which affine bounds $(\xi,\theta)$5 are available (Bierkens et al., 2016).

For more general models, NuZZ replaces analytic inversion or thinning envelopes by numerical solution of the Sellke event equation

$(\xi,\theta)$6

where $(\xi,\theta)$7. The implementation uses adaptive Gauss–Kronrod quadrature and Brent’s method, and then chooses the flipped coordinate with probability

$(\xi,\theta)$8

In this formulation, one event-time integral replaces $(\xi,\theta)$9 separate coordinate-wise integrals (Pagani et al., 2020).

NuZZ is not exact at fixed numerical tolerance, but the perturbation is explicit. Under the assumptions stated in the numerical analysis, if

$\xi\in\mathbb R^d$0

then the Kantorovich–Rubinstein distance between the exact invariant law $\xi\in\mathbb R^d$1 and the numerical occupation measure $\xi\in\mathbb R^d$2 satisfies

$\xi\in\mathbb R^d$3

where $\xi\in\mathbb R^d$4 is a positive refreshment lower bound and $\xi\in\mathbb R^d$5 bounds Hessian entries of $\xi\in\mathbb R^d$6 (Pagani et al., 2020).

3. Ergodicity, heavy tails, and speed modulation

A central theme in ZZP theory is the dependence of convergence on tail behavior. In one dimension, heavy-tailed targets invalidate geometric or exponential ergodicity for the original constant-speed process. One sharp obstruction is geometric: if ZZP starts at the origin, then by time $\xi\in\mathbb R^d$7 it cannot reach distance larger than $\xi\in\mathbb R^d$8, so for heavy-tailed targets the total variation distance cannot decay exponentially. The corresponding theorem states that if the original Zig-Zag targets a heavy-tailed distribution, then it is not exponentially ergodic (Vasdekis et al., 2021).

The one-dimensional heavy-tail theory is particularly explicit. Suppose there exist $\xi\in\mathbb R^d$9 and a compact $\theta\in\{-1,+1\}^d$0 such that, outside $\theta\in\{-1,+1\}^d$1,

$\theta\in\{-1,+1\}^d$2

and that the refreshment rate is asymptotically negligible relative to the deterministic bounce term,

$\theta\in\{-1,+1\}^d$3

Then for every $\theta\in\{-1,+1\}^d$4, there exist $\theta\in\{-1,+1\}^d$5 and $\theta\in\{-1,+1\}^d$6 such that with

$\theta\in\{-1,+1\}^d$7

$\theta\in\{-1,+1\}^d$8

Hence the process is polynomially ergodic of order $\theta\in\{-1,+1\}^d$9 (Vasdekis et al., 2021). The proof uses a subgeometric Foster–Lyapunov drift condition

$\frac{d}{dt}\Xi(t)=\Theta(t),$0

small-set theory, and a continuous-time subgeometric ergodicity theorem (Vasdekis et al., 2021).

For Student targets,

$\frac{d}{dt}\Xi(t)=\Theta(t),$1

the result is nearly sharp. For every $\frac{d}{dt}\Xi(t)=\Theta(t),$2, ZZP is polynomially ergodic of order $\frac{d}{dt}\Xi(t)=\Theta(t),$3, while for every $\frac{d}{dt}\Xi(t)=\Theta(t),$4 it is not polynomially ergodic of order $\frac{d}{dt}\Xi(t)=\Theta(t),$5. More precisely, for sufficiently large $\frac{d}{dt}\Xi(t)=\Theta(t),$6,

$\frac{d}{dt}\Xi(t)=\Theta(t),$7

Whether order exactly $\frac{d}{dt}\Xi(t)=\Theta(t),$8 holds remains open (Vasdekis et al., 2021).

To overcome the bounded-speed obstruction on heavy tails, the Speed Up Zig-Zag (SUZZ) process replaces unit speed by a state-dependent positive function $\frac{d}{dt}\Xi(t)=\Theta(t),$9, so that between events

$\Theta(t)$0

To preserve the same target $\Theta(t)$1, the rates become

$\Theta(t)$2

The effective potential is therefore $\Theta(t)$3 (Vasdekis et al., 2021).

This modification changes the tail geometry substantially. Under the assumptions stated in the stability theory, SUZZ is non-explosive, has invariant law

$\Theta(t)$4

and can be exponentially ergodic on heavy-tailed targets (Vasdekis et al., 2021). In one dimension, if the deterministic flow has finite explosion time in both directions, the process is even uniformly ergodic: $\Theta(t)$5 A notable consequence is that explosive deterministic dynamics need not imply an unstable PDMP; the switching mechanism can interrupt the trajectory before explosion and yield a stable, faster-mixing algorithm (Vasdekis et al., 2021).

4. Limit theorems and large deviations

For one-dimensional ZZP, ergodic averages admit a detailed fluctuation theory. In the general CLT framework, for centered observables $\Theta(t)$6 one defines

$\Theta(t)$7

and the asymptotic variance is

$\Theta(t)$8

If $\Theta(t)$9 depends only on position, then

$i$0

This makes explicit that excess switching $i$1 increases asymptotic variance (Bierkens et al., 2016).

In the canonical unimodal one-dimensional setting, the process admits an explicit regenerative decomposition, yielding a closed-form asymptotic variance formula: $i$2 The same analysis also gives functional central limit theorems under Foster–Lyapunov conditions in exponentially ergodic and suitable heavy-tailed regimes (Bierkens et al., 2016).

A second asymptotic regime appears when the excess switching becomes very large. If

$i$3

then, after time acceleration $i$4,

$i$5

where $i$6 solves

$i$7

Thus large excess switching drives ZZP toward a diffusion limit (Bierkens et al., 2016).

Large deviations theory provides a complementary description of long-time behavior. For the empirical measure

$i$8

the one-dimensional zig-zag process satisfies an LDP with speed $i$9 and Donsker–Varadhan-type rate function

$i$0

on the torus, and an analogous formula on $i$1 under additional Lyapunov and regularity conditions (Bierkens et al., 2019).

In the compact setting, the Donsker–Varadhan functional can be written explicitly. This explicit form shows that the rate function is strictly decreasing in a constant refreshment parameter $i$2; from an empirical-measure large deviations perspective, smaller $i$3 is therefore better, and the optimal choice is $i$4 (Bierkens et al., 2019). This does not contradict spectral-gap-based arguments favoring positive refreshment, because convergence to equilibrium and empirical-measure concentration are different criteria (Bierkens et al., 2019).

5. Bayesian inference, sub-sampling, and large-sample scaling

The Zig-Zag process is especially prominent in Bayesian big-data settings because posterior gradients decompose over observations. If

$i$5

then one can construct an exact sub-sampling scheme: along a deterministic segment, propose candidate events from dominating bounds, choose a datapoint index $i$6, and accept with probability based on $i$7. The resulting effective rates are

$i$8

and still satisfy the invariance identity

$i$9

This is the exact approximate scheme: the algorithm uses approximation internally but retains the exact posterior as invariant law (Bierkens et al., 2016).

A key refinement is the control-variate estimator

$\pi$00

where $\pi$01 is a reference point near the posterior mode. With an $\pi$02 preprocessing step to compute $\pi$03, the cost per essentially independent posterior sample can be $\pi$04, provided the posterior contracts at $\pi$05 scale and the reference point is sufficiently accurate (Bierkens et al., 2016).

Large-sample analysis clarifies why naive sub-sampling and control variates behave so differently. In the transient phase, the trajectories are well approximated by a deterministic ODE,

$\pi$06

whose drift points in the direction of decreasing KL divergence between the assumed model and the true distribution (Agrawal et al., 2024). For canonical ZZ, the denominator of the drift reduces to $\pi$07, so the process moves at the optimal coordinate speed $\pi$08. For naive sub-sampling, the denominator involves

$\pi$09

which is larger and damps the motion, especially near the KL minimizer. Control variates reduce this damping when the reference point is close to the posterior center (Agrawal et al., 2024).

In the stationary phase, the distinction is sharper. With local parameter

$\pi$10

naive sub-sampling ZZ-SS converges to a stationary Ornstein–Uhlenbeck diffusion,

$\pi$11

where

$\pi$12

(Agrawal et al., 2024). By contrast, ZZ-CV with reference points satisfying

$\pi$13

retains a Zig-Zag PDMP limit after the $\pi$14-time rescaling, and the same limiting PDMP also describes canonical ZZ in the large-batch regime (Agrawal et al., 2024).

The resulting complexity comparison is: $\pi$15 More precisely, canonical ZZ mixes in $\pi$16 physical time but pays $\pi$17 per proposed switch; ZZ-SS has $\pi$18 per-switch cost but $\pi$19 switching burden; ZZ-CV combines $\pi$20 per-switch cost with $\pi$21 mixing time under suitable control-variate assumptions (Agrawal et al., 2024). This suggests that exact sub-sampling alone is not sufficient for large-$\pi$22 scalability; variance reduction is the essential ingredient.

6. Extensions, hybrid state spaces, and disambiguation

ZZP has also been extended beyond purely continuous Euclidean targets. In the hybrid construction for targets $\pi$23, the discrete state $\pi$24 is countable and the continuous state lies in a domain $\pi$25 depending on $\pi$26. The process evolves as $\pi$27, where $\pi$28, follows ordinary Zig-Zag dynamics in the interior,

$\pi$29

and uses a boundary kernel $\pi$30 to jump between domains when $\pi$31 hits an interface. Under skew-detailed balance and flux compatibility conditions,

$\pi$32

is stationary (Koskela, 2020).

This hybrid formulation does not require any structural assumptions on the discrete component. It was developed for coalescent-based phylogenetic models in which the latent state combines a tree topology with branch lengths and mutation parameters. In that setting, the continuous-time zig-zag process avoids the boundary-crossing complications that arise in Hamiltonian Monte Carlo on hybrid tree spaces, and the paper reports efficiency gains of up to several orders of magnitude over classical Metropolis–Hastings algorithms in some examples (Koskela, 2020).

A common source of confusion is nomenclature. “Zigzag” also appears in topological data analysis and combinatorial expansion, but these are different objects. “Using Zigzag Persistent Homology to Detect Hopf Bifurcations in Dynamical Systems” studies zigzag persistent homology and states explicitly that it is “not anything related to the stochastic Zig-Zag MCMC process” (Tymochko et al., 2020). Likewise, work on the zig-zag product in hypergraph expansion concerns graph-theoretic and spectral constructions rather than PDMP Monte Carlo (Karni et al., 2020). This suggests that “Zig-Zag Process” should be reserved for the stochastic position–velocity sampler, while “zigzag persistence” and “zig-zag product” belong to distinct mathematical literatures.

Overall, ZZP occupies a distinctive position within nonreversible Monte Carlo. Its defining mechanism is persistent ballistic motion punctuated by state-dependent sign flips; its invariant law is controlled by a simple antisymmetry relation on switching intensities; and its theory now spans exact simulation, sub-sampling, CLTs, empirical-measure large deviations, heavy-tail polynomial ergodicity, state-dependent speedups, numerical perturbation bounds, and hybrid discrete–continuous extensions (Bierkens et al., 2016, Bierkens et al., 2016, Bierkens et al., 2019, Vasdekis et al., 2021, Vasdekis et al., 2021, Pagani et al., 2020, Agrawal et al., 2024, Koskela, 2020).