---
title: Coordinate Samplers Overview
url: https://www.emergentmind.com/topics/coordinate-samplers
type: topic
---

# Coordinate Samplers Overview

Coordinate samplers are a family of sampling procedures organized around coordinate-level selection or coordinate-wise motion rather than full-state updates. In the turnstile streaming model, an $L_p$-sampler outputs the $i$-th coordinate with probability $|x_i|^p/\|x\|_p^p$, and for $p=0$ the $L_0$ distribution is the uniform distribution over the nonzero coordinates of $x$ [1012.4889]. In continuous-time Monte Carlo, the Coordinate Sampler is a non-reversible piecewise deterministic Markov process (PDMP) on $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$ that moves along a single active coordinate between random event times [1809.03388]. In component-wise MCMC, coordinate-wise samplers also appear as deterministic-scan and random-scan Gibbs or conditional Metropolis–Hastings kernels on multivariate targets [2006.14801]. A related but distinct literature studies coordinated samples, where similar instances have similar samples through shared randomness across instances [1206.5637].

## 1. Coordinate sampling in turnstile streams

In the streaming setting, the stream is a sequence of updates $(i,u)$, where $i\in[n]$ and $u\in\mathbb R$. These implicitly define $x\in\mathbb R^n$: initially $x=0$, and each update adds $u$ to $x_i$. The general turnstile model allows negative updates; the strict turnstile model guarantees $x_i\ge 0$ at the end. For $x\neq 0$ and $p>0$, the $L_p$ distribution over $[n]$ is
\[
\Pr[i]=\frac{|x_i|^p}{\|x\|_p^p},\qquad \|x\|_p^p=\sum_{i=1}^n |x_i|^p.
\]
A perfect $L_p$-sampler outputs index $i$ according to the exact $L_p$ distribution, and fails only if $x=0$. An approximate $L_p$-sampler with $\epsilon$ relative error and constant failure probability outputs $i$ with
\[
\Pr[i]=(1\pm \epsilon)\,\frac{|x_i|^p}{\|x\|_p^p}+O(n^{-c}),
\]
for any chosen constant $c\ge 2$, where error events of probability $n^{-c}$ are treated as negligible [1012.4889].

The principal contribution of the one-pass streaming literature is to show that coordinate sampling can be carried out in polylogarithmic space even under updates that include additions and subtraction. For $p\in(1,2)$, one-pass sampling with $\epsilon$ relative error is achieved in $O(\epsilon^{-p}\log^2 n)$ space; for $p\in(0,1)$ the space bound is $O(\epsilon^{-1}\log^2 n)$; and for $p=1$ the bound is $O(\log(1/\epsilon)\cdot \epsilon^{-1}\cdot \log^2 n)$. For $L_0$-sampling, zero relative error is achieved in $O(\log^2 n)$ bits. These bounds improve earlier $O(\log^3 n)$-type dependencies and are paired with $\Omega(\log^2 n)$ lower bounds showing tightness for constant $\epsilon>0$ [1012.4889].

The same framework yields applications beyond the sampler itself. In streams of length longer than the alphabet size, $L_1$ sampling gives an $O(\log^2 n)$ space algorithm for finding duplicates, improving a previous $O(\log^3 n)$ bound. For heavy hitters in turnstile streams, the same sketching apparatus gives an upper bound of $O(\phi^{-p}\log^2 n)$ space, and the lower bound is $\Omega(\phi^{-p}\log^2 n)$ in the strict turnstile model [1012.4889].

## 2. Precision sampling, sketching, and optimality

The algorithmic core of one-pass coordinate sampling combines random scaling, Count-Sketch, and norm estimation. For parameter $m$ and $l=O(\log n)$ repetitions, Count-Sketch maintains
\[
y_{k,j}=\sum_{i:h_j(i)=k} g_j(i)\,x_i,
\]
and estimates coordinates by
\[
x_i^*=\operatorname{median}_{j\in[l]}\big(g_j(i)\,y_{h_j(i),j}\big).
\]
The sampling reduction then draws $k$-wise independent $t_i\sim \mathrm{Uniform}[0,1]$ and sets
\[
z_i=\frac{x_i}{t_i^{1/p}}.
\]
Heavy coordinates in $z$ correspond to coordinates whose $t_i$ are small, and the event $|z_i|>\tau$ occurs with probability approximately $|x_i|^p/\tau^p$. Setting $\tau\approx \epsilon^{-1/p}\|x\|_p$ calibrates this event to the target sampling distribution [1012.4889].

The refinement over earlier precision sampling lies in tail control. The analysis uses that the Count-Sketch error depends on the $L_2$ norm of the tail of $z$; under the $t_i$ scaling, the tail’s $L_2$ norm can be bounded by $O(\|x\|_p)$, saving a $\log n$ factor. The sampler computes $r$, a $2$-approximation of $\|x\|_p$, and $s$, a $2$-approximation of $\|z-\hat z\|_2$, where $\hat z$ is the best $m$-sparse approximation to $z^*$. It chooses the index $i$ with maximal $|z_i^*|$, aborts if $s>\beta m^{1/2}r$ or if $|z_i^*|<\epsilon^{-1/p}r$, and otherwise outputs $i$ together with $x_i^*=z_i^* t_i^{1/p}$. For $p\in(1,2)$ the parameter choice is $m=\Theta(\epsilon^{-(p-1)})$; for $p\in(0,1)$, $m=\Theta(1)$; and for $p=1$, $m=\Theta(\log(1/\epsilon))$ [1012.4889].

The resulting theorem states that for $\epsilon,\delta\in(0,1)$ and $p\in(0,2)$, there is a one-pass $L_p$-sampling algorithm with $\epsilon$-relative error and failure probability at most $\delta$. For $p\neq 1$, the space is
\[
O_p\big(\epsilon^{-\max(1,p)}\log^2 n\,\log(1/\delta)\big),
\]
while for $p=1$ it is
\[
O\big(\epsilon^{-1}\log(1/\epsilon)\log^2 n\,\log(1/\delta)\big).
\]
Update time per stream item is $O(\log n)$, and recovery is polylogarithmic in $n$. For $L_0$-sampling, the one-pass zero relative error construction uses exact sparse recovery over a hierarchy of random index subsets, and Nisan’s PRG reduces randomness to $O(\log^2 n)$ seed bits while preserving the zero relative error property and failure at most $\delta+O(n^{-c})$ [1012.4889].

Lower bounds establish that these space bounds are not artifacts of the proof technique. Any one-pass $L_p$-sampler whose output distribution has variation distance at most $1/3$ from the true $L_p$ distribution must use $\Omega(\log^2 n)$ bits, even when $x\in\{-1,0,1\}^n$. The same communication-complexity methodology, via reductions from augmented indexing and the Universal Relation problem, yields tight lower bounds for duplicates and heavy hitters [1012.4889].

## 3. The PDMP Coordinate Sampler in continuous-time MCMC

The Coordinate Sampler of Wu and Robert is a non-reversible, continuous-time Markov chain Monte Carlo sampler based on a PDMP [1809.03388]. It targets
\[
\pi(x)\propto \exp\{-U(x)\}, \qquad x\in\mathbb R^d,
\]
and augments the position with a velocity $v$ in the set
\[
\mathcal V=\{\pm e_i:i=1,\dots,d\}.
\]
Between events, only one coordinate moves at a time:
\[
\frac{dX_t}{dt}=V_t,\qquad \frac{dV_t}{dt}=0.
\]
Hence, if $v=\pm e_i$, then $X_j(t)=X_j(s)$ for $j\neq i$ and $X_i(t)=X_i(s)\pm (t-s)$.

The event rate is
\[
\lambda(x,v)=\langle v,\nabla U(x)\rangle_+ + \lambda^{ref},
\]
with total rate
\[
\lambda(x)=\sum_{v\in\mathcal V}\lambda(x,v)=2d\lambda^{ref}+\sum_{i=1}^d \left|\frac{\partial U(x)}{\partial x_i}\right|.
\]
At an event time, the position remains fixed and the new velocity is sampled from
\[
Q((dx',dv')\mid(x,v))
=
\delta_x(dx')\sum_{v^*\in\mathcal V}\frac{\lambda(x,-v^*)}{\lambda(x)}\,\delta_{v^*}(dv').
\]
The extended generator is
\[
(\mathcal L f)(x,v)
=
v^\top \nabla_x f(x,v)
+
\lambda(x,v)
\left[
\sum_{v'\in\mathcal V}\frac{\lambda(x,-v')}{\lambda(x)}(f(x,v')-f(x,v))
\right].
\]
The invariant distribution is $\rho(dx,dv)=\pi(dx)\phi(dv)$, where $\phi$ is the uniform distribution over $\mathcal V$ [1809.03388].

The construction is “Gibbs-like” because, at fixed $x$, the new velocity is chosen according to weights derived from the local gradient. For $\lambda^{ref}=0$, the probability mass concentrates on directions that decrease $U$. The process violates detailed balance, but global balance holds. With $\lambda^{ref}>0$, the refreshment mechanism makes the process irreducible on $\mathbb R^d\times\mathcal V$ and allows reaching any $(x^*,v^*)$ in finite time from any initial $(x_0,v_0)$, implying uniqueness of the invariant distribution [1809.03388].

Geometric ergodicity is established through the Lyapunov function
\[
V(x,v)=\frac{\exp(U(x)/2)}{\sqrt{\lambda^{ref}+\langle \nabla U(x),-v\rangle_+}},
\]
under assumptions A.1–A.4 and one of two tail conditions. Under C.1, as $|x|\to\infty$, $\liminf |\nabla U(x)|=\infty$, $\limsup \|\Delta U(x)\|\le \alpha_1<\infty$, and $\lambda^{ref}>\sqrt{8\alpha_1}$. Under C.2, as $|x|\to\infty$, $\liminf |\nabla U(x)|=2\alpha_2>0$, $\limsup \|\Delta U(x)\|=0$, and $\lambda^{ref}<\alpha_2/(14d)$. The theorem states that CS is $V$-uniformly ergodic:
\[
\|P^t(z,\cdot)-\rho\|_V \le V(z)\Gamma \gamma^t
\]
for some $\Gamma<\infty$ and $0<\gamma<1$ [1809.03388].

The sampler is presented as a variant of the Zigzag sampler. In CS, the event rate is tied to one active coordinate and is independent of $d$ in the same sense that Zigzag’s total event rate is a sum over coordinates. The paper’s informal analysis concludes that CS is $O(d)$ more efficient per coordinate trajectory length. Numerical examples on banana-shaped targets, correlated Gaussians, Bayesian logistic regression, and a log-Gaussian Cox point process show higher effective sample size per second than Zigzag in the reported settings; for a $20$-dimensional MVN2 example, the reported KS distance means are $7.00\times 10^{-3}$ for CS, $16.53\times 10^{-3}$ for ZS, $8.98\times 10^{-3}$ for BPS, and $2.12\times 10^{-3}$ for HMC [1809.03388].

## 4. Coupled coordinate samplers and debiasing

A later development studies couplings for PDMP samplers in order to debias finite-time estimators [2306.15422]. For the Coordinate Sampler, the paper introduces an on-and-off generalization, I/O-CS, with velocity space
\[
\mathcal V_{IO-CS}=\mathcal V_{CS}\cup\{0\}.
\]
If $v=0$, the position is frozen. The rate remains
\[
\lambda(x,v)=\langle v,\nabla U(x)\rangle_+ + \lambda_r,
\]
and post-event velocities are sampled from weights proportional to $\lambda(x,-v')$. The paper shows $\pi(x,v)=\pi(x)\phi(v)$ is invariant for I/O-CS and argues ergodicity under mild conditions, including $U$ continuously differentiable and compact sets petite [2306.15422].

The objective is a $\Delta$-coupling: there exists a random time $\kappa<\infty$ such that
\[
X_{\kappa+\Delta+t}=Y_{\kappa+t},\qquad V_{\kappa+\Delta+t}=W_{\kappa+t},
\]
for all $t\ge 0$. The construction interleaves two synchronization mechanisms. When both velocities are zero, event times are homogeneous Poisson with rate $\lambda_r$, so shifted exponential maximal coupling can maximize the probability that the clocks align after a delay $\Delta$. When velocities are co-linear, coupled thinning is used for the non-homogeneous Poisson hazards along coordinate flows, and a maximal coupling of the categorical post-event velocity distributions promotes agreement of the axis index and then the sign. The paper recommends coupling the absolute axis first and then coupling signs so as to minimize the “bad” configuration that prevents imminent meeting [2306.15422].

These couplings underpin continuous-time unbiased estimators. For a test function $h$, the averaged continuous estimator is
\[
\mathrm{ACRG}(k,m,\Delta)
=
\frac{1}{[(m-k+1)\Delta]}\int_{k\Delta}^{(m+1)\Delta} h(Z^1_s)\,ds
+
\sum_{\ell=k+1}^{\lfloor(\kappa+\Delta)/\Delta\rfloor}
\min\!\left(1,\frac{\ell-k}{m-k+1}\right)
\frac{1}{\Delta}
\int_{\ell\Delta}^{(\ell+1)\Delta}
\big[h(Z^1_s)-h(Z^2_{s-\Delta})\big]\,ds.
\]
Under marginal equality, $\Delta$-coupling, exponential tails for $\kappa$, and integrability of $h$, the estimator has mean $\pi(h)$, finite variance, and finite expected computational time [2306.15422].

For Gaussian targets, the experiments show average $\Delta$-coupling times scaling linearly with dimension $d$ for CS. The same study notes that CS requires only one partial derivative per event rather than the full gradient, so computational cost per event can be smaller, but overall mixing per unit time is $O(d)$ worse than BPS or Boomerang since CS moves one coordinate at a time. This suggests a division of labor: CS is attractive when partial derivatives are cheap, while coupling-based debiasing makes the resulting estimator embarrassingly parallel [2306.15422].

## 5. Coordinate-wise scan samplers in two-component MCMC

In two-component MCMC, coordinate-wise samplers arise as deterministic-scan and random-scan updates for a joint target $\Pi(dx,dy)$ with full conditionals $\Pi_{X|Y}(dx|y)$ and $\Pi_{Y|X}(dy|x)$ [2006.14801]. The deterministic-scan Gibbs kernel is
\[
P_{ds}((x,y),(dx',dy'))=\Pi_{X|Y}(dx'|y')\Pi_{Y|X}(dy'|x),
\]
and the random-scan Gibbs kernel with selection probability $r\in(0,1)$ is
\[
P_{rs}((x,y),(dx',dy'))
=
r\,\Pi_{X|Y}(dx'|y)\delta_y(dy')
+
(1-r)\,\Pi_{Y|X}(dy'|x)\delta_x(dx').
\]
$P_{rs}$ is reversible with respect to $\Pi$, while $P_{ds}$ is generally not reversible [2006.14801].

The paper establishes an exact quantitative relationship between the $L^2$ convergence rates of the two samplers. If $\rho_D=\rho(P_{ds})$ and $\rho_R=\rho(P_{rs})$, then
\[
\rho(P_{rs})
=
\frac{1+\sqrt{\,1-4r(1-r)[1-\rho(P_{ds})]\,}}{2}.
\]
Equivalently, if $\bar\gamma$ is the maximal correlation between the two components, then $\rho(P_{ds})=\bar\gamma^2$ and
\[
\rho(P_{rs})
=
\frac{1+\sqrt{\,1-4r(1-r)(1-\bar\gamma^2)\,}}{2}.
\]
The consequences are explicit: if $\rho_D<1$ then $\rho_R>\rho_D$, and $\rho_R=1$ iff $\rho_D=1$. The optimal random-scan choice is $r=1/2$, giving
\[
\rho(P_{rs})=\frac{1+\sqrt{\rho(P_{ds})}}{2}.
\]
Thus the deterministic-scan sampler converges faster in the two-component setting [2006.14801].

The same work studies conditional Metropolis–Hastings variants. Under proposal regularity Condition C,
\[
C=\sup_{(x',x,y)} \frac{\pi_{X|Y}(x'|y)}{q(x'|x,y)}<\infty,
\]
geometric ergodicity relations propagate from Gibbs samplers to CMH samplers. For example, if $\rho(P_{ds})<1$ then $\rho(P_{DC})<1$, and if $\rho(P_{rs})<1$ then $\rho(P_{RC})<1$. The scope is specifically two components; the paper states that the exact deterministic–random scan rate relation does not extend directly to $k>2$ because deterministic-scan with more than two blocks lacks reversible marginal chains [2006.14801].

## 6. Related terminology: coordinated samples

A related but distinct line of work studies coordinated samples rather than coordinate samplers in the MCMC or turnstile-stream sense [1206.5637]. Here the defining idea is that similar instances have similar samples. The same random seed $u(h)$ is used for each item $h$ across instances, and the information extracted from an outcome is summarized by the set
\[
V^*(S)=
\left\{
z\in V \;\middle|\;
\forall i,\;
(i\in S \wedge z_i=v_i)\;\vee\; (i\notin S \wedge z_i<\tau_i(u))
\right\}.
\]
The lower bound function is
\[
\underline f(S)=\inf\{f(z)\mid z\in V^*(S)\}.
\]
The paper gives a characterization of when unbiased, nonnegative, finite-variance, and bounded single-item estimators exist [1206.5637].

Its main theorem states that an unbiased nonnegative estimator exists iff
\[
\lim_{u\to 0^+}\underline f^{(v)}(u)=f(v),
\]
and that an unbiased nonnegative estimator with finite variance exists iff
\[
\int_0^1 \left(\frac{dH_f^{(v)}(u)}{du}\right)^2du<\infty,
\]
where $H_f^{(v)}$ is the lower hull of $\underline f^{(v)}$. The $v$-optimal estimator satisfies
\[
\hat f(u,v)=-\frac{dH_f^{(v)}(u)}{du}
\]
for almost all $u$, and the paper constructs a single universal estimator, the $J$ estimator, that is $O(1)$-competitive. Specifically,
\[
\int_0^1 (\hat f^{(J)}(u,v))^2du
\le
84\cdot
\int_0^1 (\hat f^{(v)}(u))^2du.
\]
This literature is not a theory of coordinate-wise state evolution, but it clarifies a nearby terminology in which coordination refers to shared randomness across instances and to estimators for sums such as distinct counts, max/min sums, and $L_p$-difference sums [1206.5637].

Taken together, these literatures show that “coordinate sampler” is not a single method but a family of constructions indexed by problem class. In streaming, the object is a coordinate-selection primitive with near-optimal one-pass space and matching lower bounds [1012.4889]. In continuous-time MCMC, it is a non-reversible PDMP with single-coordinate motion, explicit invariance, and geometric ergodicity under stated tail conditions [1809.03388]. In two-component MCMC, it denotes scan rules whose convergence rates can be compared exactly [2006.14801]. A plausible implication is that the common theme is not a single algorithmic template, but the repeated use of coordinate-level structure to trade global updates for provable efficiency or tractability in high-dimensional inference and data reduction.

Source: https://www.emergentmind.com/topics/coordinate-samplers