Coordinate Samplers Overview

• Coordinate samplers are a family of methods that update one coordinate at a time, applicable in streaming models, non-reversible PDMPs, and block-wise MCMC.
• They enable near-optimal one-pass streaming with polylogarithmic space and precise tail control via Count-Sketch and random scaling techniques.
• Coupled coordinate samplers and coordinate-scan methods provide unbiased estimators and explicit convergence rate comparisons across different MCMC settings.

Coordinate samplers are a family of sampling procedures organized around coordinate-level selection or coordinate-wise motion rather than full-state updates. In the turnstile streaming model, an $L_p$-sampler outputs the $i$-th coordinate with probability $|x_i|^p/\|x\|_p^p$, and for $p=0$ the $L_0$ distribution is the uniform distribution over the nonzero coordinates of $x$ (Jowhari et al., 2010). In continuous-time Monte Carlo, the Coordinate Sampler is a non-reversible piecewise deterministic Markov process (PDMP) on $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$ that moves along a single active coordinate between random event times (Wu et al., 2018). In component-wise MCMC, coordinate-wise samplers also appear as deterministic-scan and random-scan Gibbs or conditional Metropolis–Hastings kernels on multivariate targets (Qin et al., 2020). A related but distinct literature studies coordinated samples, where similar instances have similar samples through shared randomness across instances (Cohen et al., 2012).

1. Coordinate sampling in turnstile streams

In the streaming setting, the stream is a sequence of updates $(i,u)$, where $i\in[n]$ and $u\in\mathbb R$. These implicitly define $i$0: initially $i$1, and each update adds $i$2 to $i$3. The general turnstile model allows negative updates; the strict turnstile model guarantees $i$4 at the end. For $i$5 and $i$6, the $i$7 distribution over $i$8 is

$i$9

A perfect $|x_i|^p/\|x\|_p^p$0-sampler outputs index $|x_i|^p/\|x\|_p^p$1 according to the exact $|x_i|^p/\|x\|_p^p$2 distribution, and fails only if $|x_i|^p/\|x\|_p^p$3. An approximate $|x_i|^p/\|x\|_p^p$4-sampler with $|x_i|^p/\|x\|_p^p$5 relative error and constant failure probability outputs $|x_i|^p/\|x\|_p^p$6 with

$|x_i|^p/\|x\|_p^p$7

for any chosen constant $|x_i|^p/\|x\|_p^p$8, where error events of probability $|x_i|^p/\|x\|_p^p$9 are treated as negligible (Jowhari et al., 2010).

The principal contribution of the one-pass streaming literature is to show that coordinate sampling can be carried out in polylogarithmic space even under updates that include additions and subtraction. For $p=0$0, one-pass sampling with $p=0$1 relative error is achieved in $p=0$2 space; for $p=0$3 the space bound is $p=0$4; and for $p=0$5 the bound is $p=0$6. For $p=0$7-sampling, zero relative error is achieved in $p=0$8 bits. These bounds improve earlier $p=0$9-type dependencies and are paired with $L_0$0 lower bounds showing tightness for constant $L_0$1 (Jowhari et al., 2010).

The same framework yields applications beyond the sampler itself. In streams of length longer than the alphabet size, $L_0$2 sampling gives an $L_0$3 space algorithm for finding duplicates, improving a previous $L_0$4 bound. For heavy hitters in turnstile streams, the same sketching apparatus gives an upper bound of $L_0$5 space, and the lower bound is $L_0$6 in the strict turnstile model (Jowhari et al., 2010).

2. Precision sampling, sketching, and optimality

The algorithmic core of one-pass coordinate sampling combines random scaling, Count-Sketch, and norm estimation. For parameter $L_0$7 and $L_0$8 repetitions, Count-Sketch maintains

$L_0$9

and estimates coordinates by

$x$0

The sampling reduction then draws $x$1-wise independent $x$2 and sets

$x$3

Heavy coordinates in $x$4 correspond to coordinates whose $x$5 are small, and the event $x$6 occurs with probability approximately $x$7. Setting $x$8 calibrates this event to the target sampling distribution (Jowhari et al., 2010).

The refinement over earlier precision sampling lies in tail control. The analysis uses that the Count-Sketch error depends on the $x$9 norm of the tail of $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$0; under the $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$1 scaling, the tail’s $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$2 norm can be bounded by $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$3, saving a $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$4 factor. The sampler computes $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$5, a $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$6-approximation of $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$7, and $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$8, a $\mathbb R^d \times \{\pm e_i:i=1,\dots,d\}$9-approximation of $(i,u)$0, where $(i,u)$1 is the best $(i,u)$2-sparse approximation to $(i,u)$3. It chooses the index $(i,u)$4 with maximal $(i,u)$5, aborts if $(i,u)$6 or if $(i,u)$7, and otherwise outputs $(i,u)$8 together with $(i,u)$9. For $i\in[n]$0 the parameter choice is $i\in[n]$1; for $i\in[n]$2, $i\in[n]$3; and for $i\in[n]$4, $i\in[n]$5 (Jowhari et al., 2010).

The resulting theorem states that for $i\in[n]$6 and $i\in[n]$7, there is a one-pass $i\in[n]$8-sampling algorithm with $i\in[n]$9-relative error and failure probability at most $u\in\mathbb R$0. For $u\in\mathbb R$1, the space is

$u\in\mathbb R$2

while for $u\in\mathbb R$3 it is

$u\in\mathbb R$4

Update time per stream item is $u\in\mathbb R$5, and recovery is polylogarithmic in $u\in\mathbb R$6. For $u\in\mathbb R$7-sampling, the one-pass zero relative error construction uses exact sparse recovery over a hierarchy of random index subsets, and Nisan’s PRG reduces randomness to $u\in\mathbb R$8 seed bits while preserving the zero relative error property and failure at most $u\in\mathbb R$9 (Jowhari et al., 2010).

Lower bounds establish that these space bounds are not artifacts of the proof technique. Any one-pass $i$00-sampler whose output distribution has variation distance at most $i$01 from the true $i$02 distribution must use $i$03 bits, even when $i$04. The same communication-complexity methodology, via reductions from augmented indexing and the Universal Relation problem, yields tight lower bounds for duplicates and heavy hitters (Jowhari et al., 2010).

3. The PDMP Coordinate Sampler in continuous-time MCMC

The Coordinate Sampler of Wu and Robert is a non-reversible, continuous-time Markov chain Monte Carlo sampler based on a PDMP (Wu et al., 2018). It targets

$i$05

and augments the position with a velocity $i$06 in the set

$i$07

Between events, only one coordinate moves at a time: $i$08 Hence, if $i$09, then $i$10 for $i$11 and $i$12.

The event rate is

$i$13

with total rate

$i$14

At an event time, the position remains fixed and the new velocity is sampled from

$i$15

The extended generator is

$i$16

The invariant distribution is $i$17, where $i$18 is the uniform distribution over $i$19 (Wu et al., 2018).

The construction is “Gibbs-like” because, at fixed $i$20, the new velocity is chosen according to weights derived from the local gradient. For $i$21, the probability mass concentrates on directions that decrease $i$22. The process violates detailed balance, but global balance holds. With $i$23, the refreshment mechanism makes the process irreducible on $i$24 and allows reaching any $i$25 in finite time from any initial $i$26, implying uniqueness of the invariant distribution (Wu et al., 2018).

Geometric ergodicity is established through the Lyapunov function

$i$27

under assumptions A.1–A.4 and one of two tail conditions. Under C.1, as $i$28, $i$29, $i$30, and $i$31. Under C.2, as $i$32, $i$33, $i$34, and $i$35. The theorem states that CS is $i$36-uniformly ergodic: $i$37 for some $i$38 and $i$39 (Wu et al., 2018).

The sampler is presented as a variant of the Zigzag sampler. In CS, the event rate is tied to one active coordinate and is independent of $i$40 in the same sense that Zigzag’s total event rate is a sum over coordinates. The paper’s informal analysis concludes that CS is $i$41 more efficient per coordinate trajectory length. Numerical examples on banana-shaped targets, correlated Gaussians, Bayesian logistic regression, and a log-Gaussian Cox point process show higher effective sample size per second than Zigzag in the reported settings; for a $i$42-dimensional MVN2 example, the reported KS distance means are $i$43 for CS, $i$44 for ZS, $i$45 for BPS, and $i$46 for HMC (Wu et al., 2018).

4. Coupled coordinate samplers and debiasing

A later development studies couplings for PDMP samplers in order to debias finite-time estimators (Corenflos et al., 2023). For the Coordinate Sampler, the paper introduces an on-and-off generalization, I/O-CS, with velocity space

$i$47

If $i$48, the position is frozen. The rate remains

$i$49

and post-event velocities are sampled from weights proportional to $i$50. The paper shows $i$51 is invariant for I/O-CS and argues ergodicity under mild conditions, including $i$52 continuously differentiable and compact sets petite (Corenflos et al., 2023).

The objective is a $i$53-coupling: there exists a random time $i$54 such that

$i$55

for all $i$56. The construction interleaves two synchronization mechanisms. When both velocities are zero, event times are homogeneous Poisson with rate $i$57, so shifted exponential maximal coupling can maximize the probability that the clocks align after a delay $i$58. When velocities are co-linear, coupled thinning is used for the non-homogeneous Poisson hazards along coordinate flows, and a maximal coupling of the categorical post-event velocity distributions promotes agreement of the axis index and then the sign. The paper recommends coupling the absolute axis first and then coupling signs so as to minimize the “bad” configuration that prevents imminent meeting (Corenflos et al., 2023).

These couplings underpin continuous-time unbiased estimators. For a test function $i$59, the averaged continuous estimator is

$i$60

Under marginal equality, $i$61-coupling, exponential tails for $i$62, and integrability of $i$63, the estimator has mean $i$64, finite variance, and finite expected computational time (Corenflos et al., 2023).

For Gaussian targets, the experiments show average $i$65-coupling times scaling linearly with dimension $i$66 for CS. The same study notes that CS requires only one partial derivative per event rather than the full gradient, so computational cost per event can be smaller, but overall mixing per unit time is $i$67 worse than BPS or Boomerang since CS moves one coordinate at a time. This suggests a division of labor: CS is attractive when partial derivatives are cheap, while coupling-based debiasing makes the resulting estimator embarrassingly parallel (Corenflos et al., 2023).

5. Coordinate-wise scan samplers in two-component MCMC

In two-component MCMC, coordinate-wise samplers arise as deterministic-scan and random-scan updates for a joint target $i$68 with full conditionals $i$69 and $i$70 (Qin et al., 2020). The deterministic-scan Gibbs kernel is

$i$71

and the random-scan Gibbs kernel with selection probability $i$72 is

$i$73

$i$74 is reversible with respect to $i$75, while $i$76 is generally not reversible (Qin et al., 2020).

The paper establishes an exact quantitative relationship between the $i$77 convergence rates of the two samplers. If $i$78 and $i$79, then

$i$80

Equivalently, if $i$81 is the maximal correlation between the two components, then $i$82 and

$i$83

The consequences are explicit: if $i$84 then $i$85, and $i$86 iff $i$87. The optimal random-scan choice is $i$88, giving

$i$89

Thus the deterministic-scan sampler converges faster in the two-component setting (Qin et al., 2020).

The same work studies conditional Metropolis–Hastings variants. Under proposal regularity Condition C,

$i$90

geometric ergodicity relations propagate from Gibbs samplers to CMH samplers. For example, if $i$91 then $i$92, and if $i$93 then $i$94. The scope is specifically two components; the paper states that the exact deterministic–random scan rate relation does not extend directly to $i$95 because deterministic-scan with more than two blocks lacks reversible marginal chains (Qin et al., 2020).

6. Related terminology: coordinated samples

A related but distinct line of work studies coordinated samples rather than coordinate samplers in the MCMC or turnstile-stream sense (Cohen et al., 2012). Here the defining idea is that similar instances have similar samples. The same random seed $i$96 is used for each item $i$97 across instances, and the information extracted from an outcome is summarized by the set

$i$98

The lower bound function is

$i$99

The paper gives a characterization of when unbiased, nonnegative, finite-variance, and bounded single-item estimators exist (Cohen et al., 2012).

Its main theorem states that an unbiased nonnegative estimator exists iff

$|x_i|^p/\|x\|_p^p$00

and that an unbiased nonnegative estimator with finite variance exists iff

$|x_i|^p/\|x\|_p^p$01

where $|x_i|^p/\|x\|_p^p$02 is the lower hull of $|x_i|^p/\|x\|_p^p$03. The $|x_i|^p/\|x\|_p^p$04-optimal estimator satisfies

$|x_i|^p/\|x\|_p^p$05

for almost all $|x_i|^p/\|x\|_p^p$06, and the paper constructs a single universal estimator, the $|x_i|^p/\|x\|_p^p$07 estimator, that is $|x_i|^p/\|x\|_p^p$08-competitive. Specifically,

$|x_i|^p/\|x\|_p^p$09

This literature is not a theory of coordinate-wise state evolution, but it clarifies a nearby terminology in which coordination refers to shared randomness across instances and to estimators for sums such as distinct counts, max/min sums, and $|x_i|^p/\|x\|_p^p$10-difference sums (Cohen et al., 2012).

Taken together, these literatures show that “coordinate sampler” is not a single method but a family of constructions indexed by problem class. In streaming, the object is a coordinate-selection primitive with near-optimal one-pass space and matching lower bounds (Jowhari et al., 2010). In continuous-time MCMC, it is a non-reversible PDMP with single-coordinate motion, explicit invariance, and geometric ergodicity under stated tail conditions (Wu et al., 2018). In two-component MCMC, it denotes scan rules whose convergence rates can be compared exactly (Qin et al., 2020). A plausible implication is that the common theme is not a single algorithmic template, but the repeated use of coordinate-level structure to trade global updates for provable efficiency or tractability in high-dimensional inference and data reduction.