Conclique-based Gibbs Sampling

• Conclique-based Gibbs Sampling is a method that partitions MRF nodes into non-neighboring groups (concliques) to allow efficient, parallel updates.
• By using graph-coloring techniques, the approach reduces the computational cost per iteration, achieving speedups of up to two orders of magnitude over single-site Gibbs sampling.
• Its effectiveness is model-dependent, showing limited gains in complete graphs or highly dependent systems, making it ideal for large-scale spatial and network models with bounded local dependencies.

Conclique-based Gibbs sampling is a methodological innovation in Markov random field (MRF) simulation designed to address the computational bottlenecks of standard single-site Gibbs sampling. By exploiting the independence structure in MRFs, the conclique-based approach simultaneously updates disjoint sets of non-neighboring sites—termed “concliques”—resulting in considerable computational acceleration, particularly for large or high-dimensional spatial and network models with bounded local dependencies (Kaplan et al., 2018).

1. Definition and Identification of Concliques

Let $G = (V, E)$ denote a finite undirected graph, where $V = \{1, \dots, n\}$ indexes sites or locations and $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$ denotes the neighborhood of site $i$. A conclique $C \subset V$ is defined as a subset for which $i, j \in C$, $i \neq j$ implies $j \notin \mathcal{N}(i)$. No two sites in a conclique are neighbors, i.e., $\mathcal{N}(i) \cap C = \varnothing$ for each $i \in C$. This property ensures conditional independence within the conclique given the configuration outside.

To partition $V = \{1, \dots, n\}$0 into concliques, one seeks a conclique cover $V = \{1, \dots, n\}$1 with disjoint concliques $V = \{1, \dots, n\}$2. In graph-theoretic terms, this is equivalent to a proper vertex-coloring: the minimal number $V = \{1, \dots, n\}$3 is the chromatic number $V = \{1, \dots, n\}$4. Conclique covers can be found by:

• Greedy (Welch–Powell) algorithm: orders vertices by degree and assigns the smallest available color.
• DSatur algorithm: Brélaz's saturation-degree heuristic, often finding $V = \{1, \dots, n\}$5 in practice.
• Manual construction: for regular lattices, concliques can be described explicitly (e.g., $V = \{1, \dots, n\}$6 for two- or four-nearest neighbor, $V = \{1, \dots, n\}$7 for eight-nearest).

A brief pseudocode for the greedy algorithm is:

$\mathcal{N}(i) \cap C = \varnothing$4

2. Markov Random Field Model Structure

The MRF is defined on random variables $V = \{1, \dots, n\}$8, where the joint is implicitly specified by local conditionals:

$V = \{1, \dots, n\}$9

Assuming compatibility, there exists a joint density

$\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$0

with normalizing constant $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$1. Many relevant models use a one-parameter exponential family:

$\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$2

This MRF setup underlies both spatial and network models, with structure enforced by $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$3.

3. Conclique-based Gibbs Sampler: Algorithmic Description

With a conclique partition $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$4, the conclique-based Gibbs sampler (CGS) executes sequential “block” updates over concliques; within each, all $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$5, $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$6, are updated independently and in parallel given current values outside $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$7.

Precisely, for $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$8 denoting the iteration and $\mathcal{N}(i) = \{j \neq i : (i, j) \in E\}$9 indexing concliques,

• For each $i$0,

$i$1

where

$i$2

• For $i$3, set $i$4.

After $i$5 conclique updates, let $i$6.

This mechanism ensures that for each conclique $i$7:

$i$8

holding the remaining variables fixed at their current or updated values.

4. Invariance and Ergodicity Properties

Each conclique update is a valid Gibbs step on the subvector $i$9 conditional on the complement, preserving the target joint $C \subset V$0. The entire composite step—sequentially applying all $C \subset V$1 conclique updates—also leaves $C \subset V$2 invariant.

Ergodicity holds under the usual positivity condition: if every conditional $C \subset V$3 has full support, the CGS chain $C \subset V$4 is Harris ergodic with unique invariant $C \subset V$5. When the number of concliques is $C \subset V$6, the chain is a two-component Gibbs sampler, which can be shown to be geometrically ergodic via standard drift/minorization theory.

5. Computational Complexity and Comparative Performance

Let $C \subset V$7 and $C \subset V$8 denote the number of concliques. Single-site Gibbs sampling performs $C \subset V$9 univariate updates per iteration ($i, j \in C$0 per iteration). CGS decomposes each iteration into $i, j \in C$1 block-updates, each processing $i, j \in C$2 sites; the total per-iteration work remains $i, j \in C$3, but the constant is reduced by a factor of roughly $i, j \in C$4.

For models with bounded local neighborhoods (i.e., maximum neighborhood size does not increase with $i, j \in C$5), a classical result gives $i, j \in C$6. For spatial lattices with neighborhood radius $i, j \in C$7 in $i, j \in C$8, $i, j \in C$9, leading to a practical speedup of $i \neq j$0 per full sweep. Empirical studies demonstrate speed gains of one to two orders of magnitude or more, depending on the model structure and $i \neq j$1 (Kaplan et al., 2018).

Model	$i \neq j$2 (No. concliques)	Single-site Gibbs Runtime	CGS Runtime	Reported Speedup
Gaussian MRF ($i \neq j$3 grid, $i \neq j$4)	2	$i \neq j$5 3 hr	$i \neq j$6 15 sec	$i \neq j$7
Binary autologistic ($i \neq j$8 grid)	2	$i \neq j$9 s	$j \notin \mathcal{N}(i)$0 s	$j \notin \mathcal{N}(i)$1
Exponential-graph triad ($j \notin \mathcal{N}(i)$2)	$j \notin \mathcal{N}(i)$3	$j \notin \mathcal{N}(i)$4 9.6 hr (1000 sweeps)	$j \notin \mathcal{N}(i)$5 13 min	$j \notin \mathcal{N}(i)$6

Empirical mixing (e.g., inverse IACT) remains essentially identical for both CGS and single-site Gibbs, confirming that gains are in computational cost per sweep, not in mixing rate per sweep.

6. Situations with Limited Efficacy

The speedup advantage of CGS is model-dependent. In the worst case $j \notin \mathcal{N}(i)$7, as for complete or nearly-complete graphs, so each conclique is a singleton and CGS collapses to single-site Gibbs with no computational gain. In models exhibiting long-range or global dependence (where $j \notin \mathcal{N}(i)$8 grows with $j \notin \mathcal{N}(i)$9), $\mathcal{N}(i) \cap C = \varnothing$0 may also grow rapidly, reducing $\mathcal{N}(i) \cap C = \varnothing$1. Additionally, CGS does not address intrinsic slow mixing of Gibbs samplers in near-critical or highly dependent regimes; it only lowers per-iteration cost, not the number of sweeps to reach stationarity.

7. Practical Implications and Applications

Conclique-based Gibbs sampling is a "plug-in" replacement for single-site updating in any conditionally specified MRF for which a conclique cover is available. The only additional computational requirement is a one-time graph-coloring (conclique-finding) step. For most spatial and network MRFs with bounded or slowly growing neighborhood size, $\mathcal{N}(i) \cap C = \varnothing$2 is small relative to $\mathcal{N}(i) \cap C = \varnothing$3, enabling large speedups—often of two orders of magnitude or more—for applications requiring repeated simulation (e.g., bootstrap, null-distribution estimation). Available implementations (such as in the R package conclique) make the method accessible for practical use in high-performance MRF simulation environments (Kaplan et al., 2018).