Gibbs Sampling with LLMs

• Large Language Gibbs is a method that applies Gibbs-style iterative resampling using LLM local conditionals to perform probabilistic inference over structured assignments.
• It replaces one-pass autoregressive generation with iterative updates, reducing order dependence and inconsistencies in variable assignments.
• Empirical results demonstrate improvements in marginal fidelity and internal consistency, with notable applications in prior elicitation and Bayesian structure learning.

Large Language Gibbs denotes a Gibbs-style use of LLMs for probabilistic inference over structured states rather than one-pass left-to-right text generation. In its most direct formulation, introduced in "Structured Inference with Large Language Gibbs" (Choi et al., 17 Jun 2026), an LLM is queried for local conditionals of one variable given the others, and those conditionals are used as MCMC transition operators over a discrete structured state $X=(X_1,\dots,X_n)$. In a broader research sense, the label also touches adjacent Gibbs-like uses of LLMs, including iterated-learning procedures for prior elicitation and particle Gibbs over diffusion trajectories, but those settings differ sharply in state space, target semantics, and validity conditions (Cui et al., 12 Jun 2025, Dang et al., 11 Jul 2025).

1. Conceptual motivation and scope

The central motivation is that many inference problems posed to LLMs are not naturally autoregressive. If the object of interest is a structured assignment

$X=(X_1,\dots,X_n),$

where variables correspond to related claims, record fields, or attributes of a world state, then ordinary left-to-right generation imposes an arbitrary order on variables that may have no semantic priority. The reported consequences are order dependence, prompt-order sensitivity, inconsistency between early and late variables, and a lack of probabilistic coherence over the full structured object (Choi et al., 17 Jun 2026).

Large Language Gibbs addresses this by replacing single-pass generation with iterative local resampling. Rather than asking the model to emit the whole structure once, it asks for the distribution of one variable $X_i$ conditioned on the current values of the other variables $X_{-i}$, then repeatedly updates individual coordinates. The resulting Markov chain has a stationary distribution even when the local LLM conditionals are noisy or mutually inconsistent, and that stationary distribution is intended to reflect a compromise among all local conditional beliefs rather than the artifact of a fixed serialization order (Choi et al., 17 Jun 2026).

Three closely related but distinct uses of Gibbs ideas in language modeling are worth separating.

Usage	State being updated	Central issue
Large Language Gibbs	Structured assignment $X$	Using LLM local conditionals as Gibbs-style transition operators
Simulated Gibbs for prior elicitation	Iterated latent states such as $\theta$	Whether LLM outputs are genuinely stochastic rather than near-deterministic
Particle Gibbs for diffusion LLMs	Full diffusion trajectories $x_{0:T}$	Inference-time scaling toward a reward-weighted target

This separation matters because convergence of an iterative procedure, validity of a Gibbs interpretation, and faithfulness to an underlying joint distribution are not equivalent claims.

2. Formal construction from LLM conditionals

The structured state space is

$$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$

with each $X_i$ discrete and associated with a natural-language description or context $c_i$. Classical Gibbs sampling assumes access to exact full-conditionals from a joint $X=(X_1,\dots,X_n),$0: $X=(X_1,\dots,X_n),$1 Large Language Gibbs instead uses approximate local conditionals

$X=(X_1,\dots,X_n),$2

where the $X=(X_1,\dots,X_n),$3 are defined from the LLM (Choi et al., 17 Jun 2026).

Because an autoregressive LLM natively exposes next-token probabilities over prefixes rather than unary conditionals over structured variables, the construction serializes the other variables into a prompt in a random order and asks the model to generate the missing variable. The paper defines

$X=(X_1,\dots,X_n),$4

where $X=(X_1,\dots,X_n),$5 is a uniformly random permutation of all indices except $X=(X_1,\dots,X_n),$6, $X=(X_1,\dots,X_n),$7 serializes the corresponding variable descriptions and values, and $X=(X_1,\dots,X_n),$8 denotes sequence concatenation. The randomization is meant to average out prompt-order asymmetries (Choi et al., 17 Jun 2026).

The same work defines an order-averaged or symmetrized joint

$X=(X_1,\dots,X_n),$9

together with

$X_i$0

so that $X_i$1, and

$X_i$2

as a function of $X_i$3 (Choi et al., 17 Jun 2026).

A sufficient compatibility condition is

$X_i$4

Under this assumption,

$X_i$5

and $X_i$6 is stationary for the chain. If each local conditional has full support, the chain is irreducible and its unique stationary distribution is $X_i$7 (Choi et al., 17 Jun 2026).

Without that assumption, the local conditionals can be incompatible: there may be no single joint distribution whose exact unary conditionals equal all of them. The chain still has a unique stationary distribution under full-support conditions, but generally

$X_i$8

The stationary law is then best understood as an induced equilibrium distribution, described in the paper as a compromise among local conditionals. This places Large Language Gibbs near the literature on Gibbs sampling with approximate or inconsistent conditionals, dependency networks, and pseudo-likelihood-like constructions (Choi et al., 17 Jun 2026).

3. Algorithmic realization and implementation details

At the algorithmic level, Large Language Gibbs is simple but expensive. One initializes $X_i$9, for example by autoregressive generation, then iterates: sample a variable index $X_{-i}$0, sample a random permutation of the remaining variables, construct a prompt containing the other variables and their current values, and resample $X_{-i}$1 from the LLM-defined conditional. After burn-in, one collects samples, optionally with thinning, and estimates marginals or majority-vote predictions from the retained states (Choi et al., 17 Jun 2026).

The representation assumption is important. Each $X_{-i}$2 must be encodable as a constrained token sequence, such as an enum, a bounded integer, a binary label, or another short categorical string. In the reported experiments, all runs use structured output generation to enforce JSON or binary formats. This makes the method naturally suited to discrete structured variables. Continuous variables can be handled only indirectly through textual numeric representations, which the paper treats as imperfect (Choi et al., 17 Jun 2026).

Prompt construction is part of the statistical design. In the toy cat-record example, to resample Coat, the prompt lists the current values of Sleep, Breed, and Age in random order, then presents the Coat field with its value omitted. The point is not merely missing-value completion; it is repeated conditional resampling under random serialization, so that no single prompt order dominates the induced joint behavior (Choi et al., 17 Jun 2026).

The theoretical exposition is framed in random-scan Gibbs, but the appendix reports that the implementation uses sweeping by default. With exact compatible conditionals, sweeping preserves the same target as random scan. With incompatible approximate conditionals, the stationary distribution under sweeping may differ from that of random scan. Scan schedule is therefore not a neutral engineering choice in this setting (Choi et al., 17 Jun 2026).

Several practical details are explicit. In simple-distribution experiments, burn-in is $X_{-i}$3 and thinning is $X_{-i}$4 for state size $X_{-i}$5 and block size $X_{-i}$6. In consistent reasoning, burn-in is $X_{-i}$7, thinning is $X_{-i}$8, 25 retained samples are used, and the final answer is obtained by majority vote. In Bayesian structure learning, multiple parallel chains generate synthetic datapoints, again with burn-in and thinning depending on $X_{-i}$9 and block size. Temperature $X$0 is used generally, while Gambling Gibbs uses greedy decoding because it asks for a deterministic accept/reject decision (Choi et al., 17 Jun 2026).

The computational trade-off is explicit. The method is much more expensive than one-shot generation because it requires many LLM calls per sample. Reported runtimes are nevertheless practical for the studied problem sizes: under 10 minutes for 25 seeds on the simple-distribution experiments with Llama-3.1-8B, up to 40 minutes for consistent reasoning at $X$1, and up to 5 minutes for synthetic-data generation in structure learning, although the downstream structure-learning stage adds roughly 30 minutes (Choi et al., 17 Jun 2026).

4. Empirical behavior on structured reasoning and synthetic sampling

The first empirical setting tests whether the method can recover simple target distributions better than direct prompting. Two targets are used: the discrete uniform distribution on $X$2 and the standard Gaussian $X$3. The state is

$X$4

and ideally the $X$5 are i.i.d. from the target. The reported baselines are independent sampling, batch sampling, Gibbs sampling, block Gibbs, and, for instruction-tuned models, Barker Gibbs and Gambling Gibbs. The figures report empirical sample distributions and maximum autocorrelation over lags 1–128. For base models, independent sampling shows strong bias, batch sampling improves marginals but produces high autocorrelation, and Gibbs sampling substantially improves both marginal fidelity and dependence structure. For instruction-tuned models, Gambling Gibbs fails badly for Llama-3.1-8B-Instruct, while appendix results show that OLMo-3-32B-Think partially rescues it (Choi et al., 17 Jun 2026).

The second setting treats consistent reasoning as inference over binary variables. On TruthfulQA and GSM8K-Verification, each question contributes four candidate claims, and if $X$6 questions are grouped together then the state has $X$7 binary variables. The reported settings are $X$8, corresponding to $X$9. Relative to zero-shot greedy, 1-pass autoregressive, 4-pass autoregressive, and a corrected implementation of Internal Coherence Maximisation, Large Language Gibbs improves consistently as the context group grows. On TruthfulQA, Llama-3.1-8B rises from 0.597 for 1-pass and 0.617 for 4-pass to 0.736 at $\theta$0; OLMo-3-32B rises from 0.604 and 0.638 to 0.743. On GSM8K-Verification, Llama-3.1-8B reaches 0.895 at $\theta$1, above ICM at 0.724 and 4-pass at 0.626; OLMo-3-32B reaches 0.840, above ICM at 0.750 and 4-pass at 0.665 (Choi et al., 17 Jun 2026).

These results are interpreted as evidence for three effects. Iterative consistency enforcement helps. Random shuffling helps, since the progression from 1-pass to 4-pass to Gibbs at $\theta$2 suggests that fixed-order artifacts matter. Larger grouped contexts also help, since performance generally improves as $\theta$3 increases from 4 to 16 to 64, indicating that conditioning on more related variables can elicit stronger internal consistency (Choi et al., 17 Jun 2026).

The third setting, Bayesian structure learning, is methodologically distinctive. Here the LLM is not asked to emit a graph directly. Instead it generates synthetic records over dataset variables, and those synthetic datapoints define an LLM-informed prior over Bayesian network structures. The reported datasets are tubercolosis ($\theta$4), knowledge ($\theta$5), disputed1 ($\theta$6), and consequenceCovid ($\theta$7). A synthetic record is a structured state whose fields are feature values; the method generates $\theta$8 synthetic datapoints by repeated conditional resampling, using block size $\theta$9 for smaller datasets and $x_{0:T}$0 for larger ones (Choi et al., 17 Jun 2026).

5. Stationarity, compromise distributions, and the critique of false priors

Large Language Gibbs is easiest to justify when local conditionals are genuinely stochastic and approximately compatible. A separate line of work on simulated Gibbs sampling for LLM prior elicitation shows why this assumption cannot be taken for granted. In "Do LLMs Have Bayesian Brains? Distinguishing Stochastic and Deterministic Decision Patterns within LLMs" (Cui et al., 12 Jun 2025), the core claim is that prior work often assumes an LLM is a stochastic sampler from a meaningful posterior-like distribution, but in practice many LLMs can behave nearly deterministically, even when temperature is non-zero.

The canonical analysis uses a proportion-estimation task. There is a latent variable $x_{0:T}$1 and an observation $x_{0:T}$2, with the intended posterior-sampling interpretation

$x_{0:T}$3

In the coin task, one starts from an observation $x_{0:T}$4, prompts the model to predict the number of heads $x_{0:T}$5 in $x_{0:T}$6 future tosses, converts that answer to

$x_{0:T}$7

and then generates the next observation by

$x_{0:T}$8

If the model actually samples $x_{0:T}$9 from a posterior, the resulting iterated-learning process is interpretable as Gibbs sampling and the stationary marginal on $$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$0 can be read as a prior. The critique is that the model may instead behave like a deterministic maximum-likelihood estimator (Cui et al., 12 Jun 2025).

In the deterministic account,

$$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$1

This is no longer Gibbs sampling. The randomness comes only from the external resampling step. The chain has absorbing states at $$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$2 and $$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$3, so the stationary distribution has support only on $$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$4. The conservation-of-expectation argument yields

$$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$5

The induced stationary distribution over $$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$6 is therefore bimodal at $$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$7 and $$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$8, and if $$X=(X_1,\dots,X_n)\in \mathcal{X}=\mathcal{X}_1\times\cdots\times\mathcal{X}_n,$$9 it can look qualitatively like a sharply U-shaped Beta prior. The paper calls this a false prior because the apparent prior is produced by deterministic dynamics plus external noise rather than by posterior sampling inside the model (Cui et al., 12 Jun 2025).

The practical diagnostic proposed there is initialization invariance. The test is: vary the initial value $X_i$0 and compare the resulting stationary distributions. If the distribution remains consistent across different $X_i$1 values, the process is likely stochastic. If not, it is more likely deterministic. Empirically, this criterion separates models and tasks. On CoinFlip at temperature $X_i$2, GPT-4o-mini behaves deterministically and aligns with the MLE prediction; Claude-3.5-Sonnet, LLaMA-3.1-70B-Instruct, and Claude-3-Haiku behave stochastically by the paper’s criterion; GPT-4o and Gemma-2-2B-it are intermediate. On LifeExpectancy, all models show convergent distributions that are similar across initial conditions, which the authors interpret as evidence of stochastic decision-making (Cui et al., 12 Jun 2025).

For the broader Large Language Gibbs literature, this critique has a narrow but important implication. Convergence of an LLM-driven iterative procedure is not, by itself, evidence that one has recovered a meaningful prior or a coherent posterior sampler. A stationary distribution may be an equilibrium of the update rule rather than a faithful readout of the model’s latent beliefs. This suggests that initialization dependence, effective determinism under constrained prompts, and scan dynamics must be treated as first-class validity questions rather than afterthoughts.

6. Applications to prior elicitation, structure learning, and related Gibbs-style extensions

The Bayesian structure-learning application in Large Language Gibbs makes the prior-elicitation role explicit. Given observed data $X_i$3, Bayesian structure learning seeks

$X_i$4

where $X_i$5 is a DAG and $X_i$6 is the Bayesian network marginal likelihood. The LLM-generated synthetic dataset defines an LLM-informed prior

$X_i$7

which yields

$X_i$8

In the reported experiments, Gibbs-generated synthetic data usually improves over the uniform-prior baseline and generally outperforms direct autoregressive synthetic data generation. Appendix ablations show that moderate $X_i$9 values such as $c_i$0 or $c_i$1 work best, while $c_i$2 often hurts, indicating that the LLM should regularize rather than dominate the graph posterior (Choi et al., 17 Jun 2026).

A separate but related extension appears in reward-guided generation for discrete diffusion LLMs. "Inference-Time Scaling of Diffusion LLMs with Particle Gibbs Sampling" formulates a reward-weighted target

$c_i$3

then builds a Markov chain over full diffusion trajectories $c_i$4 using particle Gibbs with conditional Sequential Monte Carlo as the transition kernel. This is not coordinate-wise Gibbs over structured variables, but it is a genuine Gibbs-style PMCMC method for language generation. Its state is the full trajectory rather than a record-like assignment, and its main contribution is iterative multi-trajectory refinement toward a reward-weighted posterior under fixed inference-time compute budgets (Dang et al., 11 Jul 2025).

The broader Gibbs literature also suggests methodological directions for future Large Language Gibbs systems. One such direction is scan-order optimization. A non-uniform random-scan Gibbs sampler with fixed positive probabilities

$c_i$5

still leaves the target distribution invariant, because any convex mixture of $c_i$6-invariant coordinate-update kernels is itself $c_i$7-invariant. This suggests that scan order is itself an optimization variable. A plausible implication is that future structured LLM samplers could prioritize uncertain or consequential variables rather than scanning uniformly, although the exact weighting rule $c_i$8 from the mean-field analysis does not transfer directly to strongly dependent language-model state spaces (Wang et al., 2024).

Taken together, these developments define Large Language Gibbs as a family of methods for replacing one-pass generation with iterative probabilistic refinement. In its structured-inference form, it uses LLM local conditionals over discrete variables and targets a stationary distribution over full assignments. In prior-elicitation settings, it raises hard questions about whether apparent convergence reflects genuine stochastic sampling or only false priors generated by deterministic dynamics. In diffusion-language settings, it generalizes to particle Gibbs over latent trajectories. Across all of these variants, the common theme is that an LLM’s local predictive behavior is used as a transition mechanism for MCMC, and the central challenge is to understand when the resulting stationary distribution is a trustworthy proxy for the model’s underlying probabilistic knowledge.