Bayesian Thinning Algorithm

• Bayesian thinning algorithm is a family of methods that applies probabilistic retention or deletion of latent candidates to refine inference outputs.
• It is employed in diverse applications such as PDMP samplers for Bayesian neural networks, heat-equation inverse problems, KSD-based MCMC compression, and Bayesian nonparametrics.
• The approach optimizes computational efficiency and model accuracy by adaptively balancing bias and efficiency in thinning procedures across various Bayesian frameworks.

Searching arXiv for relevant papers on “Bayesian thinning algorithm” and related thinning-based Bayesian methods. arxiv_search(query="\"Bayesian thinning algorithm\" OR thinning Bayesian inference OR thinning MCMC Bayesian", max_results=10, sort_by="relevance") Current arXiv usage suggests that the expression Bayesian thinning algorithm denotes a family of Bayesian and Bayesian-adjacent procedures built around a common operation: starting from an overcomplete, latent, or proposal representation and then retaining or discarding elements by a probabilistic rule. The retained objects differ substantially across domains. In Bayesian neural-network inference with piecewise deterministic Markov processes (PDMPs), thinning is used to simulate event times from an inhomogeneous Poisson process (Goan et al., 2023). In inverse problems for the heat equation, it filters candidate point sources generated by a Bayesian level-set representation (Deng et al., 4 Sep 2025). In MCMC compression, it retains only posterior samples that satisfy a discrepancy criterion based on the Kernelized Stein Discrepancy (KSD) (Hawkins et al., 2022, Bénard et al., 2023). In Bayesian nonparametrics, thinning can zero out stick-breaking variables to induce dependence across Dirichlet processes (D'Angelo et al., 23 Jun 2025). In sigmoidal Gaussian Cox processes, it appears as a latent point-process construction whose thinned points are introduced for data augmentation (Alie et al., 2022).

1. Conceptual scope of thinning in Bayesian methodology

The term covers several technically distinct constructions. In each case, thinning is not a generic subsampling heuristic but a structurally defined operation embedded in a probabilistic model or inference algorithm.

Context	What is thinned	Purpose
PDMP samplers for BNNs	Candidate event times from an auxiliary IPP	Simulate bounce or refresh events
Heat-equation inverse problem	Candidate point sources from a level set	Remove redundant or spurious sources
KSD-based MCMC thinning	Posterior samples in a dictionary	Compress the posterior representation
Thinned dependent DP	Stick-breaking beta variables via indicators	Create shared and unique atoms across groups
SGCP data augmentation	Base Poisson points	Construct observed and latent thinned processes

In the PDMP setting, the target object is the event-time law of an inhomogeneous Poisson process with rate $\lambda(\omega(t),v(t))$, and thinning is a simulation device. In the heat-equation setting, the candidate set is already a posterior-dependent source configuration, and thinning acts as a leave-one-out posterior filter. In KSD-based methods, thinning operates directly on MCMC output and is explicitly designed to preserve a low-complexity approximation to the posterior. In thinned dependent Dirichlet processes and sigmoidal Gaussian Cox processes, thinning is part of the prior or generative construction itself.

A common misconception is that “Bayesian thinning” must refer to a prior over thinning probabilities. The literature here does not support such a single definition. In particular, the PDMP paper explicitly states that its method is not Bayesian in the sense of a prior over the intensity, but rather a Bayesian inference tool that makes PDMP-based Bayesian MCMC practical for large neural networks (Goan et al., 2023).

2. Adaptive thinning for PDMP event times in Bayesian neural networks

In PDMP samplers such as the Bouncy Particle Sampler (BPS), preconditioned BPS, and Boomerang, the stochasticity arises only through event times, while the path between events is deterministic. The event time $\tau$ must be drawn from an inhomogeneous Poisson process with rate $\lambda(\omega(t),v(t))$. For BPS,

$$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$

with

$$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$

Analytic simulation would require inversion of

$\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$

which is infeasible for neural networks. The proposed solution is a generic, adaptive thinning scheme that builds a piecewise-linear envelope $h(t)$ from past evaluations of an inflated rate

$$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$

Candidate event times are sampled analytically from the proposal IPP with linear-rate segments via a closed-form expression due to Klein (1984), and are accepted with probability $\lambda(t)/h(t)$ together with a safeguard condition $\lambda(t)/h(t)<R$, where the experiments use $\tau$0.

The construction is adaptive in a precise sense. At the start of each deterministic segment, the rate is evaluated at $\tau$1 and at a small positive $\tau$2, thereby defining the first linear segment. If a proposal is rejected, the rejected time and its evaluated rate are appended to the interpolation set, and the next segment of $\tau$3 is recomputed between the last two points. This progressively tightens the envelope where thinning has been inefficient. The method is compatible with subsampled gradients,

$\tau$4

and with automatic differentiation frameworks, specifically TensorFlow Probability.

The method is explicitly approximate because the piecewise-linear envelope is only an approximate upper bound. The authors state that “bias will likely be introduced during inference,” diagnose violations through ratios $\tau$5, and use $\tau$6 to control the bias–efficiency tradeoff. They set $\tau$7 in all experiments to favor computational efficiency. Empirically, the approach makes PDMP-based inference computationally feasible for large CNNs on MNIST, SVHN, and CIFAR-10/100, and Boomerang with this thinning achieves ESS $\tau$8 versus ESS $\tau$9–$\lambda(\omega(t),v(t))$0 for BPS, preconditioned BPS, SGLD, and SGHMC on Boston housing, while also producing higher predictive variance away from training data and higher predictive entropies on OOD data (Goan et al., 2023).

3. Posterior-informed thinning for point-source identification in the heat equation

In the heat-equation inverse problem, the forward model is a 2D heat equation on a bounded domain with homogeneous Dirichlet boundary conditions, zero initial data, and a source

$\lambda(\omega(t),v(t))$1

observed through noisy boundary flux measurements

$\lambda(\omega(t),v(t))$2

The Bayesian formulation uses a Gaussian prior on a latent level-set field $\lambda(\omega(t),v(t))$3 defined on a discretized mesh. Point sources are generated by thresholding: $\lambda(\omega(t),v(t))$4 or, in the equal-intensity case,

$\lambda(\omega(t),v(t))$5

The posterior on $\lambda(\omega(t),v(t))$6 is sampled by pCN, and each accepted field yields a candidate point configuration

$\lambda(\omega(t),v(t))$7

The thinning step is leave-one-out and posterior-based. For each candidate point $\lambda(\omega(t),v(t))$8, one forms a configuration $\lambda(\omega(t),v(t))$9 with that point deleted and computes

$$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$0

If $$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$1 for $$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$2, the point is deleted. This yields a delete-only stochastic filter that prunes redundant, spurious, or clustered points. The paper interprets the candidate configuration as a realization of a marked Poisson point process, with the thinning step acting as a posterior-informed filtering mechanism.

Methodologically, the proposal avoids explicit reversible-jump moves by embedding births and disappearances in the level-set field and then applying a thinning correction. The paper notes that the resulting chain on $$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$3 is nonstandard and that no formal convergence theorem is given for the combined pCN plus thinning scheme. Numerical results nevertheless show that thinning is crucial: without it, reconstructions deteriorate and relative errors increase. With thinning, reconstructed positions and intensities are reported to be very close to the truth; for example, in one experiment intensity $$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$4 is reconstructed as $$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$5–$$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$6, and intensity $$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$7 as approximately $$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$8. A small ghost point remains in one $$\lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\},$$9 case, showing that thinning greatly reduces but does not completely eliminate overfitting (Deng et al., 4 Sep 2025).

4. KSD-based thinning of posterior samples

A separate line of work uses thinning to compress MCMC output while preserving a principled approximation to the target posterior. In online KSD Thinning, the retained set is a dictionary $$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$0 supporting an empirical measure

$$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$1

The discrepancy to the target $$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$2 is measured by the Kernelized Stein Discrepancy,

$$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$3

which depends only on the score $$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$4 and a Stein kernel $$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$5. The algorithm processes samples online: after receiving a new MCMC draw, it tentatively adds it, then runs a destructive inner loop that removes the least influential point as long as the thinned dictionary satisfies a squared-KSD budget of the form

$$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$6

The theory quantifies consistency–complexity tradeoffs through the growth function $$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$7 and the budget schedule $$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$8. For decaying budgets with $$U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big).$$9 and $\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$0, the paper proves

$\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$1

under regularity conditions for either i.i.d. sampling or MALA-generated candidates (Hawkins et al., 2022).

A related but distinct approach is Stein thinning as post-processing of a completed MCMC run. The later analysis of this method identifies two pathologies: mode proportion blindness for well-separated multimodal targets and spurious minima that over-concentrate selected points in low-density stationary regions. The proposed fix is regularized Stein thinning, which augments the objective with an entropic term and a truncated-Laplacian correction. The entropic regularized discrepancy is

$\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$2

and the empirical L-KSD estimator adds

$\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$3

to the diagonal term. The paper recommends $\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$4 in practice and proves weak convergence of the selected empirical measure under stated MCMC assumptions. Empirically, regularized Stein thinning yields lower MMD than the original algorithm across multimodal synthetic targets and improves posterior-predictive AUC on several Bayesian logistic-regression datasets; for NUTS with $\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$5, Breast Wisconsin improves from ST $\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$6 to RST $\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$7, and Liver Disorders from $\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$8 to $\Lambda(\tau)=\int_0^\tau \lambda(s)\,ds,$9 (Bénard et al., 2023).

These KSD-based methods differ sharply from the PDMP and heat-equation usages: they thin posterior samples themselves, not latent events or latent point configurations. This suggests that the expression “Bayesian thinning algorithm” has broadened to include discrepancy-controlled posterior compression.

5. Thinning in dependent Dirichlet-process constructions

In Bayesian nonparametrics, thinning can be built into the prior. The thinned dependent Dirichlet process begins with a parent stick-breaking representation

$h(t)$0

and introduces binary indicators $h(t)$1 for groups $h(t)$2. Group-specific weights are

$h(t)$3

yielding

$h(t)$4

If $h(t)$5, the $h(t)$6th atom is skipped for group $h(t)$7, but the remaining mass is redistributed over later atoms. Conditionally on the thinning pattern, each $h(t)$8 remains marginally a $h(t)$9.

The thinning pattern controls dependence through shared and unique atoms. For two groups, only indices with $$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$0 contribute to the correlation between $$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$1 and $$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$2. Under Bernoulli thinning,

$$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$3

the marginal correlation is given explicitly, and when $$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$4,

$$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$5

As $$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$6, sharing is maximal; as $$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$7, sharing disappears.

Posterior inference is performed by a blocked Gibbs sampler that updates thinning indicators $$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$8, shared stick-breaking variables $$\hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1.$$9, latent allocation variables $\lambda(t)/h(t)$0, atom parameters, and the group-specific thinning probabilities $\lambda(t)/h(t)$1. In simulations, the model is compared with complete pooling, no pooling, the Common Atoms Model, and GM-DDP. The reported effect of thinning is to reduce uncertainty in group-specific inferences while preventing excessive borrowing of information. In the Collaborative Perinatal Project application on gestational age from 12 hospitals, the model identifies three global clusters with means approximately $\lambda(t)/h(t)$2, $\lambda(t)/h(t)$3, and $\lambda(t)/h(t)$4 weeks, and produces a hospital-level partition into two clusters based on sharing of mixture components (D'Angelo et al., 23 Jun 2025).

6. Thinning as latent augmentation in sigmoidal Gaussian Cox processes

For sigmoidal Gaussian Cox processes (SGCPs), thinning is the generative mechanism linking a tractable base process to an intractable observed process. Conditional on a Gaussian process $\lambda(t)/h(t)$5, the observed process is a Poisson point process with intensity

$\lambda(t)/h(t)$6

where $\lambda(t)/h(t)$7 is a scaling constant and $\lambda(t)/h(t)$8 is a sigmoid, originally logistic in the SGCP construction. Since $\lambda(t)/h(t)$9, one may construct the model by first drawing a homogeneous PPP with intensity $\lambda(t)/h(t)<R$0 and then retaining each point independently with probability $\lambda(t)/h(t)<R$1.

The paper’s main contribution is measure-theoretic clarification. Working with the counting–scattering measure, it derives the joint density of the observed and thinned processes by a colouring theorem. For the SGCP, if $\lambda(t)/h(t)<R$2 and $\lambda(t)/h(t)<R$3 denote the thinned and observed marked configurations, the joint density is available in closed form, while the marginal density of observed points alone remains intractable. This justifies data augmentation: the latent thinned process $\lambda(t)/h(t)<R$4 can be sampled together with GP values and hyperparameters, thereby converting an intractable marginal model into a tractable complete-data model.

A major part of the paper resolves contradictions in earlier SGCP data-augmentation schemes. It argues that some retrospective updates implicitly treated $\lambda(t)/h(t)<R$5 as though it were simply a GP conditioned on finitely many observed GP values, which is incorrect because the thinning event itself is informative about $\lambda(t)/h(t)<R$6. The paper validates a corrected retrospective update and proves its reversibility with respect to the correct conditional law of the thinned process. It also extends the framework to a multitype SGCP driven by a multivariate GP and applies it to Lansing Woods tree data, where the estimated cross pair correlation between maples and hickories is less than $\lambda(t)/h(t)<R$7 for small distances, indicating short-range repulsion (Alie et al., 2022).

This point-process usage highlights a final distinction. Here thinning is neither posterior compression nor posterior filtering of candidate parameters; it is a latent-process construction whose augmented thinned points are indispensable for exact specification of the complete-data likelihood. That contrast, together with the other usages surveyed above, indicates that “Bayesian thinning algorithm” is best understood as a cross-domain methodological label for Bayesian procedures in which probabilistic deletion, retention, or zeroing-out is an essential inferential operation rather than a single canonical algorithm.