--- title: Bayesian Thinning Algorithm url: https://www.emergentmind.com/topics/bayesian-thinning-algorithm type: topic --- # Bayesian Thinning Algorithm Searching arXiv for relevant papers on “Bayesian thinning algorithm” and related thinning-based Bayesian methods. arxiv_search(query="\"Bayesian thinning algorithm\" OR thinning Bayesian inference OR thinning MCMC Bayesian", max_results=10, sort_by="relevance") Current arXiv usage suggests that the expression **Bayesian thinning algorithm** denotes a family of Bayesian and Bayesian-adjacent procedures built around a common operation: starting from an overcomplete, latent, or proposal representation and then **retaining or discarding elements by a probabilistic rule**. The retained objects differ substantially across domains. In Bayesian neural-network inference with piecewise deterministic Markov processes (PDMPs), thinning is used to simulate event times from an inhomogeneous Poisson process [2302.08724]. In inverse problems for the heat equation, it filters candidate point sources generated by a Bayesian level-set representation [2509.14245]. In MCMC compression, it retains only posterior samples that satisfy a discrepancy criterion based on the Kernelized Stein Discrepancy (KSD) [2201.07130, 2301.13528]. In Bayesian nonparametrics, thinning can zero out stick-breaking variables to induce dependence across Dirichlet processes [2506.18223]. In sigmoidal Gaussian Cox processes, it appears as a latent point-process construction whose thinned points are introduced for data augmentation [2203.06743]. ## 1. Conceptual scope of thinning in Bayesian methodology The term covers several technically distinct constructions. In each case, thinning is not a generic subsampling heuristic but a structurally defined operation embedded in a probabilistic model or inference algorithm. | Context | What is thinned | Purpose | |---|---|---| | PDMP samplers for BNNs | Candidate event times from an auxiliary IPP | Simulate bounce or refresh events | | Heat-equation inverse problem | Candidate point sources from a level set | Remove redundant or spurious sources | | KSD-based MCMC thinning | Posterior samples in a dictionary | Compress the posterior representation | | Thinned dependent DP | Stick-breaking beta variables via indicators | Create shared and unique atoms across groups | | SGCP data augmentation | Base Poisson points | Construct observed and latent thinned processes | In the PDMP setting, the target object is the event-time law of an inhomogeneous Poisson process with rate $\lambda(\omega(t),v(t))$, and thinning is a simulation device. In the heat-equation setting, the candidate set is already a posterior-dependent source configuration, and thinning acts as a leave-one-out posterior filter. In KSD-based methods, thinning operates directly on MCMC output and is explicitly designed to preserve a low-complexity approximation to the posterior. In thinned dependent Dirichlet processes and sigmoidal Gaussian Cox processes, thinning is part of the prior or generative construction itself. A common misconception is that “Bayesian thinning” must refer to a prior over thinning probabilities. The literature here does not support such a single definition. In particular, the PDMP paper explicitly states that its method is **not Bayesian in the sense of a prior over the intensity**, but rather a **Bayesian inference tool** that makes PDMP-based Bayesian MCMC practical for large neural networks [2302.08724]. ## 2. Adaptive thinning for PDMP event times in Bayesian neural networks In PDMP samplers such as the Bouncy Particle Sampler (BPS), preconditioned BPS, and Boomerang, the stochasticity arises only through **event times**, while the path between events is deterministic. The event time $\tau$ must be drawn from an inhomogeneous Poisson process with rate $\lambda(\omega(t),v(t))$. For BPS, \[ \lambda(\omega(t),v)=\max\{0,\nabla U(\omega(t))\cdot v\}, \] with \[ U(\omega)=-\log\big(p(\omega)p(\mathcal D\mid\omega)\big). \] Analytic simulation would require inversion of \[ \Lambda(\tau)=\int_0^\tau \lambda(s)\,ds, \] which is infeasible for neural networks. The proposed solution is a **generic, adaptive thinning scheme** that builds a piecewise-linear envelope $h(t)$ from past evaluations of an inflated rate \[ \hat{\lambda}(\omega(t),v)=\max\{0,\alpha \nabla U(\omega(t))\cdot v\},\qquad \alpha\ge 1. \] Candidate event times are sampled analytically from the proposal IPP with linear-rate segments via a closed-form expression due to Klein (1984), and are accepted with probability $\lambda(t)/h(t)$ together with a safeguard condition $\lambda(t)/h(t)1$, and use $\alpha$ to control the bias–efficiency tradeoff. They set $\alpha=1.0$ in all experiments to favor computational efficiency. Empirically, the approach makes PDMP-based inference computationally feasible for large CNNs on MNIST, SVHN, and CIFAR-10/100, and Boomerang with this thinning achieves ESS $\sim 900$ versus ESS $\sim 2$–$3$ for BPS, preconditioned BPS, SGLD, and SGHMC on Boston housing, while also producing higher predictive variance away from training data and higher predictive entropies on OOD data [2302.08724]. ## 3. Posterior-informed thinning for point-source identification in the heat equation In the heat-equation inverse problem, the forward model is a 2D heat equation on a bounded domain with homogeneous Dirichlet boundary conditions, zero initial data, and a source \[ f(x)=\sum_{i=1}^N w_i\,\delta_{x^{(i)}}(x), \] observed through noisy boundary flux measurements \[ g=\frac{\partial u}{\partial n}\Big|_{\partial\Omega}+\eta,\qquad \eta\sim\mathcal N(0,\Xi). \] The Bayesian formulation uses a Gaussian prior on a latent level-set field $\phi$ defined on a discretized mesh. Point sources are generated by thresholding: \[ f(x)=F(\phi(x)):=\sum_{\phi(x^{(i)})>c}\phi(x^{(i)})\,\delta_{x^{(i)}}(x), \] or, in the equal-intensity case, \[ f(x)=\sum_{\phi(x^{(i)})>c}\delta_{x^{(i)}}(x). \] The posterior on $\phi$ is sampled by pCN, and each accepted field yields a candidate point configuration \[ \theta=\{(x^{(i)},w_i)\}_{i:\phi(x^{(i)})>c}. \] The thinning step is **leave-one-out and posterior-based**. For each candidate point $j$, one forms a configuration $\breve f_j$ with that point deleted and computes \[ \alpha_j=\min\left\{1,\frac{p(\breve f_j\mid g)}{p(f\mid g)}\right\}. \] If $\alpha_j>U$ for $U\sim \mathrm{Uniform}(0,1)$, the point is deleted. This yields a delete-only stochastic filter that prunes redundant, spurious, or clustered points. The paper interprets the candidate configuration as a realization of a marked Poisson point process, with the thinning step acting as a posterior-informed filtering mechanism. Methodologically, the proposal avoids explicit reversible-jump moves by embedding births and disappearances in the level-set field and then applying a thinning correction. The paper notes that the resulting chain on $(\phi,\theta)$ is **nonstandard** and that no formal convergence theorem is given for the combined pCN plus thinning scheme. Numerical results nevertheless show that thinning is crucial: without it, reconstructions deteriorate and relative errors increase. With thinning, reconstructed positions and intensities are reported to be very close to the truth; for example, in one experiment intensity $0.7$ is reconstructed as $0.701$–$0.702$, and intensity $0.5$ as approximately $0.50$. A small ghost point remains in one $N=4$ case, showing that thinning greatly reduces but does not completely eliminate overfitting [2509.14245]. ## 4. KSD-based thinning of posterior samples A separate line of work uses thinning to compress MCMC output while preserving a principled approximation to the target posterior. In **online KSD Thinning**, the retained set is a dictionary $D_t$ supporting an empirical measure \[ q_D(\cdot)=\frac{1}{|D|}\sum_{x_i\in D}\delta_{x_i}(\cdot). \] The discrepancy to the target $p$ is measured by the Kernelized Stein Discrepancy, \[ \operatorname{KSD}^2(Q,p)=\mathbb E_{x,x'\sim Q}[k_0(x,x')], \] which depends only on the score $\nabla\log p(x)$ and a Stein kernel $k_0$. The algorithm processes samples online: after receiving a new MCMC draw, it tentatively adds it, then runs a destructive inner loop that removes the **least influential** point as long as the thinned dictionary satisfies a squared-KSD budget of the form \[ (q_{D_t})^2<(q_{\tilde D_t})^2+\epsilon_t. \] The theory quantifies consistency–complexity tradeoffs through the growth function $f(t)$ and the budget schedule $\epsilon_t$. For decaying budgets with $\epsilon_i=\log(i)/f(i)^2$ and $f(i)=\Omega(\sqrt{i\log i})$, the paper proves \[ \mathbb E[(q_{D_n})^2]\le C\,\frac{n\log n}{f(n)^2}, \] under regularity conditions for either i.i.d. sampling or MALA-generated candidates [2201.07130]. A related but distinct approach is **Stein thinning** as post-processing of a completed MCMC run. The later analysis of this method identifies two pathologies: **mode proportion blindness** for well-separated multimodal targets and **spurious minima** that over-concentrate selected points in low-density stationary regions. The proposed fix is **regularized Stein thinning**, which augments the objective with an entropic term and a truncated-Laplacian correction. The entropic regularized discrepancy is \[ \mathrm{KSD}_\lambda^2(P,Q)=\mathbb E[k_p(Z,Z')]-\lambda\,\mathbb E[\log p(Z)], \] and the empirical L-KSD estimator adds \[ \Delta^+\log p(x)=\sum_{j=1}^d\left(\frac{\partial^2 \log p(x)}{\partial x^{(j)2}}\right)^+ \] to the diagonal term. The paper recommends $\lambda=1/m$ in practice and proves weak convergence of the selected empirical measure under stated MCMC assumptions. Empirically, regularized Stein thinning yields lower MMD than the original algorithm across multimodal synthetic targets and improves posterior-predictive AUC on several Bayesian logistic-regression datasets; for NUTS with $m=300$, Breast Wisconsin improves from ST $0.93\,(0.01)$ to RST $0.96\,(0.00)$, and Liver Disorders from $0.61\,(0.04)$ to $0.70\,(0.01)$ [2301.13528]. These KSD-based methods differ sharply from the PDMP and heat-equation usages: they thin **posterior samples themselves**, not latent events or latent point configurations. This suggests that the expression “Bayesian thinning algorithm” has broadened to include discrepancy-controlled posterior compression. ## 5. Thinning in dependent Dirichlet-process constructions In Bayesian nonparametrics, thinning can be built into the prior. The thinned dependent Dirichlet process begins with a parent stick-breaking representation \[ p(\cdot)=\sum_{j=1}^\infty \omega_j\delta_{\theta_j}(\cdot),\qquad \omega_j=v_j\prod_{h0$ is a scaling constant and $\sigma$ is a sigmoid, originally logistic in the SGCP construction. Since $\lambda\sigma(g(x))\le \lambda$, one may construct the model by first drawing a homogeneous PPP with intensity $\lambda$ and then retaining each point independently with probability $\sigma(g(x))$. The paper’s main contribution is measure-theoretic clarification. Working with the counting–scattering measure, it derives the joint density of the observed and thinned processes by a **colouring theorem**. For the SGCP, if $\tilde S_0$ and $\tilde S_1$ denote the thinned and observed marked configurations, the joint density is available in closed form, while the marginal density of observed points alone remains intractable. This justifies data augmentation: the latent thinned process $\tilde X_0$ can be sampled together with GP values and hyperparameters, thereby converting an intractable marginal model into a tractable complete-data model. A major part of the paper resolves contradictions in earlier SGCP data-augmentation schemes. It argues that some retrospective updates implicitly treated $g(\cdot)\mid \tilde X_1$ as though it were simply a GP conditioned on finitely many observed GP values, which is incorrect because the thinning event itself is informative about $g$. The paper validates a corrected retrospective update and proves its reversibility with respect to the correct conditional law of the thinned process. It also extends the framework to a **multitype SGCP** driven by a multivariate GP and applies it to Lansing Woods tree data, where the estimated cross pair correlation between maples and hickories is less than $1$ for small distances, indicating short-range repulsion [2203.06743]. This point-process usage highlights a final distinction. Here thinning is neither posterior compression nor posterior filtering of candidate parameters; it is a latent-process construction whose augmented thinned points are indispensable for exact specification of the complete-data likelihood. That contrast, together with the other usages surveyed above, indicates that “Bayesian thinning algorithm” is best understood as a cross-domain methodological label for Bayesian procedures in which probabilistic deletion, retention, or zeroing-out is an essential inferential operation rather than a single canonical algorithm. Source: https://www.emergentmind.com/topics/bayesian-thinning-algorithm