---
title: '3DGS-MCMC: Overview & Applications'
url: https://www.emergentmind.com/topics/3dgs-mcmc
type: topic
---

# 3DGS-MCMC: Overview & Applications

3DGS-MCMC is a formulation of 3D Gaussian Splatting (3DGS) in which the Gaussian primitives are interpreted as samples from an implicit distribution over scene representations. It replaces much of conventional 3DGS adaptive density control—heuristic cloning, splitting, pruning, and opacity resets—with stochastic-gradient Langevin dynamics (SGLD)-inspired optimization, opacity- and scale-sparsity regularization, probability-guided sample insertion, and relocalization of inactive Gaussians. The method was introduced in “3D Gaussian Splatting as Markov Chain Monte Carlo” [2404.09591]. The designation MCMC describes a sampling formulation and interpretation; the practical algorithm, which uses Adam, changing population size, scheduled transitions, regularization, and no Metropolis–Hastings acceptance step, is not an exact textbook stationary MCMC sampler with a formal convergence guarantee.

## 1. Representation and motivation

A 3DGS scene consists of anisotropic Gaussian primitives

$$
g_i=(\boldsymbol{\mu}_i,\boldsymbol{\Sigma}_i,o_i,\mathbf{c}_i),
$$

where $\boldsymbol{\mu}_i\in\mathbb{R}^3$ is the center, $\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}$ is the covariance, $o_i$ is opacity, and $\mathbf{c}_i$ is a view-dependent color representation, commonly expressed through spherical-harmonic coefficients. For a camera view, the Gaussians are projected into screen space and sorted by depth. The rendered color at pixel $x$ is

$$
C(x)=\sum_{i=1}^{N}\mathbf{c}_i\alpha_i(x)
\prod_{j=1}^{i-1}(1-\alpha_j(x)),
$$

where the projected opacity contribution is approximately

$$
\alpha_i(x)=o_i\exp\left[
-\frac{1}{2}
(x-\mathbf{J}\boldsymbol{\mu}_i)^\top
(\mathbf{J}\boldsymbol{\Sigma}_i\mathbf{J}^\top)^{-1}
(x-\mathbf{J}\boldsymbol{\mu}_i)
\right].
$$

Here $\mathbf{J}$ denotes the relevant camera projection Jacobian. The differentiable rasterizer can be abstracted as

$$
C=R(\{g_i\}_{i=1}^{N}).
$$

The conventional 3DGS objective combines an $L_1$ image loss and D-SSIM:

$$
L_{\mathrm{3DGS}}
=(1-\lambda_{\mathrm{D\text{-}SSIM}})L_1
+\lambda_{\mathrm{D\text{-}SSIM}}L_{\mathrm{D\text{-}SSIM}},
$$

with $\lambda_{\mathrm{D\text{-}SSIM}}=0.2$. Gaussian parameters are ordinarily optimized by gradient descent or an Adam-preconditioned equivalent.

Conventional 3DGS uses adaptive density control based on gradient magnitudes, projected and world-space sizes, opacity, and manually selected thresholds. Gaussians may be cloned, split, pruned, or subjected to periodic opacity resets. This procedure has several consequences:

- the final Gaussian count is difficult to predict from nominal hyperparameters;
- poor point-cloud initialization can lead to poor Gaussian placement;
- cloning and splitting rules can behave poorly across scenes with different geometry, texture, or initialization;
- opacity-based pruning can remove primitives that later become useful;
- memory and rendering budgets are difficult to control directly.

3DGS-MCMC treats density control as population management. Instead of considering Gaussian primitives solely as rendering elements, it regards them as sample locations moving through parameter space and representing an implicit scene distribution.

## 2. Sampling interpretation and objective

Under the sampling view, a sufficiently large Gaussian population is regarded as samples from an underlying distribution $G$:

$$
g_i\sim G.
$$

Rendering is consequently conceptualized as

$$
C=R\left(\{g_1,\ldots,g_N\mid g_i\sim G\}\right).
$$

The distribution is not explicitly parameterized. Gaussian parameters themselves are treated as samples that move under stochastic updates. If the target distribution is $P$, a generic SGLD update has the form

$$
g_i\leftarrow g_i+a\nabla_{g_i}\log P+b\boldsymbol{\eta}_i,
$$

where $a$ controls the drift, $b$ controls exploration, and

$$
\boldsymbol{\eta}_i\sim\mathcal{N}(\mathbf{0},\mathbf{I}).
$$

When the target is defined implicitly through the total rendering loss,

$$
P\propto\exp(-L_{\mathrm{total}}),
$$

the score becomes

$$
\nabla_{g_i}\log P=-\nabla_{g_i}L_{\mathrm{total}},
$$

yielding the SGLD-like update

$$
g_i\leftarrow
g_i-\lambda_{\mathrm{lr}}\nabla_{g_i}L_{\mathrm{total}}
+\lambda_{\mathrm{noise}}\boldsymbol{\epsilon}_i.
$$

The gradient term moves Gaussians toward configurations that improve reconstruction, while the stochastic term enables exploration of alternative configurations. Vanilla 3DGS corresponds approximately to the zero-noise case: it retains the deterministic drift but lacks explicit exploration.

3DGS-MCMC adds sparsity regularization to the conventional image objective:

$$
L_{\mathrm{total}}
=(1-\lambda_{\mathrm{D\text{-}SSIM}})L_1
+\lambda_{\mathrm{D\text{-}SSIM}}L_{\mathrm{D\text{-}SSIM}}
+\lambda_o\frac{1}{N}\sum_{i=1}^{N}|o_i|
+\lambda_s\frac{1}{N}\sum_{i=1}^{N}
\left\|\sqrt{\operatorname{eig}(\boldsymbol{\Sigma}_i)}\right\|_1.
$$

The opacity penalty encourages unnecessary Gaussians to become low-opacity. The scale penalty acts on the standard deviations along the covariance principal axes and encourages small-scale Gaussians. An inactive Gaussian therefore tends to become low-opacity, small-scale, visually ineffective, and suitable for removal or reallocation.

The regularizer is coupled to the population dynamics. Adding the same regularizer to conventional 3DGS degraded performance in the reported ablation, whereas it improved the complete 3DGS-MCMC procedure. This indicates that the opacity and scale penalties are not independent drop-in improvements; their effect depends on stochastic exploration, sample insertion, and relocalization.

## 3. SGLD-inspired optimization

The practical optimizer uses Adam for the deterministic drift and adds stochastic perturbations primarily to Gaussian centers:

$$
g_i\leftarrow
g_i-\lambda_{\mathrm{lr}}
\operatorname{AdamUpdate}
\left(\nabla_{g_i}L_{\mathrm{total}}\right)
+\boldsymbol{\xi}_i.
$$

The final implementation adds noise to position but not to opacity, scale, orientation, or color. Position noise is covariance- and opacity-dependent:

$$
\boldsymbol{\xi}_i=
\lambda_{\mathrm{lr}}\boldsymbol{\Sigma}_i
\sigma\!\left(-k(o_i-t)\right)
\boldsymbol{\eta}_i,
$$

where

$$
\boldsymbol{\eta}_i\sim\mathcal{N}(\mathbf{0},\mathbf{I}),
$$

and

$$
k=100,\qquad t=1-0.005=0.995.
$$

The sigmoid creates a transition near the conventional opacity-pruning threshold. High-opacity Gaussians receive approximately suppressed noise, while low-opacity Gaussians receive stronger exploratory motion. Covariance determines the spatial scale of the perturbation: narrow Gaussians receive smaller spatial perturbations than broad Gaussians.

The covariance dependence is motivated by the differing spatial support of Gaussians. A fixed perturbation could move a narrow Gaussian outside a region from which it could recover, whereas covariance-scaled noise better matches the Gaussian’s spatial extent.

The practical method is SGLD-inspired rather than an exact fixed-target Langevin chain. Adam changes the effective drift metric; learning rates are scheduled; the Gaussian population changes dimension; relocalization is a nonlocal state transition; regularization changes the objective; and no Metropolis–Hastings acceptance or rejection step is used. The method therefore does not establish that the optimized Gaussians are independent samples from an invariant distribution.

## 4. Sample addition and relocalization

### Sample addition

In conventional 3DGS, densification is triggered by hand-designed conditions involving accumulated image-space gradients, projected size, world-space scale, opacity, and pruning thresholds. In 3DGS-MCMC, adding a Gaussian is interpreted as increasing the sample population.

An existing Gaussian is selected with probability related to opacity:

$$
w_i=\frac{o_i}{\sum_j o_j},
$$

followed by a categorical draw over Gaussian indices. A new small Gaussian is created at the selected Gaussian’s location, assigned a default opacity, and subsequently separated and refined by optimization. The procedure therefore changes the interpretation of densification from “clone when a gradient crosses a threshold” to “draw another sample near an important existing region.”

The number of newly added Gaussians is limited to at most $5\%$ of the current population per insertion event. This limits abrupt population changes and provides gradual progression toward a predefined Gaussian budget.

### Relocalization of inactive Gaussians

A Gaussian with opacity

$$
o_i<0.005
$$

is effectively invisible and receives negligible rendering gradient. Rather than permanently pruning it, 3DGS-MCMC relocalizes its parameter slot. The operation is performed every $100$ iterations:

1. a high-opacity Gaussian is selected using opacity-weighted sampling;
2. useful parameters are copied to the inactive Gaussian;
3. the copied Gaussian is assigned lower opacity;
4. the optimizer state associated with the original Gaussian is reset.

This operation preserves the number of parameter slots while moving inactive capacity toward a region already contributing to the scene. It is analogous to teleporting an inactive walker toward a higher-probability region.

The method’s conceptual probability argument is that duplicating a source sample while dividing its contribution among the resulting samples can approximately preserve aggregate probability mass. However, the final implementation uses the simpler copy-and-lower-opacity rule rather than the earlier mass-preserving resampling formula. Probability preservation is therefore approximate and interpretive, not an exact property of the implemented transition.

## 5. Training procedure and population control

3DGS-MCMC supports initialization from SfM or LiDAR points and random initialization. For random initialization, approximately $100{,}000$ Gaussians are sampled uniformly within a region extending to three times the camera-bounding-box extent.

A typical optimization cycle is:

1. render training views using the current Gaussian population;
2. compute the image loss and opacity/scale regularization;
3. backpropagate through the differentiable rasterizer;
4. apply Adam-preconditioned updates;
5. apply covariance- and opacity-dependent stochastic center perturbations;
6. every $100$ iterations, add opacity-guided samples subject to the $5\%$ population-increase limit;
7. every $100$ iterations, relocalize Gaussians with $o_i<0.005$;
8. continue until the prescribed Gaussian budget is reached.

The position learning rate begins at

$$
1.6\times10^{-4}
$$

and exponentially decays to

$$
1.6\times10^{-6}.
$$

The other relevant settings follow the original 3DGS learning-rate schedule. The population is gradually increased toward a predefined maximum rather than allowed to grow through uncontrolled heuristic cloning.

The intended distribution is often written as

$$
P(g)\propto\exp(-L_{\mathrm{total}}(g)).
$$

Low-loss configurations have higher probability, and stochastic motion is intended to prevent premature collapse into a single local solution. In practice, however, the target distribution is implicit, nonconvex, finite-view, and dynamically modified by regularization, insertion, and relocalization. The method should therefore be understood as an optimization-and-sampling framework rather than a theorem-backed asymptotic sampler.

## 6. Empirical behavior and subsequent developments

The original evaluation used scenes from the NeRF dataset, Mip-NeRF 360, Tanks and Temples, and Deep Blending, with PSNR, SSIM, and LPIPS as the principal metrics. Reported random-initialization averages include:

| Dataset | 3DGS PSNR | 3DGS-MCMC PSNR | 3DGS-MCMC LPIPS |
|---|---:|---:|---:|
| NeRF scenes | 33.42 | 33.85 | 0.04 |
| Mip-NeRF 360 | 27.89 | 29.58 | 0.19 |
| Tanks and Temples | 21.93 | 24.47 | 0.19 |
| Deep Blending | 29.55 | 29.66 | 0.31 |

The strongest improvements occur with random initialization, supporting the claim that stochastic exploration reduces dependence on a good initial point set. With point-cloud initialization, improvements are more modest but remain positive in the reported averages.

Subsequent work has extended or modified the population and optimization mechanisms:

- **Metropolis–Hastings 3DGS** replaces opacity-guided population operations with posterior-inspired birth proposals, multi-view error scores, voxel sparsity, coarse-to-fine position proposals, and probabilistic acceptance [2506.12945]. Unlike 3DGS-MCMC, it adaptively changes the Gaussian count through proposed births and rejection. Its practical acceptance rule is itself an approximation rather than an exact reversible birth-death sampler.

- **AdamW-GS** decomposes optimization into Sparse Adam, Re-State Regularization, and Decoupled Attribute Regularization [2601.16736]. Sparse Adam avoids updates and moment decay for invisible Gaussians; Re-State Regularization explicitly rescales optimizer states; and decoupled regularization prevents opacity and scale gradients from contaminating photometric Adam moments. In the reported Mip-NeRF 360 experiments, AdamW-GS improved the original MCMC aggregate from $27.948$ to $28.219$ PSNR, from $0.833$ to $0.840$ SSIM, and reduced runtime from $46.81$ to $39.77$ minutes.

- **Post-training dictionary learning** compresses the spherical-harmonic appearance parameters of a trained 3DGS-MCMC model without changing its Gaussian population or MCMC training process [2605.30396]. At the reported operating point, the method achieved a $3.10\times$ compression ratio, increased rendering speed from $113.0$ to $140.5$ FPS, and produced approximately $0.19$ dB mean PSNR degradation against ground truth.

- **SalientGS** retains the SGLD dynamics of 3DGS-MCMC while guiding birth and relocation using multi-view underfit and redundancy scores [2607.11285]. It is explicitly described as a heuristic allocation layer rather than a Metropolis–Hastings proposal. On Mip-NeRF 360, the reported result was $28.82$ PSNR, compared with $28.01$ for 3DGS-MCMC, using a $1.5$M-Gaussian budget.

- **Confidence-aware densification** replaces opacity-driven source sampling and relocation with image-error-driven signals, learned per-primitive confidence, structural error, and rendered contribution [2610.13541]. It retains random or prior-free initialization, gradual growth, a fixed primitive budget, and relocation. Averaged over 24 scenes, it outperformed 3DGS-MCMC in PSNR, SSIM, and LPIPS at budgets from $100\mathrm{k}$ to $2\mathrm{M}$ primitives; at $100\mathrm{k}$, the reported PSNR improvement over 3DGS-MCMC was $0.84$ dB.

These developments expose a central design tension. Opacity is computationally convenient and correlates with rendered contribution in many regions, but it can reinforce foreground allocation and neglect poorly represented backgrounds. Error-driven and confidence-aware methods attempt to allocate capacity according to reconstruction deficiency rather than existing opacity. Conversely, stronger error guidance introduces additional schedules, residual computations, and hyperparameters.

## 7. Scope, limitations, and interpretation

3DGS-MCMC’s principal contribution is the reinterpretation of adaptive density control as stochastic population management. Its components have distinct roles:

- gradient-based drift improves rendering configurations;
- position noise enables exploration;
- opacity-guided insertion allocates samples to currently important regions;
- opacity and scale regularization promotes sparsification;
- relocalization recycles inactive Gaussian slots;
- an explicit population budget improves memory and rendering predictability.

The following claims are empirically supported within the reported experiments: 3DGS-MCMC can improve rendering quality at comparable Gaussian counts, is more robust to random initialization than conventional 3DGS, benefits from explicit stochastic exploration, provides direct population control, and reduces reliance on the original clone/split/prune/reset framework.

The following properties are not formally established for the practical algorithm:

- exact sampling from $P\propto\exp(-L_{\mathrm{total}})$;
- exact preservation of probability under relocalization;
- reversibility of the opacity-dependent noise schedule;
- formal convergence of Adam-based dynamics with changing population size;
- independence of the resulting Gaussian states.

The method is therefore best characterized as a principled sampling formulation integrated with differentiable rendering and adaptive population control. Its significance lies less in introducing a new rasterizer or Gaussian parameterization than in replacing heuristic density control with a framework that combines stochastic exploration, sparsity-driven inactivity, probabilistic sample allocation, and Gaussian-slot recycling.

Source: https://www.emergentmind.com/topics/3dgs-mcmc