Papers
Topics
Authors
Recent
Search
2000 character limit reached

3DGS-MCMC: Overview & Applications

Updated 14 October 2026
  • 3DGS-MCMC is a 3D Gaussian Splatting variant where Gaussians are treated as stochastic samples drawing samples from an implicit distribution during rendering.
  • The method offers that its practical density control framework utilizes stochastic-gradient Langevin dynamics to enhance robustness and reduce dependence on initial point placement.
  • 3DGS-MCMC improves rendering efficiency by regularizing opacity and scale to manage Gaussian primitives, enabling better memory and compute resource management during training.

3DGS-MCMC is a formulation of 3D Gaussian Splatting (3DGS) in which the Gaussian primitives are interpreted as samples from an implicit distribution over scene representations. It replaces much of conventional 3DGS adaptive density control—heuristic cloning, splitting, pruning, and opacity resets—with stochastic-gradient Langevin dynamics (SGLD)-inspired optimization, opacity- and scale-sparsity regularization, probability-guided sample insertion, and relocalization of inactive Gaussians. The method was introduced in “3D Gaussian Splatting as Markov Chain Monte Carlo” (Kheradmand et al., 2024). The designation MCMC describes a sampling formulation and interpretation; the practical algorithm, which uses Adam, changing population size, scheduled transitions, regularization, and no Metropolis–Hastings acceptance step, is not an exact textbook stationary MCMC sampler with a formal convergence guarantee.

1. Representation and motivation

A 3DGS scene consists of anisotropic Gaussian primitives

gi=(μi,Σi,oi,ci),g_i=(\boldsymbol{\mu}_i,\boldsymbol{\Sigma}_i,o_i,\mathbf{c}_i),

where μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^3 is the center, Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3} is the covariance, oio_i is opacity, and ci\mathbf{c}_i is a view-dependent color representation, commonly expressed through spherical-harmonic coefficients. For a camera view, the Gaussians are projected into screen space and sorted by depth. The rendered color at pixel xx is

C(x)=∑i=1Nciαi(x)∏j=1i−1(1−αj(x)),C(x)=\sum_{i=1}^{N}\mathbf{c}_i\alpha_i(x) \prod_{j=1}^{i-1}(1-\alpha_j(x)),

where the projected opacity contribution is approximately

αi(x)=oiexp⁡[−12(x−Jμi)⊤(JΣiJ⊤)−1(x−Jμi)].\alpha_i(x)=o_i\exp\left[ -\frac{1}{2} (x-\mathbf{J}\boldsymbol{\mu}_i)^\top (\mathbf{J}\boldsymbol{\Sigma}_i\mathbf{J}^\top)^{-1} (x-\mathbf{J}\boldsymbol{\mu}_i) \right].

Here J\mathbf{J} denotes the relevant camera projection Jacobian. The differentiable rasterizer can be abstracted as

C=R({gi}i=1N).C=R(\{g_i\}_{i=1}^{N}).

The conventional 3DGS objective combines an μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^30 image loss and D-SSIM:

μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^31

with μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^32. Gaussian parameters are ordinarily optimized by gradient descent or an Adam-preconditioned equivalent.

Conventional 3DGS uses adaptive density control based on gradient magnitudes, projected and world-space sizes, opacity, and manually selected thresholds. Gaussians may be cloned, split, pruned, or subjected to periodic opacity resets. This procedure has several consequences:

  • the final Gaussian count is difficult to predict from nominal hyperparameters;
  • poor point-cloud initialization can lead to poor Gaussian placement;
  • cloning and splitting rules can behave poorly across scenes with different geometry, texture, or initialization;
  • opacity-based pruning can remove primitives that later become useful;
  • memory and rendering budgets are difficult to control directly.

3DGS-MCMC treats density control as population management. Instead of considering Gaussian primitives solely as rendering elements, it regards them as sample locations moving through parameter space and representing an implicit scene distribution.

2. Sampling interpretation and objective

Under the sampling view, a sufficiently large Gaussian population is regarded as samples from an underlying distribution μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^33:

μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^34

Rendering is consequently conceptualized as

μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^35

The distribution is not explicitly parameterized. Gaussian parameters themselves are treated as samples that move under stochastic updates. If the target distribution is μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^36, a generic SGLD update has the form

μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^37

where μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^38 controls the drift, μi∈R3\boldsymbol{\mu}_i\in\mathbb{R}^39 controls exploration, and

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}0

When the target is defined implicitly through the total rendering loss,

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}1

the score becomes

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}2

yielding the SGLD-like update

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}3

The gradient term moves Gaussians toward configurations that improve reconstruction, while the stochastic term enables exploration of alternative configurations. Vanilla 3DGS corresponds approximately to the zero-noise case: it retains the deterministic drift but lacks explicit exploration.

3DGS-MCMC adds sparsity regularization to the conventional image objective:

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}4

The opacity penalty encourages unnecessary Gaussians to become low-opacity. The scale penalty acts on the standard deviations along the covariance principal axes and encourages small-scale Gaussians. An inactive Gaussian therefore tends to become low-opacity, small-scale, visually ineffective, and suitable for removal or reallocation.

The regularizer is coupled to the population dynamics. Adding the same regularizer to conventional 3DGS degraded performance in the reported ablation, whereas it improved the complete 3DGS-MCMC procedure. This indicates that the opacity and scale penalties are not independent drop-in improvements; their effect depends on stochastic exploration, sample insertion, and relocalization.

3. SGLD-inspired optimization

The practical optimizer uses Adam for the deterministic drift and adds stochastic perturbations primarily to Gaussian centers:

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}5

The final implementation adds noise to position but not to opacity, scale, orientation, or color. Position noise is covariance- and opacity-dependent:

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}6

where

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}7

and

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}8

The sigmoid creates a transition near the conventional opacity-pruning threshold. High-opacity Gaussians receive approximately suppressed noise, while low-opacity Gaussians receive stronger exploratory motion. Covariance determines the spatial scale of the perturbation: narrow Gaussians receive smaller spatial perturbations than broad Gaussians.

The covariance dependence is motivated by the differing spatial support of Gaussians. A fixed perturbation could move a narrow Gaussian outside a region from which it could recover, whereas covariance-scaled noise better matches the Gaussian’s spatial extent.

The practical method is SGLD-inspired rather than an exact fixed-target Langevin chain. Adam changes the effective drift metric; learning rates are scheduled; the Gaussian population changes dimension; relocalization is a nonlocal state transition; regularization changes the objective; and no Metropolis–Hastings acceptance or rejection step is used. The method therefore does not establish that the optimized Gaussians are independent samples from an invariant distribution.

4. Sample addition and relocalization

Sample addition

In conventional 3DGS, densification is triggered by hand-designed conditions involving accumulated image-space gradients, projected size, world-space scale, opacity, and pruning thresholds. In 3DGS-MCMC, adding a Gaussian is interpreted as increasing the sample population.

An existing Gaussian is selected with probability related to opacity:

Σi∈R3×3\boldsymbol{\Sigma}_i\in\mathbb{R}^{3\times3}9

followed by a categorical draw over Gaussian indices. A new small Gaussian is created at the selected Gaussian’s location, assigned a default opacity, and subsequently separated and refined by optimization. The procedure therefore changes the interpretation of densification from “clone when a gradient crosses a threshold” to “draw another sample near an important existing region.”

The number of newly added Gaussians is limited to at most oio_i0 of the current population per insertion event. This limits abrupt population changes and provides gradual progression toward a predefined Gaussian budget.

Relocalization of inactive Gaussians

A Gaussian with opacity

oio_i1

is effectively invisible and receives negligible rendering gradient. Rather than permanently pruning it, 3DGS-MCMC relocalizes its parameter slot. The operation is performed every oio_i2 iterations:

  1. a high-opacity Gaussian is selected using opacity-weighted sampling;
  2. useful parameters are copied to the inactive Gaussian;
  3. the copied Gaussian is assigned lower opacity;
  4. the optimizer state associated with the original Gaussian is reset.

This operation preserves the number of parameter slots while moving inactive capacity toward a region already contributing to the scene. It is analogous to teleporting an inactive walker toward a higher-probability region.

The method’s conceptual probability argument is that duplicating a source sample while dividing its contribution among the resulting samples can approximately preserve aggregate probability mass. However, the final implementation uses the simpler copy-and-lower-opacity rule rather than the earlier mass-preserving resampling formula. Probability preservation is therefore approximate and interpretive, not an exact property of the implemented transition.

5. Training procedure and population control

3DGS-MCMC supports initialization from SfM or LiDAR points and random initialization. For random initialization, approximately oio_i3 Gaussians are sampled uniformly within a region extending to three times the camera-bounding-box extent.

A typical optimization cycle is:

  1. render training views using the current Gaussian population;
  2. compute the image loss and opacity/scale regularization;
  3. backpropagate through the differentiable rasterizer;
  4. apply Adam-preconditioned updates;
  5. apply covariance- and opacity-dependent stochastic center perturbations;
  6. every oio_i4 iterations, add opacity-guided samples subject to the oio_i5 population-increase limit;
  7. every oio_i6 iterations, relocalize Gaussians with oio_i7;
  8. continue until the prescribed Gaussian budget is reached.

The position learning rate begins at

oio_i8

and exponentially decays to

oio_i9

The other relevant settings follow the original 3DGS learning-rate schedule. The population is gradually increased toward a predefined maximum rather than allowed to grow through uncontrolled heuristic cloning.

The intended distribution is often written as

ci\mathbf{c}_i0

Low-loss configurations have higher probability, and stochastic motion is intended to prevent premature collapse into a single local solution. In practice, however, the target distribution is implicit, nonconvex, finite-view, and dynamically modified by regularization, insertion, and relocalization. The method should therefore be understood as an optimization-and-sampling framework rather than a theorem-backed asymptotic sampler.

6. Empirical behavior and subsequent developments

The original evaluation used scenes from the NeRF dataset, Mip-NeRF 360, Tanks and Temples, and Deep Blending, with PSNR, SSIM, and LPIPS as the principal metrics. Reported random-initialization averages include:

Dataset 3DGS PSNR 3DGS-MCMC PSNR 3DGS-MCMC LPIPS
NeRF scenes 33.42 33.85 0.04
Mip-NeRF 360 27.89 29.58 0.19
Tanks and Temples 21.93 24.47 0.19
Deep Blending 29.55 29.66 0.31

The strongest improvements occur with random initialization, supporting the claim that stochastic exploration reduces dependence on a good initial point set. With point-cloud initialization, improvements are more modest but remain positive in the reported averages.

Subsequent work has extended or modified the population and optimization mechanisms:

  • Metropolis–Hastings 3DGS replaces opacity-guided population operations with posterior-inspired birth proposals, multi-view error scores, voxel sparsity, coarse-to-fine position proposals, and probabilistic acceptance (Kim et al., 15 Jun 2025). Unlike 3DGS-MCMC, it adaptively changes the Gaussian count through proposed births and rejection. Its practical acceptance rule is itself an approximation rather than an exact reversible birth-death sampler.
  • AdamW-GS decomposes optimization into Sparse Adam, Re-State Regularization, and Decoupled Attribute Regularization (Ding et al., 23 Jan 2026). Sparse Adam avoids updates and moment decay for invisible Gaussians; Re-State Regularization explicitly rescales optimizer states; and decoupled regularization prevents opacity and scale gradients from contaminating photometric Adam moments. In the reported Mip-NeRF 360 experiments, AdamW-GS improved the original MCMC aggregate from ci\mathbf{c}_i1 to ci\mathbf{c}_i2 PSNR, from ci\mathbf{c}_i3 to ci\mathbf{c}_i4 SSIM, and reduced runtime from ci\mathbf{c}_i5 to ci\mathbf{c}_i6 minutes.
  • Post-training dictionary learning compresses the spherical-harmonic appearance parameters of a trained 3DGS-MCMC model without changing its Gaussian population or MCMC training process (Gong et al., 28 May 2026). At the reported operating point, the method achieved a ci\mathbf{c}_i7 compression ratio, increased rendering speed from ci\mathbf{c}_i8 to ci\mathbf{c}_i9 FPS, and produced approximately xx0 dB mean PSNR degradation against ground truth.
  • SalientGS retains the SGLD dynamics of 3DGS-MCMC while guiding birth and relocation using multi-view underfit and redundancy scores (Xiong et al., 13 Jul 2026). It is explicitly described as a heuristic allocation layer rather than a Metropolis–Hastings proposal. On Mip-NeRF 360, the reported result was xx1 PSNR, compared with xx2 for 3DGS-MCMC, using a xx3M-Gaussian budget.
  • Confidence-aware densification replaces opacity-driven source sampling and relocation with image-error-driven signals, learned per-primitive confidence, structural error, and rendered contribution (Hofer et al., 9 Oct 2026). It retains random or prior-free initialization, gradual growth, a fixed primitive budget, and relocation. Averaged over 24 scenes, it outperformed 3DGS-MCMC in PSNR, SSIM, and LPIPS at budgets from xx4 to xx5 primitives; at xx6, the reported PSNR improvement over 3DGS-MCMC was xx7 dB.

These developments expose a central design tension. Opacity is computationally convenient and correlates with rendered contribution in many regions, but it can reinforce foreground allocation and neglect poorly represented backgrounds. Error-driven and confidence-aware methods attempt to allocate capacity according to reconstruction deficiency rather than existing opacity. Conversely, stronger error guidance introduces additional schedules, residual computations, and hyperparameters.

7. Scope, limitations, and interpretation

3DGS-MCMC’s principal contribution is the reinterpretation of adaptive density control as stochastic population management. Its components have distinct roles:

  • gradient-based drift improves rendering configurations;
  • position noise enables exploration;
  • opacity-guided insertion allocates samples to currently important regions;
  • opacity and scale regularization promotes sparsification;
  • relocalization recycles inactive Gaussian slots;
  • an explicit population budget improves memory and rendering predictability.

The following claims are empirically supported within the reported experiments: 3DGS-MCMC can improve rendering quality at comparable Gaussian counts, is more robust to random initialization than conventional 3DGS, benefits from explicit stochastic exploration, provides direct population control, and reduces reliance on the original clone/split/prune/reset framework.

The following properties are not formally established for the practical algorithm:

  • exact sampling from xx8;
  • exact preservation of probability under relocalization;
  • reversibility of the opacity-dependent noise schedule;
  • formal convergence of Adam-based dynamics with changing population size;
  • independence of the resulting Gaussian states.

The method is therefore best characterized as a principled sampling formulation integrated with differentiable rendering and adaptive population control. Its significance lies less in introducing a new rasterizer or Gaussian parameterization than in replacing heuristic density control with a framework that combines stochastic exploration, sparsity-driven inactivity, probabilistic sample allocation, and Gaussian-slot recycling.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to 3DGS-MCMC.