3DGS-MCMC Optimization Overview

• The paper introduces 3DGS-MCMC optimization, which formulates scene representation via anisotropic 3D Gaussians and leverages stochastic updates with regularized primitive birth and relocation.
• It employs AW-SGLD for global search and LQNAdam for local curvature-aware refinement, resulting in improved performance metrics such as PSNR, SSIM, and reduced active primitives.
• The approach decouples gradient updates using Sparse Adam and DAR while integrating post-training dictionary-learning compression to enhance rendering speed and memory efficiency.

3DGS-MCMC optimization denotes the family of training and deployment procedures that treat 3D Gaussian Splatting (3DGS) not only as an explicit radiance-field parameterization but also as an optimization problem with stochastic exploration, regularized primitive birth and relocation, and increasingly specialized optimizer design. In this setting, a scene is represented by anisotropic 3D Gaussians with position, covariance, opacity, and spherical-harmonic color parameters, while training combines photometric reconstruction with opacity and scale control. The 3DGS-MCMC variant augments 3DGS with exploration and respawning under an SGLD-inspired framework, and later work extends this line with adaptive exploration and curvature-guided exploitation in Opt3DGS (Huang et al., 17 Nov 2025), with visibility-aware and regularization-decoupled optimization in AdamW-GS (Ding et al., 23 Jan 2026), and with post-training dictionary-learning compression for faster deployment of trained 3DGS-MCMC models (Gong et al., 28 May 2026).

1. Core formulation and optimization objective

In 3DGS, a scene is modeled as a set of $N$ explicit Gaussian primitives. In the Opt3DGS formulation, each primitive $i$ has position $\mu_i \in \mathbb{R}^3$, covariance $\Sigma_i = R_i s_i s_i^T R_i^T$ where $s_i \in \mathbb{R}^3$ is axis-aligned scale and $R_i \in SO(3)$ is rotation, opacity $o_i \in [0,1]$, and color $c_i$ computed by spherical harmonics coefficients. Under splatting and front-to-back alpha compositing, the pixel color at screen location $x$ is

$c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$

with

$i$0

where $i$1 is the accumulated transmittance up to the $i$2-th Gaussian in the sorted front-to-back list intersecting pixel $i$3 (Huang et al., 17 Nov 2025).

The global parameter set is

$i$4

Opt3DGS builds on the 3DGSMCMC loss with $i$5/SSIM fidelity terms and regularization on opacity and covariance scales:

$i$6

Here, $i$7 is an $i$8 image reconstruction loss over pixels, $i$9 is a structural similarity term, and $\mu_i \in \mathbb{R}^3$0 are regularization weights. In the MCMC interpretation used by Opt3DGS, this $\mu_i \in \mathbb{R}^3$1 acts as the energy function for stochastic updates (Huang et al., 17 Nov 2025).

A parallel description in the decoupling analysis writes the per-primitive attributes as $\mu_i \in \mathbb{R}^3$2 and the training objective as

$\mu_i \in \mathbb{R}^3$3

That work characterizes the standard pipeline as Warm-up $\mu_i \in \mathbb{R}^3$4 densification (clone/split; prune low-opacity) $\mu_i \in \mathbb{R}^3$5 pure optimization (P-Op), with gradients propagated to all attributes and standard Adam as the default optimizer (Ding et al., 23 Jan 2026).

Within 3DGS-MCMC, optimization still uses Adam and the same photometric and regularization losses, but exploration and respawning are added through an SGLD-inspired framework. Densification samples new primitives and reallocates “dead” ones, and a noise-driven position regularization encourages exploration. The regularizers, especially opacity and scale, play a larger role in activating respawn and reducing redundancy (Ding et al., 23 Jan 2026).

2. Stochastic exploration and adaptive posterior flattening

The exploration stage in Opt3DGS is framed through Langevin dynamics. For sampling from a target density $\mu_i \in \mathbb{R}^3$6, the overdamped Langevin SDE is

$\mu_i \in \mathbb{R}^3$7

During exploration, Opt3DGS defines

$\mu_i \in \mathbb{R}^3$8

so that $\mu_i \in \mathbb{R}^3$9. With a mini-batch stochastic gradient and stepsize $\Sigma_i = R_i s_i s_i^T R_i^T$0, the discretized SGLD update is

$\Sigma_i = R_i s_i s_i^T R_i^T$1

In practice for 3DGS, the mini-batch estimator is obtained by sampling images and rays or pixels, splatting visibilities, accumulating transmittance for the intersecting Gaussians, backpropagating through compositing, and aggregating parameter gradients across pixels. Opt3DGS uses Adam’s stabilized gradient estimates in the drift term to improve robustness, same as in 3DGSMCMC (Huang et al., 17 Nov 2025).

The practical noise injection mechanism inherited from 3DGSMCMC is position-focused. In that formulation,

$\Sigma_i = R_i s_i s_i^T R_i^T$2

with $\Sigma_i = R_i s_i s_i^T R_i^T$3, and $\Sigma_i = R_i s_i s_i^T R_i^T$4 using a scene/opacity-dependent scaling. Opt3DGS retains this mechanism but modifies the drift via adaptive weighting (Huang et al., 17 Nov 2025).

Its central exploration mechanism is Adaptive Weighted SGLD (AW-SGLD). To reduce energy barriers and encourage mode transitions, Opt3DGS constructs a flattened target density

$\Sigma_i = R_i s_i s_i^T R_i^T$5

where $\Sigma_i = R_i s_i s_i^T R_i^T$6, $\Sigma_i = R_i s_i s_i^T R_i^T$7 controls flattening strength, and $\Sigma_i = R_i s_i s_i^T R_i^T$8 is a smooth weighting function defined on energy levels. The energy range is partitioned into $\Sigma_i = R_i s_i s_i^T R_i^T$9 bins, a weight vector $s_i \in \mathbb{R}^3$0 is maintained and adapted via stochastic approximation, and a smooth piecewise exponential interpolation avoids vanishing gradients from piecewise constants (Huang et al., 17 Nov 2025).

The resulting log-density gradient introduces a multiplicative drift factor $s_i \in \mathbb{R}^3$1, yielding the AW-SGLD update

$s_i \in \mathbb{R}^3$2

In compact preconditioned form,

$s_i \in \mathbb{R}^3$3

The online weight adaptation targets equal visitation across energy bins through a flat-histogram stochastic approximation scheme. In the paper’s interpretation, this gradually estimates a “density-of-states” style compensation that flattens the effective energy barriers (Huang et al., 17 Nov 2025).

The reported exploration schedule uses 30,000 total iterations, a warm-up of 2,500 iterations before activating adaptive weighting $s_i \in \mathbb{R}^3$4, $s_i \in \mathbb{R}^3$5 bins on $s_i \in \mathbb{R}^3$6 for most scenes, $s_i \in \mathbb{R}^3$7 for Tanks and Temples “train,” a fixed flattening coefficient $s_i \in \mathbb{R}^3$8, a Gaussian growth rate fixed at 5%, and noise injection focused on $s_i \in \mathbb{R}^3$9. The stated intuition is that AW-SGLD effectively performs a tempered walk that reduces energy barriers between modes, combats opacity-driven sampling bias, and addresses the clustering effect observed in 3DGSMCMC (Huang et al., 17 Nov 2025).

3. Curvature-aware exploitation and two-stage refinement

Opt3DGS divides optimization into adaptive exploration followed by curvature-aware exploitation. The second stage replaces global-search behavior with a local refinement procedure called Local Quasi-Newton Direction-guided Adam (LQNAdam). The Adam baseline is written as

$R_i \in SO(3)$0

$R_i \in SO(3)$1

Opt3DGS then estimates local curvature for each Gaussian’s position $R_i \in SO(3)$2 independently using limited-memory BFGS with history size $R_i \in SO(3)$3, with $R_i \in SO(3)$4 in experiments (Huang et al., 17 Nov 2025).

Let $R_i \in SO(3)$5 denote the approximate inverse Hessian for $R_i \in SO(3)$6, represented implicitly by L-BFGS two-loop recursion. The local direction is

$R_i \in SO(3)$7

where $R_i \in SO(3)$8 is the gradient of $R_i \in SO(3)$9 with respect to $o_i \in [0,1]$0. Rather than a line search, Opt3DGS treats this quasi-Newton direction as a pseudo-gradient and feeds it to Adam:

$o_i \in [0,1]$1

$o_i \in [0,1]$2

with $o_i \in [0,1]$3. This is the LQNAdam update (Huang et al., 17 Nov 2025).

The exploitation stage is triggered near the end of training, for example at iteration 29,000 of 30,000. At that point, $o_i \in [0,1]$4 is replaced by $o_i \in [0,1]$5 for finer convergence, the flattening multiplier $o_i \in [0,1]$6 is disabled to return to the original posterior, quasi-Newton directions are computed per Gaussian’s $o_i \in [0,1]$7 in parallel on CUDA, and Adam continues on other parameters with standard gradients (Huang et al., 17 Nov 2025).

This local curvature model is deliberately restricted. L-BFGS provides an implicit diagonal-block inverse-Hessian approximation per Gaussian position, requiring only storage of $o_i \in [0,1]$8 pairs of $o_i \in [0,1]$9. No explicit Hessian is formed, and updates use the two-loop recursion at $c_i$0 per Gaussian. The paper states that this choice leverages weak coupling among Gaussians and keeps computation feasible. Reported overhead is small: AW-SGLD adds negligible $c_i$1 per iteration for $c_i$2 and $c_i$3, while LQNAdam adds per-Gaussian L-BFGS recursion and maintaining $c_i$4 history pairs, with time increases of approximately $c_i$5–$c_i$6 minute in the reported cases (Huang et al., 17 Nov 2025).

4. Decoupling visibility, moments, and regularization

A distinct line of work argues that standard 3DGS and 3DGS-MCMC optimization suffer from two overlooked couplings: update-step coupling and gradient coupling in the moment. With synchronous Adam, all primitives are processed every iteration regardless of visibility. Even if a primitive is invisible and its instantaneous gradient is zero, Adam’s moments are still updated by decay, which rescales optimizer state and can trigger implicit attribute updates later. The decoupling analysis states that this implicit update is wasteful, updates invisible Gaussians, and can bias regularization activation (Ding et al., 23 Jan 2026).

The reported evidence includes the claim that, under vanilla 3DGS with Adam, many primitives outside current views still undergo state changes and subsequent updates. This produces more dead primitives, for example “GS1: Nd = 0.232M,” than Sparse Adam, “GS3: 0.048M,” and incurs non-trivial step-time cost. In 3DGS-MCMC, Adam’s second moment for opacity is reported to be systematically smaller than Sparse Adam’s, so $c_i$7 can be larger, making regularization more active (Ding et al., 23 Jan 2026).

The proposed remedy begins with Sparse Adam. For each primitive, a binary visibility mask $c_i$8 modifies moment decay so that invisible primitives experience no decay and no accumulation:

$c_i$9

$x$0

The parameter update is similarly masked,

$x$1

The stated benefits are elimination of implicit updates, stabilization across viewpoints, and an approximately 50% reduction in step time in 3DGS-MCMC, with the stated drawback that Sparse Adam alone tends to be less explorative (Ding et al., 23 Jan 2026).

The second component is Re-State Regularization (RSR), an explicit periodic moment reconditioning:

$x$2

RSR is applied to a uniformly sampled subset of primitives at fixed intervals under the State Sampling Schedule. The paper gives the safe default $x$3 and states that $x$4 works well across scenes. Its rationale is that moment rescaling is the mechanism by which Adam’s implicit updates strengthened regularization, so RSR reintroduces this control in a transparent and scene-tunable way (Ding et al., 23 Jan 2026).

The third component is Decoupled Attribute Regularization (DAR). Here the moments are updated with photometric gradients only, while regularization is applied separately in an AdamW-style but geometry-aware form:

$x$5

$x$6

$x$7

Opacity and scale receive explicit per-attribute forms, while position $x$8 uses noise-driven exploration regularization gated by opacity and shaped by $x$9. The paper’s motivations are that decoupling removes the instability caused by Adam’s gradient coupling, makes per-step regularization strength explicitly controlled, and uses $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$0 scaling to make regularization geometry-aware (Ding et al., 23 Jan 2026).

These three components are recomposed into AdamW-GS: visibility-aware Sparse Adam, DAR for opacity and scale, periodic RSR via StSS, and optional position noise regularization for outdoor scenes. The paper states that AdamW-GS removes gradient coupling, removes implicit updates, and reintroduces controlled moment rescaling, while remaining crucially visibility-aware because of physical attribute meanings and viewpoint sparsity in 3DGS and 3DGS-MCMC (Ding et al., 23 Jan 2026).

5. Empirical behavior, benchmark trends, and failure modes

Opt3DGS reports evaluation on MipNeRF360, Tanks and Temples, and DeepBlending with consistent settings. The reported headline numbers are: MipNeRF360 with PSNR 29.96, SSIM 0.897, LPIPS 0.143 versus 3DGSMCMC 29.89, 0.900, 0.190; Tanks and Temples with PSNR 24.80, SSIM 0.875, LPIPS 0.139 versus 24.29, 0.860, 0.190; and DeepBlending with PSNR 30.09, SSIM 0.911, LPIPS 0.229 versus 29.67, 0.900, 0.320. The paper also states that random initialization shows Opt3DGS outperforms 3DGSMCMC and 3DGS across all metrics, indicating stronger ability to recover from poor starts (Huang et al., 17 Nov 2025).

Its ablations on Tanks and Temples further isolate the two stages. For Train, baseline 3DGSMCMC reports PSNR 22.47, SSIM 0.830, LPIPS 0.240, Time 11m; adding AW-SGLD gives 22.74, 0.841, 0.180, Time 12m; adding LQNAdam gives 23.01, 0.846, 0.176, Time 12m. For Truck, the sequence is 26.11/0.890/0.140 to 26.49/0.901/0.104 to 26.61/0.903/0.102, with time 22 to 22 to 23m. A separate ablation indicates best $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$1 near 0.8, while the main setting uses $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$2 (Huang et al., 17 Nov 2025).

AdamW-GS reports a complementary set of improvements on MipNeRF360. For 3DGS-MCMC, baseline Adam reports PSNR 27.948, SSIM 0.833, LPIPS 0.199; adding Sparse Adam gives 27.998/0.832/0.199 with +4.28% active primitives; DAR variants report 28.185/0.839/0.182 and 28.219/0.840/0.182 with +4.5% active primitives and 39.77 mins total time versus 46.81. For vanilla 3DGS, baseline Adam reports PSNR 27.506, SSIM 0.815, LPIPS 0.216 with 3.098M active primitives, while AdamW-GS reports PSNR 27.678, SSIM 0.822, LPIPS 0.220 with −49.3% active primitives and runtime 18.53 mins versus 30.58 (Ding et al., 23 Jan 2026).

The same study reports that Deep Blending and Tanks & Temples also improve in PSNR and SSIM while reducing primitives; for example, Deep Blending 3DGS + AdamW-GS reports PSNR 30.260 versus 29.694 baseline. On OMMO long sequences, 3DGS-MCMC + AdamW-GS reports PSNR 30.716 versus 30.359, while 3DGS + AdamW-GS reduces active primitives from 1.640M to 1.211M with slight PSNR gain (Ding et al., 23 Jan 2026).

Several limitations recur across these methods. Opt3DGS lists over-flattening when $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$3 is too large, parameter delicacy when noise is injected into opacity $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$4 or scales $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$5, curvature instability when L-BFGS directions are noisy, and energy-bin miscalibration when bounds or bin counts are inappropriate. It recommends $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$6, warm-up of $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$7, restriction of noise to $c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$8, and 200 bins with tuned intervals (Huang et al., 17 Nov 2025). AdamW-GS lists overfitting or early stopping, boundary artifacts with poor view coverage, background blurriness, geometric inconsistency, position noise degrading indoor scenes with few primitives, and the hand-crafted nature of the StSS schedule (Ding et al., 23 Jan 2026).

A common misconception is to equate 3DGS-MCMC exclusively with explicit Metropolis–Hastings accept-reject steps. The decoupling analysis states that an MCMC-style view is used to analyze proposals and acceptance behavior conceptually, and that most steps are realized as gradient updates and respawn operations rather than explicit accept-reject moves (Ding et al., 23 Jan 2026). By contrast, the deployment-oriented compression work describes the MCMC variant as performing Markov chain Monte Carlo over scene parameters through birth, death, split, merge, propose, and accept moves, yielding posterior-consistent Gaussian sets with strong fidelity and stability (Gong et al., 28 May 2026). Taken together, these descriptions indicate that “MCMC” in this literature spans both a stochastic-optimization interpretation and a more literal proposal-based reading, depending on the layer of analysis.

6. Deployment-time optimization and post-training compression

A further optimization axis concerns the trained model at deployment rather than the training dynamics. The dictionary-learning compression framework for 3DGS compresses the dominant storage cost in spherical-harmonic color parameters and applies directly to 3DGS-MCMC without retraining or modifying its MCMC process (Gong et al., 28 May 2026).

For degree-$c(x) = \sum_{i=1}^N c_i \cdot o_i \cdot T_i,$9 spherical harmonics, each Gaussian stores 48 SH parameters: 16 coefficients per channel across 3 channels. The compression leaves the 3 DC coefficients unchanged and replaces the 45 view-dependent SH coefficients with a shared dictionary and sparse codes. The representation becomes a shared dictionary $i$00, for example $i$01, together with per-Gaussian sparse codes $i$02 satisfying $i$03 so that $i$04, where $i$05 is the original 45-dimensional non-DC vector (Gong et al., 28 May 2026).

The dictionary-learning objective is the standard $i$06-constrained sparse coding problem,

$i$07

with an $i$08-regularized variant also given. Sparse coding is performed with Orthogonal Matching Pursuit, dictionary update can use K-SVD, MOD, or online dictionary learning, and the paper reports a tolerance sweep $i$09, identifying $i$10 as a good operating point for 3DGS-MCMC (Gong et al., 28 May 2026).

At render time, the original non-DC color evaluation $i$11 becomes

$i$12

The sparse codes are stored in Compressed Sparse Column format, while the dictionary is stored once, for example as a 45×90 FP32 tensor of approximately 15.8 KB. The paper reports that baseline degree-3 SH reads 48 SH floats or 192 B per Gaussian and performs about 96 FLOPs for non-DC color evaluation, whereas the compressed variant reads approximately $i$13 B per Gaussian plus the shared dictionary; for $i$14 at $i$15, this is about 110 B per Gaussian, with higher FLOPs but lower memory traffic. The stated conclusion is that radiance evaluation is memory-bound on modern GPUs, so reducing bytes per Gaussian outweighs the added compute (Gong et al., 28 May 2026).

For 3DGS-MCMC specifically, the paper states that only the 45 view-dependent SH coefficients are replaced; positions $i$16, covariances $i$17, and opacity $i$18 are untouched. No MCMC-specific state, proposals, or acceptance stats are stored or needed at deployment. Empirically, averaged over 13 scenes with OMP tolerance $i$19, the reported results are a compression ratio of 3.10×, rendering speed increase of +24.3% from 113.0 FPS to 140.5 FPS, and similarity to original MCMC renderings of PSNR 42.95 dB, SSIM 0.9961, and LPIPS 0.0076. Against ground truth, the average PSNR drop of compressed versus original baselines is reported as marginal, approximately 0.14–0.19 dB across methods (Gong et al., 28 May 2026).

This deployment result suggests a separation between training-time optimization and deployment-time optimization. Training-time work such as AW-SGLD, LQNAdam, Sparse Adam, RSR, and DAR alters how Gaussians are explored, regularized, or refined; post-training dictionary learning instead optimizes the memory-bandwidth profile of the converged Gaussian set while preserving the geometry and coverage determined by the trained 3DGS-MCMC model.