---
title: 'Splannequin: Frozen 3D Scene Synthesis'
url: https://www.emergentmind.com/topics/splannequin
type: topic
---

# Splannequin: Frozen 3D Scene Synthesis

Splannequin is a regularization framework designed for synthesizing high-fidelity frozen 3D scenes from monocular Mannequin-Challenge (MC) videos, allowing user-controlled selection of “frozen” timestamps with minimal artifacts. Unlike standard dynamic scene reconstruction methods that focus on accurate motion modeling, Splannequin explicitly addresses the “freeze-time” challenge: rendering artifact-free static scenes from dynamic, sparsely supervised monocular sequences, where common approaches suffer from ghosting and blur due to ill-supervised 3D primitives. Splannequin introduces a dual-detection anchoring method that detects two ill-posed states of 3D Gaussians and applies temporally anchored regularization, integrating seamlessly into any dynamic Gaussian splatting pipeline with zero inference overhead [2512.05113].

## 1. Problem Setup: Freezing Monocular Mannequin-Challenge Footage

MC videos consist of casual, single-camera recordings in which actors strive to remain stationary, but often slight micro-motions persist. The task is, given $N$ training images $\{(I_n,R_n,b_n,t_n)\}$—where $I_n$ is the input image, $R_n$ the camera matrix, $b_n$ any additional metadata, and $t_n$ the timestamp—to generate bullet-time sequences $\{\hat I(R,b,t^\star)\}$ at arbitrary, user-selected timestamps $t^\star$. A core difficulty arises because dynamic 3D scene reconstruction approaches are inherently trained along the (typically) diagonal trajectory formed by the camera path in space–time, while “freeze-time” rendering queries require slicing the learned representation horizontally at $t = t^\star$. As a result, many Gaussians are never or only weakly supervised at the desired $t^\star$, producing ghosting and blurred artifacts in conventional methods [2512.05113].

## 2. Dynamic Gaussian Splatting for Video and Naive Freeze-Time Rendering

Dynamic Gaussian splatting models scenes with a set of $K$ canonical 3D Gaussians $\{G_k\}$, each parameterized by a static mean $\boldsymbol{\mu}_k$ and covariance $\Sigma_k$. Temporal variation is captured using a deformation MLP $f_\theta$, such that for any time $t$:

$$(\Delta\mu_{k,t}, \Delta\Sigma_{k,t}) = f_\theta(\mu_k, t)$$

and the time-dependent primitive is

$$G_k(t) = (\mu_k + \Delta\mu_{k,t}, \Sigma_k + \Delta\Sigma_{k,t})$$

Training minimizes a photometric reconstruction loss,

$$\mathcal L_{\rm recon} = \sum_{n=1}^N \ell\bigl(\hat I(R_n, b_n, t_n), I_n\bigr)$$

where $\hat I$ is rendered via differentiable rasterization of the current set of deformed Gaussians at $t_n$. At inference, naive freeze-time rendering substitutes $t = t^\star$; however, since many Gaussians are either unobserved or poorly supervised at $t^\star$ (due to occlusions, frustum exclusion, or camera path sparsity), this leads to artifacts such as floating, blurred, or ghost blobs [2512.05113].

## 3. Ill-Supervised Gaussian Detection: Hidden and Defective States

Splannequin introduces automatic detection of two ill-supervised Gaussian states at each training time $t$:

- **Hidden State**: A Gaussian $G_k(t)$ is “hidden” if its projected center is outside the camera frustum at $t$. Formally,
  $$
  s_{\rm hidden}(k,t) = 
    \begin{cases}
      1, & \text{if projected center of $G_k(t)$ is out of frustum} \\
      0, & \text{otherwise}
    \end{cases}
  $$
- **Defective State**: A Gaussian $G_k(t)$ is “defective” if it is inside the frustum but receives negligible photometric supervision, measured by the per-primitive gradient norm:
  $$
  s_{\rm defective}(k,t)=
    \begin{cases}
      1, & \|\partial \mathcal L_{\rm recon} / \partial \psi_k(t)\| \leq \epsilon\ (\epsilon \approx 10^{-9}) \\
      0, & \text{otherwise}
    \end{cases}
  $$
A Gaussian is “well-supervised” only if both indicators are zero. These detector criteria allow systematic identification of Gaussians prone to drift at freeze-time surfaces [2512.05113].

## 4. Temporally Anchored Regularization Loss

When hidden or defective Gaussians are detected, Splannequin applies a temporally anchored regularization. Let $\psi_k(t)$ denote the full parameter vector of Gaussian $k$ at time $t$ (position, covariance, opacity, and spherical harmonic coefficients). For a specified discrepancy measure $\mathcal D$ (L1 or L2 distance variants),

$$
\mathcal D(\psi_k(t), \psi_k(t_{\rm ref}))=
\begin{cases}
  \|\psi_k(t)-\psi_k(t_{\rm ref})\|_1 & \text{(L1)} \\
  \|\psi_k(t)-\psi_k(t_{\rm ref})\|_2^2 & \text{(L2)}
\end{cases}
$$

with a confidence weight,

$$
\phi(t, t_{\rm ref}) = \exp(-\tau |t - t_{\rm ref}|),\ \tau>0
$$

the anchored losses are constructed as follows:

- **Hidden-Gaussian Anchoring** (for $s_{\rm hidden}(k,t)=1$): Sample a reference time $t_{\rm ref}<t$ where the Gaussian is well-supervised, applying
  $$
  \mathcal L_{\rm hidden}^{(k,t)} = \phi(t, t_{\rm ref})\, \mathcal D\big(\psi_k(t), \psi_k(t_{\rm ref})\big)
  $$
- **Defective-Gaussian Anchoring** (for $s_{\rm defective}(k,t)=1$): Sample $t_{\rm ref}>t$ well-supervised, and regularize similarly,
  $$
  \mathcal L_{\rm defective}^{(k,t)} = \phi(t, t_{\rm ref})\, \mathcal D\big(\psi_k(t), \psi_k(t_{\rm ref})\big)
  $$
- **Full Objective**: The combined training loss is
  $$
  \mathcal L = \mathcal L_{\rm recon} + \lambda_{\rm hidden} \sum_{k,t} \mathcal L_{\rm hidden}^{(k,t)} + \lambda_{\rm defective} \sum_{k,t} \mathcal L_{\rm defective}^{(k,t)}
  $$
  with $\lambda_{\rm hidden} = \lambda_{\rm defective} = 10$ used in practice and confidence decay $\tau = 5$.

This approach effectively anchors drift-prone Gaussians to nearby, well-supervised past (hidden) or future (defective) reference states, limiting their freedom to introduce artifacts at poorly supervised timestamps [2512.05113].

## 5. Integration and Implementation Details

Splannequin is architecture-agnostic and incurs zero inference overhead. The dual-detection loss terms are added directly to existing dynamic Gaussian splatting systems, requiring no change to network structure or rendering procedures. During training, at each iteration:

- Two random (view, $t$) pairs are sampled.
- Each Gaussian is classified as hidden/defective/well-supervised.
- For each ill-supervised primitive, a suitable reference time $t_{\rm ref}$ is sampled and the anchor loss computed.

At inference, “freeze-time” rendering remains unchanged: the original dynamic model is queried at $t = t^\star$, yielding high throughput (e.g., 280 FPS on an RTX 4090). The training framework is based on PyTorch and standard progressive densification, with 30,000 iterations per scene. The regularization schedule involves beginning anchored regularization at iteration 10,000 with L2 distance; at iteration 20,000, the loss switches to L1 distance. Losses are computed every 10 iterations using randomly sampled anchors [2512.05113].

## 6. Experimental Evaluation and Quantitative Gains

**Benchmarks:**  
- *Real-world:* 10 MC-style videos (2,869 input frames, 361 freeze-time clips, 640×360 resolution, <10% consistent visibility).
- *Synthetic:* 10 Blender scenes (2,400 frames, 300 freeze-time renders, perfect ground-truth).

**Compared baselines:** 4DGaussians, D-3DGS, SC-GS (with matched hyperparameters).

**Metrics:**  
- *Reference (synthetic):* PSNR (↑), SSIM (↑), LPIPS (↓), FVD (↓).
- *No-reference (real):* CQA (composition), TOPIQ-NR, CLIP-IQA, MUSIQ, HyperIQA, and COVER (semantic, technical, aesthetic, overall).

**Results:**  

| Setting      | Baseline | +Splannequin          |
|--------------|----------|----------------------|
| PSNR         | 28.03    | 28.85                |
| SSIM         | 0.81     | 0.83                 |
| LPIPS        | 0.09     | 0.08                 |
| FVD          | 98.93    | 82.73                |
| CQA          | —        | +26.4%               |
| COVER (overall) | —     | +6.6% (aesthetic +95.6%) |
| D-3DGS+Splannequin COVER (tech) | — | +339.9%   |

In real-world evaluation, Splannequin significantly improves all IQA metrics and dramatically reduces ghosting and blur. A user study with 23 participants reported 96% preference for Splannequin-rendered clips for visual appeal and 80% preference for “more perfectly frozen” scenes. Ablations demonstrate extreme degradations if either the hidden or defective loss terms are removed: e.g., removing hidden loss drops COVER overall by 1072% and CQA by 162%, and removing defective loss drops COVER by 1027% and CQA by 779%. Omitting confidence distance weighting results in over-smoothed frames [2512.05113].

## 7. Significance, Limitations, and Future Directions

Splannequin demonstrates that dual-state anchored regularization for dynamic Gaussian splatting robustly mitigates freeze-time artifacts in monocular MC videos, enabling artifact-free, high-fidelity, user-selectable time slices at arbitrary points, with no architectural or inference penalties. These results extend the practical utility of dynamic Gaussian pipelines to the MC “freezing” regime, previously a source of substantive ghosting artifacts and fidelity loss. The approach’s simplicity and compatibility with any dynamic-GS method suggest broad applicability. 

A plausible implication is that future work could target more challenging scenarios with even sparser temporal supervision, or extend the dual-detection regularization approach to other temporal 3D representations beyond Gaussians. The strong quantitative and user study gains motivate further exploration of adaptive or semantic-guided anchoring, as well as more refined measures of per-Gaussian supervision. 

For comprehensive implementation details, architecture-agnostic integration steps, and source code, see the project page: https://chien90190.github.io/splannequin/ [2512.05113].

Source: https://www.emergentmind.com/topics/splannequin