4C4D: Compositional 4D Scene Modeling

• 4C4D is a paradigm for dynamic 4D scene representation that integrates sparse camera inputs with compositional and correspondence-based techniques.
• It employs innovations like 4D Gaussian splatting, per-object deformations, and dual temporal correspondences to recover 3D geometry and motion.
• The approach enhances efficiency and quality through methods such as Neural Decaying Functions and hybrid diffusion losses, ensuring temporal consistency and realistic rendering.

4C4D refers to a class of methods and paradigms for 4D scene representation, reconstruction, and generation that emphasize compositionality, correspondence, or camera efficiency. Three principal and recently published frameworks—4C4D (4 Camera 4D Gaussian Splatting) (Zhou et al., 5 Apr 2026), Comp4D (“Compositional 4D Generation” encapsulating a four-component pipeline sometimes denoted as “4C4D”) (Xu et al., 2024), and C4D (“4D made from 3D through Dual Correspondences”; also described via four core correspondence steps) (Wang et al., 16 Oct 2025)—define state-of-the-art directions for high-fidelity 4D modeling from sparse input, compositional text-to-4D synthesis, and monocular dynamic reconstruction, respectively. All seek to address the inherent challenges of dynamic scene modeling over time (3D+time) in regimes previously dominated by static, dense-view, or single-object methods.

1. Motivation and Problem Statement

High-fidelity 4D scene modeling is foundational for applications in computer graphics, visual effects, robotics, and dynamic scene understanding. Historically, 4D reconstruction and generation have demanded either dense multiview arrays (Zhou et al., 5 Apr 2026), static scene assumptions (Wang et al., 16 Oct 2025), or monolithic generative models producing limited object diversity (Xu et al., 2024). Key challenges are:

• Recovering temporally consistent 3D geometry and appearance from sparse or monocular video (Zhou et al., 5 Apr 2026, Wang et al., 16 Oct 2025).
• Handling dynamic objects that violate multi-view geometric consistency (Wang et al., 16 Oct 2025).
• Achieving compositional, semantically-reasonable 4D generation from unstructured text prompts (Xu et al., 2024).

At their core, 4C4D-type frameworks advance the field by (a) reducing hardware or data requirements via novel representations and optimization pipelines, (b) leveraging learned decompositions and correspondences to address dynamic content, and (c) exploiting compositionality—whether via network design, optimization, or hybrid procedural/statistical guidance.

2. Key Representational Frameworks

4D Gaussian Splatting for Sparse Cameras

4C4D (Zhou et al., 5 Apr 2026) utilizes a 4D Gaussian Splatting (4DGS) scene parameterization, where a dynamic scene is modeled as a set of $N$ spatio-temporal Gaussians:

$$g_i = (\sigma_i, R_i, S_i, o_i, c_i, \mu_{t,i}, \sigma_{t,i})$$

with $\sigma_i \in \mathbb{R}^3$ (spatial center), $R_i \in SO(3)$ (orientation), $S_i$ (scale), $o_i$ (opacity), $c_i$ (spherical harmonics color), and temporal parameters $(\mu_{t,i}, \sigma_{t,i})$. The opacity field at $(\mathbf{x}, t)$ is

$$G_i(\mathbf{x}, t) = o_i \exp\left(-\tfrac{1}{2}(\mathbf{x}-\sigma_i(t))^\top \Sigma_{3\times 3,i}^{-1} (\mathbf{x}-\sigma_i(t))\right)\exp\left(-\tfrac{1}{2} \frac{(t-\mu_{t,i})^2}{\sigma_{t,i}^2}\right),$$

where $$g_i = (\sigma_i, R_i, S_i, o_i, c_i, \mu_{t,i}, \sigma_{t,i})$$0 encodes spatial covariance.

Compositional Gaussian Fields and Per-Object Deformations

In Comp4D (Xu et al., 2024), each semantic entity in a scene is described by a separate parameterized 3D Gaussian field (e.g., NeRF or 3D Gaussian splats), with dynamic trajectories and deformations governed by per-entity MLPs $$g_i = (\sigma_i, R_i, S_i, o_i, c_i, \mu_{t,i}, \sigma_{t,i})$$1. These entities are positioned along LLM-generated global trajectories and then composed into a scene, with dynamic consistency enforced via hybrid diffusion-based compositional losses.

Pointmap-Based Temporal Correspondence Augmentation

C4D (Wang et al., 16 Oct 2025) extends pointmap-centric static 3D reconstruction with dual temporal correspondences:

• Dense short-term optical flow between adjacent frames for local alignment.
• Sparse long-term 2D point/track correspondences across multiple frames, with mobility-aware prediction.

Per-frame pointmaps, camera parameters, and dynamic object segmentations are jointly optimized, with point trajectories lifted to world-space and smoothed for 4D recovery.

3. Algorithmic Innovations

Neural Decaying Functions for Geometric Regularization

4C4D (Zhou et al., 5 Apr 2026) introduces a Neural Decaying Function (NDF) on Gaussian opacities, modeled as a neural network $$g_i = (\sigma_i, R_i, S_i, o_i, c_i, \mu_{t,i}, \sigma_{t,i})$$2 predicting a decay factor $$g_i = (\sigma_i, R_i, S_i, o_i, c_i, \mu_{t,i}, \sigma_{t,i})$$3 applied to each Gaussian's opacity:

$$g_i = (\sigma_i, R_i, S_i, o_i, c_i, \mu_{t,i}, \sigma_{t,i})$$4

with visibility-conditional computation, promoting photometrically meaningful and geometrically consistent Gaussians under severe view sparsity.

LLM-Guided Commonsense Decomposition and Score Distillation

In Comp4D (Xu et al., 2024), the pipeline consists of:

1. Automatic entity & scale extraction from a prompt using GPT-4.
2. Per-entity static geometry synthesis (3D Gaussians/NeRFs) optimized via joint score distillation from Stable Diffusion (image) and MVDream (3D).
3. LLM-generated parametric trajectories (physics-based) for dynamic placement.
4. Compositional dynamic refinement using per-object MLP deformations and hybrid image/video diffusion loss, with additional regularizers for rigidity, acceleration, and inter-object contact.

Correspondence-Guided Motion Masking and Dynamic Point Tracking

C4D (Wang et al., 16 Oct 2025) employs:

• Dynamic-Aware Point Tracker (DynPT): Combines ViT and CNN encoders, outputting track positions, confidence, visibility, and a “mobility” score, differentiating static from dynamic points per frame.
• Motion mask creation: Intersection of sparse mobility cues and dense optical flow, with epipolar-consistency checks, yields per-frame static/dynamic segmentation for accurate camera and object motion optimization.
• Joint optimization: Loss functions target global pointmap alignment, camera-ego-motion alignment (static-only), trajectory smoothness, and point trajectory consistency.

4. Experimental Benchmarks and Results

Sparse Multi-Camera 4DGS (4C4D)

On Neural3DV, ENeRF-Outdoor, Mobile-Stage, and the Dyn4Cam datasets (Zhou et al., 5 Apr 2026), 4C4D demonstrated:

• PSNR improvements over 4DGaussians: e.g., 20.82→22.29 (Neural3DV).
• LPIPS and DSSIM scores improved (e.g., LPIPS: 0.190→0.146).
• Qualitative stability in depth and color under sparse capture, outperforming prior art in the four-camera protocol.

Ablations confirmed: removing NDF reduces PSNR by >2 points; removing visibility masking causes modest but measurable degradation.

Compositional Text-to-4D Generation (Comp4D)

On Q-Align metrics (img-Q, img-A, vid-Q, vid-A ∈ [1, 5]), Comp4D scored (avg. across canonical views) 2.93/2.19 (image quality/aesthetic) and 3.37/2.46 (video quality/aesthetic), improving substantially over 4dfy and Animate124 baselines. Training/inference speed reached 70 FPS at $$g_i = (\sigma_i, R_i, S_i, o_i, c_i, \mu_{t,i}, \sigma_{t,i})$$5 using compositional Gaussian renderers, outperforming NeRF-based baselines by nearly an order of magnitude. Qualitative analyses highlighted realistic long-range motions and rich inter-object contacts with minimal texture flicker.

Monocular 4D Dynamic Reconstruction (C4D)

C4D evaluated on Sintel, TUM-Dynamics, ScanNet, Bonn, KITTI, and TAP-Vid showed (Wang et al., 16 Oct 2025):

• Order-of-magnitude reduction in rotational pose error (dynamic scenes) vs. DUSt3R.
• Improved (Abs Rel, RMSE, δ<1.25) depth metrics and temporally smoother depth profiles.
• The DynPT achieves 87.9% D-ACC on MOVi-E point tracking, including occlusion and mobility accuracy.
• C4D robustly segments dynamic content, addressing severe motion blur and reducing temporal artifacts.

5. Limitations and Ablation Analyses

4C4D (Zhou et al., 5 Apr 2026)

• Performance relies on some spatial overlap; cannot handle non-overlapping or monocular scenarios.
• The NDF increases optimization complexity and requires a warm-up; simpler (non-learned) decays perform worse.
• No additional regularization needed beyond Gaussian pruning; the NDF delivers best results in geometry–appearance balance.

Comp4D (Xu et al., 2024)

• Necessity of fused per-object, multi-domain SDS is affirmed by ablations: single-object renders, joint renders, image guidance, and sufficient Gaussian count are all critical.
• The design is fundamentally object-centric, leveraging compositionality for scene richness and motion diversity.

C4D (Wang et al., 16 Oct 2025)

• Static point sampling limits dynamic segmentation in cluttered scenes; Sampson-error threshold regulates the static/dynamic mask trade-off.
• Point trajectory smoothing/propagation yields visible improvement in depth temporal coherence, but can admit false static correspondences if static regions are undersampled.

6. Summary of the “4C4D” Paradigm

Feature sets of “4C4D” across the three frameworks are summarized as:

Framework	Camera Regime / Input	Core Representation	Defining Innovations
4C4D (Zhou et al., 5 Apr 2026)	4 cameras (sparse video)	4D Gaussian splatting (per-Gaussian NDF)	Neural Decaying Function, visibility masking, photometric loss
Comp4D (Xu et al., 2024)	Text prompt	Compositional dynamics across 3D Gaussians	LLM-driven decomposition, compositional score distillation, per-entity MLP deformation
C4D (Wang et al., 16 Oct 2025)	Monocular video	Dynamic pointmaps with correspondence	Dual temporal correspondences, DynPT mobility prediction, correspondence-aided optimization

Collectively, “4C4D” encapsulates:

• Commonsense decomposition (entity/trajectory parsing or scene split for compositionality).
• Correspondence fusion (visibility, motion, or dual matching).
• Compositional or correspondence-guided losses (temporal smoothing, multi-domain distillation).
• Coherently lifted 4D representations (from geometry to texture to dynamic trajectory).

7. Future Directions

Research proposes several extensions:

• Extension to even sparser or monocular capture via depth priors or self-supervision (Zhou et al., 5 Apr 2026).
• Hierarchical or more expressive decaying functions for opacity learning (Zhou et al., 5 Apr 2026).
• Integration of learned deformation fields or diffusion priors for robustness (Zhou et al., 5 Apr 2026, Xu et al., 2024).
• Improving static/dynamic segmentation and multi-object tracking in cluttered or ambiguous video (Wang et al., 16 Oct 2025).
• Continued fusion of cross-modal pretrained experts (image, 3D, video diffusion) for richer compositional generation (Xu et al., 2024).

This suggests that the 4C4D paradigm unifies multiple directions for next-generation 4D scene modeling—balancing hardware efficiency, inference of compositional semantic structure, and geometrically faithful reconstruction or synthesis from minimal inputs.