4D Representations in Computational Science

• 4D representations are models that capture time-evolving 3D geometries by integrating spatial structure with motion and interaction cues.
• They enable dynamic scene reconstruction and simulation by balancing high visual fidelity with temporal coherence in applications like VR and robotics.
• Hybrid methods within 4D representations offer computational efficiency and editability by combining implicit neural fields with explicit geometric primitives.

A four-dimensional (4D) representation, in the context of contemporary computational science, refers to the explicit modeling and encoding of three-dimensional (3D) geometry that evolves over time. This temporal evolution may manifest as motion (rigid, articulated, or non-rigid deformation), interaction (entity–entity or human–object contacts), or dynamic state transitions within scenes. Modern research in this area synthesizes advances in neural implicit fields, explicit geometric primitives, structured kinematic models, and scene graph/relational abstractions to encode, analyze, and generate high-fidelity 3D content with spatiotemporal coherence. Representation selection is a pivotal consideration, dictating the balance between fidelity, interpretability, computational cost, and applicability across domains.

1. Fundamental Categories and Definitions

The foundations of 4D representations are organized along three conceptual axes—geometry, motion, and interaction (Zhao et al., 22 Oct 2025). "Geometry" addresses how the spatial extent and surface details of static or canonical 3D objects are encoded. "Motion" formalizes the evolution of this geometry: ranging from rigid transformations (e.g., $p_t = R p_{t-1} + t$), through articulated deformations (e.g., linear blend skinning: $p^m_i = \sum_{j=1}^J w_j(p^c_i) T_j p^c_i$), to learned displacement fields ($p_t = p_{t-1} + \Delta_\theta(p_{t-1}; \tau(t))$). "Interaction" concerns multi-entity systems, requiring a formalism for contacts, relative dynamics (pose, collision), and affordances, and may involve graph-based scene-graphs or tube-based temporal graphs (e.g., 4D panoptic scene graphs (Yang et al., 16 May 2024)).

4D representations are generally classified as:

• Unstructured: Meshes (vertices/faces), point clouds, neural radiance fields (NeRFs), and Gaussian Splatting, each extended to include temporal attributes or deformation fields.
• Structured: Template-based (e.g., SMPL, kinematic chains), part-based (semantic decomposition), or graph-based approaches encoding explicit relations or articulations.
• Hybrid/Compositional: Modular schemes where objects/scenes are composites of individually synthesized or decomposed 4D entities with controlled motion/trajectory (e.g., Comp4D (Xu et al., 25 Mar 2024)).

2. Core 4D Representation Models and Mathematical Formulations

Prominent paradigms for 4D representation include:

Model	Underlying Structure	Temporal Expansion / Deformation
NeRF-based	Implicit MLP radiance field	Time-conditioned: $F:(x,y,z,t,d)\rightarrow(\sigma,c)$; warping via deformation fields (Miao et al., 18 Mar 2025)
Gaussian Splatting	Explicit anisotropic Gaussian primitives	Augmented with temporal attributes; deformed via learned flow or warping fields (Zhao et al., 22 Oct 2025)
Mesh/Point-Cloud	Vertex/point sets with topology	Time-indexed trajectories or articulated/scene flow fields (Miao et al., 18 Mar 2025)
Structured/Graph	Templates, skeletons, graphs	Per-part/control-node trajectories, scene-graph edges (Zhao et al., 22 Oct 2025, Yang et al., 16 May 2024)

In NeRF-based models, temporal deformation often leverages static-to-dynamic mapping: $$(x, y, z, t) \longrightarrow (x', y', z') = (x, y, z) + D_\theta(x, y, z, t)$$ where $D_\theta$ is a deformation field parameterized by $\theta$. Gaussian Splatting represents a scene as a collection of 4D Gaussians $$G(x) = \exp(-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu))$$, where both mean $\mu$ and covariance $\Sigma$ are functions of time or explicitly parameterized via a motion field (Yin et al., 2023, Ji et al., 5 Jul 2024).

Dynamic mesh models (e.g., SMPL, STAR) typically employ skinning formulas or per-vertex trajectories, while point clouds are handled via temporal scene flow or point tracking (Zhao et al., 22 Oct 2025, Niu et al., 18 Feb 2025). In structured/part-based/artificial graph models, each semantic component or relation is tracked over time, facilitating editability and interpretability at the cost of greater modeling overhead.

3. Design Objectives and Theoretical Properties

The evaluation of 4D representations is strictly task-dependent but prioritized along several criteria (Zhao et al., 22 Oct 2025, Miao et al., 18 Mar 2025):

• Visual Fidelity: Photorealistic rendering, capturing fine geometry and view-dependent appearance.
• Temporal Consistency: Sustaining motion smoothness, coherent tracking/registration across frames, and avoiding flicker or drift, especially under sparse inputs (Ji et al., 5 Jul 2024, Wang et al., 5 Apr 2025).
• Topological Adaptability: Ability to accommodate and localize changes such as object splitting/merging, disappearance, or appearance—a domain where unstructured representations (Gaussian, NeRF, point clouds) outperform mesh or template-based models.
• Editability and Control: Structured representations (template, graph, part-based) offer semantic handles for manipulation, supporting intuitive dynamic editing and motion retargeting (Shao et al., 2023, Xu et al., 25 Mar 2024).
• Computational Efficiency and Scalability: Explicit encodings (e.g., Plane decomposition, Gaussian Splatting) enable rapid feed-forward inference and modeling of dynamic scenes (Yang et al., 2023, Ma et al., 23 Jun 2025). Optimization-based/incremental approaches trade real-time performance for generality.

Critical mathematical tools include score distillation sampling (SDS), which aligns rendered 4D outputs with diffusion model priors for both geometry and motion: $$\nabla_\theta L_{SDS} = \mathbb{E}_{t,\epsilon} [w(t)\,(\epsilon_\phi(x; y, t) - \epsilon) \,\partial x/\partial \theta ]$$ where $x = g(\theta)$ is the rendered image at parameters $\theta$, and $w(t)$ is a diffusion step-dependent weight (Miao et al., 18 Mar 2025).

4. Representative Applications and Data

4D representations underpin multiple modern applications:

• Dynamic Scene Reconstruction: Leveraging spatiotemporal priors to reconstruct scenes from multi-view or video inputs, yielding full-resolution 4D representations for VR/AR or robotics (Ma et al., 23 Jun 2025, Nizamani et al., 5 Mar 2025).
• Generative 4D Asset Synthesis: Enabling object/scene creation with user-controllable dynamics from text, video, or sketch input (Yin et al., 2023, Xu et al., 25 Mar 2024, Miao et al., 18 Mar 2025).
• Digital Human/Avatar Modeling: Human motion, facial expressions, and character animation, typically fusing structured models (SMPL, skeletons) with temporal deformation fields or diffusion-guided synthesis (Miao et al., 18 Mar 2025).
• Event and Interaction Understanding: Scene graph approaches abstract dynamic scenes into nodes (entities, objects), edges (temporal or interaction relationships), supporting reasoning and action-planning (e.g., service robotics) (Yang et al., 16 May 2024).
• Robotic Learning and Embodied AI: Spatiotemporal encodings (as low-level 3D point tracks with time) transfer knowledge from abundant human video to robotic agent control, via shared geometric structures between 4D point representations and robot states (Niu et al., 18 Feb 2025).

Datasets powering 4D research include static 3D repositories (ShapeNet, Objaverse), multi-view and motion capture corpora (ActorsHQ, ZJU-MoCap, DynaCap), video-based collections (WebVid-10M), and interaction-centric datasets (PartNet-Mobility, GRAB, BEHAVE) (Zhao et al., 22 Oct 2025). Still, a notable gap persists: large-scale public 4D datasets with high-fidelity ground truth across geometry, motion, and physical interactions are lacking.

5. 4D Representation Challenges

The main technical and practical challenges in developing and deploying 4D representations include (Miao et al., 18 Mar 2025, Zhao et al., 22 Oct 2025):

• Consistency: Achieving both spatial and temporal coherence remains non-trivial, especially under view/time sparsity or ambiguous supervision. Techniques such as part-based tracking, regularization, and compositional score distillation mitigate—but do not eliminate—these problems.
• Controllability/Diversity: Maintaining user control over motion/appearance (through text, trajectory, video) while supporting diverse dynamic behaviors is central for creative and interactive applications (Xu et al., 25 Mar 2024).
• Efficiency: Scaling iterative 4D optimization procedures is computationally demanding. Factorized/triplane/Gaussian-Plane representations and hybrid explicit–implicit networks have proven advantageous for compressing memory and accelerating inference (Yang et al., 2023, Shao et al., 2023).
• Topological Limitations: Mesh and template-based approaches are constrained by fixed connectivity, rendering them ill-suited for scenarios with dynamic topology changes. Conversely, unstructured methods may sacrifice semantic control.

The "4D dataset bottleneck" is arguably the most significant practical impediment to progress (Zhao et al., 22 Oct 2025), impeding robust training and evaluation.

6. Emerging Methodologies and Future Directions

Recent advancements demonstrate trends toward hybrid and unified frameworks (Zhao et al., 22 Oct 2025, Xu et al., 25 Mar 2024, Ma et al., 23 Jun 2025):

• Hybrid Representations: Combining interpretable, semantically structured elements (templates, parts, graphs) with smooth, expressive implicit representations (NeRF, Gaussian Splatting) to maximize both controllability and fidelity.
• Foundation Model Integration: LLMs enable reasoning about semantic content, decomposition, and motion intent (Xu et al., 25 Mar 2024, Zhao et al., 22 Oct 2025). Video foundational models (VFMs), especially diffusion models, supply strong spatiotemporal priors for both geometry and motion, though challenges remain in achieving 3D/4D-awareness and physically plausible continuity.
• Scalability and Generalization: Dataset-agnostic, large-scale pretraining (e.g., transformer-based masked autoencoding on video (Carreira et al., 19 Dec 2024)) demonstrates that 4D representations can scale and generalize to diverse vision tasks, with improvements remaining monotonic up to multi-billion parameter models.
• Physical Plausibility: Regularizing representations against differentiable physics or universal motion constraints is a developing area, aimed at ensuring not merely visual plausibility but also realism and coherence under real-world dynamics.

Open problems involve creating large, richly annotated 4D datasets, developing universal benchmarks, and extending representations to CAD-level industrial precision, robotics, and fully interactive virtual environments.

7. Summary Table of Representative 4D Representation Families

Representation Family	Geometric Primitive	Temporal Modeling	Key Applications
NeRF-based / Implicit Fields	Continuous radiance fields	Deformation fields, time input	Asset generation, AR/VR
Gaussian Splatting/Plane-based	Gaussian surfels/tri-planes	Flow-guided warping, 4D fusion	Real-time reconstruction
Mesh-/Template-based	Mesh + rig/skeleton	Skinning, part rotation	Digital avatars, animation
Point Cloud/Scene Flow	Point-sets	Flow/displacement trajectories	Robotics, LiDAR
Scene Graph/Compositional	Graph/tube abstraction	Temporal node/edge labeling	Embodied AI, reasoning

This overview synthesizes the state of 4D representations as an active area at the intersection of geometry, motion, and interaction modeling, with ongoing innovations in representation design, algorithmic efficiency, semantic integration, and physical grounding. The careful selection and customization of representation paradigm should be dictated by application requirements, available data, and desired trade-offs among fidelity, efficiency, editability, and generalization.