Forge4D: Unified 4D Modeling Approaches

• Forge4D is a 4D vision framework that unifies dynamic-scene generation with efficient human reconstruction using complementary approaches.
• The dynamic-scene formulation employs a single video diffusion backbone combined with Unified Masked Conditioning and Decoupled LoRA Control to generate synchronized RGB frames and 3D pointmaps from sparse inputs.
• The feed-forward human reconstruction model utilizes 3D Gaussian regression and state-token guided dense motion prediction to deliver real-time, metric-scale 4D human representations.

Searching arXiv for Forge4D and closely related papers to ground the article. Use arXiv search tool? Forge4D designates two distinct but technically related formulations in recent 4D vision research. In one formulation, it is a unified 4D dynamic-scene generation and reconstruction system derived from the principles of One4D, using a single video diffusion backbone to generate synchronized RGB frames and 3D pointmaps under arbitrarily sparse conditioning (Mi et al., 24 Nov 2025). In the other, it is a feed-forward 4D human reconstruction and interpolation model from uncalibrated sparse-view videos, designed to reconstruct temporally aligned representations that support both novel view and novel time synthesis (Hu et al., 29 Sep 2025). The shared label therefore spans two research directions: geometry-aware 4D generation with video diffusion, and efficient 4D human reconstruction with dynamic 3D Gaussians.

1. Scope within 4D vision and generation

The dynamic-scene formulation of Forge4D is explicitly positioned as an extension of One4D. One4D presents a unified framework for 4D generation and reconstruction that produces dynamic 4D content as synchronized RGB frames and pointmaps, and it uses Unified Masked Conditioning (UMC) and Decoupled LoRA Control (DLC) to move between single-image generation, sparse-frame conditioning, and full-video reconstruction without architectural change (Mi et al., 24 Nov 2025). By contrast, the human-reconstruction formulation of Forge4D addresses instant reconstruction of dynamic 3D humans from uncalibrated sparse-view videos, with a feed-forward pipeline that combines streaming 3D Gaussian reconstruction and dense motion prediction (Hu et al., 29 Sep 2025).

These two usages sit within a broader research landscape. WorldForge targets precise 4D controllability in video diffusion models through a fully training-free, inference-time guidance framework composed of Intra-Step Recursive Refinement (IRR), Flow-Gated Latent Fusion (FLF), and Dual-Path Self-Corrective Guidance (DSG) (Song et al., 18 Sep 2025). 4DGen, in turn, introduces grounded 4D content creation from monocular video sequences or image-to-video generations, using dynamic 3D Gaussians, spatial-temporal pseudo labels on anchor frames, 3D-aware score distillation sampling, and smoothness regularization (Yin et al., 2023). This suggests that Forge4D is best understood not as a single canonical architecture, but as a label attached to two complementary approaches to 4D modeling.

2. Unified dynamic-scene Forge4D derived from One4D

In its dynamic-scene sense, Forge4D adopts a pretrained flow-matching video diffusion model, specifically Wan2.1 Fun-V1.1 14B, and extends it to jointly generate RGB video and XYZ video. The inputs are a text prompt, a conditioning video $X_c \in \mathbb{R}^{3\times F\times H\times W}$, and a binary mask $M_c \in \{0,1\}^{1\times F\times H\times W}$ indicating observed frames (Mi et al., 24 Nov 2025).

The latent encoding stage maps RGB, geometry, and conditioning inputs into VAE latents:

$$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$

Under Rectified Flow / Flow-Matching, for random $t\in[0,1]$ and Gaussian noise,

$$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$

The DiT backbone receives

$$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$

while the geometry branch does not directly consume the conditioning latent and instead accesses conditioning through DLC links.

The denoising model predicts velocity fields

$$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$

and is trained with

$$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$

After denoising, the VAE decodes $\hat z_{\mathrm{rgb}}$ and $\hat z_{\mathrm{xyz}}$ into RGB frames and pointmaps. A post-optimization stage then recovers camera intrinsics $M_c \in \{0,1\}^{1\times F\times H\times W}$0, extrinsics $M_c \in \{0,1\}^{1\times F\times H\times W}$1, and depth maps $M_c \in \{0,1\}^{1\times F\times H\times W}$2 from generated pointmaps by minimizing a pointmap consistency term $M_c \in \{0,1\}^{1\times F\times H\times W}$3 and a smoothness term $M_c \in \{0,1\}^{1\times F\times H\times W}$4, combined as $M_c \in \{0,1\}^{1\times F\times H\times W}$5.

The resulting pipeline is explicitly intended to support pure generation, mixed generation-and-reconstruction, and full reconstruction within one model. A plausible implication is that the system treats 4D world modeling as joint appearance-and-geometry video synthesis rather than as a reconstruction-only problem.

3. Unified Masked Conditioning and Decoupled LoRA Control

UMC is the mechanism that allows this Forge4D formulation to handle arbitrary conditioning sparsity. Observed RGB frames are packed into a single conditioning video according to

$M_c \in \{0,1\}^{1\times F\times H\times W}$6

The mask is downsampled to latent resolution and concatenated with the noisy RGB latent and conditioning latent. By changing the sparsity pattern of $M_c \in \{0,1\}^{1\times F\times H\times W}$7, the same model instantiates single-image generation, sparse-frame interpolation, or full-video reconstruction (Mi et al., 24 Nov 2025).

A defining design choice is that geometry tokens never see $M_c \in \{0,1\}^{1\times F\times H\times W}$8 directly. Instead, geometry receives conditioning only through DLC. This is described as a way to avoid artifact injection while still enabling RGB–geometry coupling. DLC attaches two modality-specific LoRA adapters to every DiT submodule, one for RGB and one for pointmaps, while keeping the base model frozen. For a generic DiT submodule,

$M_c \in \{0,1\}^{1\times F\times H\times W}$9

The two branches share the pretrained base weights but compute independently.

Cross-modal consistency is introduced only at a sparse set of layers through Zero-Initialized Control Links:

$$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$0

Because the linear layers $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$1 are initialized to zero, the pretrained video prior is preserved at the start of training, and cross-modal interaction is learned gradually. The network architecture couples this mechanism with 3D causal VAE encoders, symmetric UNet-style decoders, and a DiT backbone with multi-head spatial-temporal self-attention, cross-attention to text, and MLP blocks. LoRA adapters of rank $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$2 are attached to every Linear layer in each DiT submodule for both branches, accounting for approximately $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$3M parameters, while ZCL links are inserted in $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$4 strategically chosen layers for approximately $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$5M additional parameters (Mi et al., 24 Nov 2025).

4. Training setup and reported behavior of the unified system

The dynamic-scene Forge4D formulation is trained on a mixture of synthetic and real 4D data. The synthetic set comprises OmniWorld-Game, BEDLAM, PointOdyssey, and TarTanAir with $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$6k clips, while the real set uses SpatialVID annotated with Geo4D pseudo-geometry, also with $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$7k clips. The total corpus is approximately $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$8M frames, with each segment approximately $$z_{\mathrm{rgb}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{rgb}}), \quad z_{\mathrm{xyz}} = \mathrm{VAE}_{\mathrm{enc}}(X_{\mathrm{xyz}}), \quad z_c = \mathrm{VAE}_{\mathrm{enc}}(X_c).$$9 frames at resolution $t\in[0,1]$0 (Mi et al., 24 Nov 2025).

Optimization uses Wan2.1 Fun-V1.1-14B as the base model, LoRA rank $t\in[0,1]$1, and approximately $t\in[0,1]$2M trainable parameters. Training is performed on $t\in[0,1]$3NVIDIA H800 with batch size $t\in[0,1]$4/GPU and gradient accumulation $t\in[0,1]$5, for $t\in[0,1]$6 AdamW steps at learning rate $t\in[0,1]$7. Task sampling uses ratios $t\in[0,1]$8 for single-image, $t\in[0,1]$9 for sparse-frame, and $$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$0 for full-video inputs. Inference uses $$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$1 flow-matching steps with CFG scale $$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$2 (Mi et al., 24 Nov 2025).

The reported empirical results separate generation from reconstruction.

Setting	Metric	Reported result
Single-image $$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$3 4D	User study vs. 4DNeX, Overall 4D	$$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$4 vs. $$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$5
Single-image $$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$6 4D	VBench Dynamics	$$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$7 vs. $$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$8
Single-image $$z_{\mathrm{rgb}}^t = t\,z_{\mathrm{rgb}} + (1-t)\,\epsilon_{\mathrm{rgb}}, \quad z_{\mathrm{xyz}}^t = t\,z_{\mathrm{xyz}} + (1-t)\,\epsilon_{\mathrm{xyz}}.$$9 4D	VBench I2V Consistency	$$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$0 vs. $$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$1
Full reconstruction, Sintel	AbsRel / $$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$2	$$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$3 / $$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$4
Full reconstruction, Bonn	AbsRel / $$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$5	$$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$6 / $$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$7
Sparse-frame, keep $$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$8	Sintel AbsRel / $$z_{\mathrm{input}} = \mathrm{Concat}\!\bigl(z_{\mathrm{rgb}}^t,\; z_c,\; M_c\bigr),$$9	$$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$0 / $$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$1
Sparse-frame, keep $$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$2	Sintel AbsRel / $$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$3	$$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$4 / $$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$5

The qualitative summary reports sharper geometry, more pronounced motion, stable backgrounds, richer 4D pointclouds, recovery of thin structures such as leaves and ropes, stable camera paths, and accurate depth under sparse inputs (Mi et al., 24 Nov 2025). The ablation study reports that CFG scale in $$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$6 yields similar reconstruction accuracy, and that even $$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$7 training steps yield reasonable geometry, with accuracy improving steadily to $$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$8 steps. This suggests that the main contribution is not merely a larger backbone, but the combination of unified conditioning and decoupled modality adaptation.

5. Feed-forward Forge4D for 4D human reconstruction and interpolation

The second usage of Forge4D refers to a specialized human-centered system for feed-forward 4D reconstruction from uncalibrated sparse-view videos. Its motivation is instant reconstruction of a dynamic human in 3D over time under sparse views, unknown camera calibration, and real-time constraints (Hu et al., 29 Sep 2025).

This model decomposes the problem into three sequential stages. The first stage, static 3D Gaussian reconstruction, uses a pretrained VGGT backbone to predict colored 3D Gaussians from $$v_{\mathrm{rgb}}^t = \frac{d\,z_{\mathrm{rgb}}^t}{dt}, \quad v_{\mathrm{xyz}}^t = \frac{d\,z_{\mathrm{xyz}}^t}{dt},$$9 uncalibrated RGB images at a single key timestamp while enforcing metric-gauge alignment. The second stage, streaming dynamic reconstruction, introduces a single learnable state token $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$0 that carries temporal context across frames via cross-attention. The third stage, motion prediction plus Gaussian fusion, predicts dense backward and forward motions for each view and pixel-aligned Gaussian, then interpolates arbitrary timestamps by warping and fusing Gaussians under a constant-velocity assumption (Hu et al., 29 Sep 2025).

The Gaussian representation is parameterized by mean position $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$1, covariance $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$2, density $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$3, color $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$4, opacity $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$5, rotation $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$6, and axis-aligned scales $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$7, with density

$$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$8

Static reconstruction supervises both input and held-out novel views through a photometric objective $$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_t\Bigl[ \bigl\|v_{\mathrm{rgb}}^t-(z_{\mathrm{rgb}}-\epsilon_{\mathrm{rgb}})\bigr\|_2^2 + \bigl\|v_{\mathrm{xyz}}^t-(z_{\mathrm{xyz}}-\epsilon_{\mathrm{xyz}})\bigr\|_2^2 \Bigr].$$9 and metric-gauge regularization $\hat z_{\mathrm{rgb}}$0, combined in $\hat z_{\mathrm{rgb}}$1.

For temporal modeling, the state token is used in two directions: $\hat z_{\mathrm{rgb}}$2 as Key/Value attends to frame-$\hat z_{\mathrm{rgb}}$3 features as Query, and frame-$\hat z_{\mathrm{rgb}}$4 features as Key/Value attend to $\hat z_{\mathrm{rgb}}$5 as Query, producing the updated state $\hat z_{\mathrm{rgb}}$6. The temporal camera loss extends gauge consistency across all cameras and timestamps. The dense motion module then predicts backward motion $\hat z_{\mathrm{rgb}}$7 and forward motion $\hat z_{\mathrm{rgb}}$8 using motion attention blocks and a DPT-style head. Motion supervision is self-supervised: the retargeting loss renders Gaussians warped from frame $\hat z_{\mathrm{rgb}}$9 back to $\hat z_{\mathrm{xyz}}$0 and compares them to the reconstruction at $\hat z_{\mathrm{xyz}}$1, while an occlusion-aware optical-flow loss compares projected 3D motion to pseudo-ground-truth 2D flow from SEA-RAFT. The combined matching loss is $\hat z_{\mathrm{xyz}}$2 (Hu et al., 29 Sep 2025).

Interpolation uses the warped Gaussian sets

$\hat z_{\mathrm{xyz}}$3

and an occlusion-aware Gaussian fusion process based on a dual-consistency factor $\hat z_{\mathrm{xyz}}$4. If $\hat z_{\mathrm{xyz}}$5, the Gaussian is marked occluded; the remaining visible Gaussians are fused by a lightweight MLP $\hat z_{\mathrm{xyz}}$6 and supervised through a novel-time photometric loss $\hat z_{\mathrm{xyz}}$7. The design goal is simultaneous temporal coherence, metric-scale alignment, and efficient inference.

6. Empirical profile of the human-reconstruction model

The human-reconstruction Forge4D is trained on DNA-Rendering with $\hat z_{\mathrm{xyz}}$8 human video sequences, evaluated in-domain on held-out DNA-Rendering identities, out-of-domain on Genebody, and for motion and metric evaluation on synthetic MetaHuman4D with $\hat z_{\mathrm{xyz}}$9 identities and $M_c \in \{0,1\}^{1\times F\times H\times W}$00 motions (Hu et al., 29 Sep 2025). Reported metrics include PSNR, SSIM, and LPIPS for image quality, $M_c \in \{0,1\}^{1\times F\times H\times W}$01 motion error and retargeted point distance for motion, and mean point-to-mesh distance for metric scale.

Setting	Metric	Reported result
Static novel-view synthesis, 4 input views	PSNR / SSIM / LPIPS	$M_c \in \{0,1\}^{1\times F\times H\times W}$02 dB / $M_c \in \{0,1\}^{1\times F\times H\times W}$03 / $M_c \in \{0,1\}^{1\times F\times H\times W}$04
Novel-time + novel-view, 4-view videos	PSNR	$M_c \in \{0,1\}^{1\times F\times H\times W}$05 dB
Motion prediction, MetaHuman4D	Motion error / retargeted point distance	$M_c \in \{0,1\}^{1\times F\times H\times W}$06 / $M_c \in \{0,1\}^{1\times F\times H\times W}$07
Metric scale	Mean point-to-mesh distance	$M_c \in \{0,1\}^{1\times F\times H\times W}$08 m
Runtime, NVIDIA H200	Key-frame / motion / interpolate 10 steps	$M_c \in \{0,1\}^{1\times F\times H\times W}$09 ms / $M_c \in \{0,1\}^{1\times F\times H\times W}$10 ms / $M_c \in \{0,1\}^{1\times F\times H\times W}$11 ms

The baselines listed in the same report include NoPosplat, AnySplat, and GPS-Gaussian for static novel-view synthesis; L4GM, SpaceTimeGS, and D-3DGS for novel-time plus novel-view synthesis; POMATO for motion prediction; and MoGe-2 for metric scale (Hu et al., 29 Sep 2025). Ablations attribute substantial degradation to removal of gauge alignment, which causes static PSNR to collapse to $M_c \in \{0,1\}^{1\times F\times H\times W}$12 dB; removal of the state token, which drops dynamic PSNR by approximately $M_c \in \{0,1\}^{1\times F\times H\times W}$13 dB; removal of retargeting loss, which reduces PSNR by $M_c \in \{0,1\}^{1\times F\times H\times W}$14 dB; and removal of optical-flow loss, which reduces PSNR by $M_c \in \{0,1\}^{1\times F\times H\times W}$15 dB. Removing the fusion MLP produces virtually no change in PSNR but causes visible jitter and flicker.

The stated strengths are that this is the first feed-forward 4D human reconstruction from uncalibrated sparse views at real-world scale, that joint static Gaussian regression with explicit dense motion prediction yields high-fidelity geometry and appearance, that metric-gauge regularization enables accurate real-world measurements, and that the streaming state-token design keeps memory use bounded for arbitrary video length (Hu et al., 29 Sep 2025). The listed limitations are that the linear velocity assumption breaks down for very large motions or long intervals, and that extreme occlusions or loose clothing may challenge the fusion mechanism.

7. Relation to adjacent methods and conceptual distinctions

The two Forge4D usages can be situated relative to 4DGen and WorldForge. 4DGen uses monocular video as the explicit motion specifier, represents dynamics with deformable 3D Gaussian Splatting, predicts offsets through a HexPlane plus MLP deformation network, and optimizes a loss

$M_c \in \{0,1\}^{1\times F\times H\times W}$16

with 3D-aware SDS, pseudo-label supervision on anchor frames, spatial total variation, and temporal smoothness penalties (Yin et al., 2023). WorldForge, by contrast, is explicitly training-free: it injects trajectory control at inference time using IRR for recursive masked refinement, FLF for per-channel flow-based gating, and DSG for guided-versus-unguided self-correction, reporting improvements in FID, CLIP similarity, ATE, RPE-T, RPE-R, FVD, and CLIP-V similarity (Song et al., 18 Sep 2025).

Against this background, the unified dynamic-scene Forge4D emphasizes joint RGB-and-pointmap generation with a video diffusion backbone and trainable decoupled adapters, whereas the human-reconstruction Forge4D emphasizes feed-forward Gaussian regression, temporal state propagation, and self-supervised dense motion. One source of potential confusion is therefore terminological: the same name identifies both a general dynamic-scene design derived from One4D and a specific 4D human reconstruction model. A plausible implication is that “Forge4D” has become a label for systems that couple 4D geometry with temporally coherent rendering, but the architectural commitments differ substantially depending on whether the target is general dynamic-scene generation or sparse-view human reconstruction.