Geometric 4D Stitching for Grounded 4D Generation

Abstract: Recent 4D generation methods complete scene-level missing information using generative models and reconstruct the scene into radiance-based representations. However, these pipelines often present geometric inconsistencies in the generated content, and the radiance-based reconstruction requires expensive optimization. Furthermore, radiance-based representations often absorb these geometric inconsistencies into their view-dependent nature, failing to enforce the grounded geometric consistency. To address these issues, we propose Geometric 4D Stitching, an efficient framework that explicitly identifies missing geometric regions and complements them with geometrically grounded 4D stitches. As a result, our method constructs 4D scene representations in under 10 minutes on a single NVIDIA RTX 5090 GPU per one-step scene expansion, while improving geometric consistency. Moreover, we demonstrate that our explicit 4D stitching supports interative expansion of 4D mesh as well as 4D scene editing.

• The paper introduces a geometry-centric pipeline that selectively completes missing regions using explicit geometric stitching, improving 4D scene consistency.
• The approach employs pyramidal depth refinement and render-disagreement distillation to achieve superior perceptual quality and camera-motion fidelity over radiance-based methods.
• Empirical results demonstrate significant improvements in video quality and runtime efficiency, enabling interactive scene editing and robust 4D scene generation.

Geometric 4D Stitching for Grounded 4D Generation

Motivation and Problem Statement

Recent progress in 4D scene generation leverages generative models for multi-view video synthesis and the construction of radiance-based representations (e.g., NeRFs, Gaussian splatting). However, such pipelines suffer from geometric inconsistencies in generated content and require expensive per-scene radiance optimization, while radiance-field models tend to absorb inconsistencies into their inherently view-dependent appearance models, ultimately failing to guarantee geometrically consistent results. The paper "Geometric 4D Stitching for Grounded 4D Generation" (2605.09984) addresses these challenges by proposing a geometry-centric pipeline that selectively completes only genuinely missing geometric regions through explicit geometric stitching, bypassing the limitations of dense generative view completion and radiance-based field reconstruction.

Figure 1: The core concept of Geometric 4D Stitching: leveraging geometry-aware sparse generative video to progressively construct an expandable, reliable 4D representation, refining NVS-based geometry only where necessary.

Methodological Framework

Initial 4D Asset Construction

From an input monocular video (either generated or observed), a time-indexed explicit triangular mesh is constructed using estimated per-frame depths (via DA3 [yang2025depthanything3]) and source-view camera parameters. This initial mesh sequence forms the foundation for later scene expansion. Background completion is applied at this stage to ameliorate occlusion bottlenecks under sparse views.

Incomplete Region Identification and NVS

To determine where new information is required for scene completion, the approach renders the initial mesh from a target viewpoint and detects missing geometries as projection-invalid pixels (holes) and regions exposed by occlusion (curtain effect), using a combination of mesh and point cloud projections. The inpainting mask for a given view transition is thus an explicit union of classical warp-induced holes and occlusion-induced additions, formally:

$$\tilde{M}_{i \to j}^t = M_{i \to j}^t \cup M_{i \to j}^{t,+}$$

This region guides the NVS module to synthesize novel content strictly where geometrically justified, instead of producing dense generative completions.

Figure 2: Pipeline overview. (a) Initial mesh is constructed. (b) Geometry is used to anchor NVS target-view refinement. (c-d) New content is detected and stitched, enabling progressive expansion. (e) Lightweight diffusion further refines the explicit geometry for novel view synthesis.

Pyramidal Depth Refinement

The synthesized NVS geometry typically fails to align with the global 4D mesh due to local and cross-view miscalibrations. A local, patch-wise scale-shift pyramidal refinement is introduced, producing a spatially-varying correction field that aligns novel-view depth estimates to the source mesh geometry by minimizing anchor alignment, geometry-aware smoothness, and stabilization objectives. This yields stable and coherent novel-view geometry well-suited for mesh insertion.

Geometric 4D Stitching and Robust Candidate Selection

Candidate content from the NVS stage is not naively inserted, but instead "stitched" only in the information-addition regions where geometric support has been refined. Each candidate addition is explicitly back-projected and appended to the global mesh, producing the updated 4D asset. To further guard against introducing erroneous geometry, render-disagreement distillation is employed: newly inserted regions are checked for persistent disagreement against anchor views, and inconsistent additions are culled.

Figure 3: Stitching image regions: warping preserves source-aligned structure but leaves holes; sampling fills but may deviate; stitching fuses reliable projection-aligned content with generated content only where needed, yielding coherence.

Visual Refinement

To ameliorate mesh-level visual artifacts (e.g., boundary jagging), a lightweight diffusion-based refinement (TrajectoryCrafter) is applied, conditioned on the mesh renderings. Notably, this step enhances visual quality without modifying the underlying, explicitly constructed geometry.

Empirical Results and Quantitative Insights

The authors conduct extensive evaluations encompassing perceptual video quality (VBench [huang2024vbench]), camera-motion fidelity (pose estimation error via DA3), and geometry-motion self-consistency via reprojection. Their method is compared primarily against state-of-the-art radiance-based approaches (D-NeRF [Pumarola_2021_CVPR], 4DGS [Wu_2024_CVPR]).

Strong Numerical Results

• Perceptual video quality: The proposed method achieves a mean VBench score of 0.85, outstripping both D-NeRF and 4DGS, particularly in subject consistency (0.94 vs. 0.79/0.84) and image quality (0.71 vs. 0.46/0.38).
• Camera-motion fidelity: Achieves low absolute translation error (ATE mean 0.019 vs. 0.13 for D-NeRF and 0.033 for 4DGS) and substantially improved rotation error (mean 8.78°, substantially lower than D-NeRF’s 66.5°).
• Runtime efficiency: The 4D scene can be constructed in under 10 minutes per expansion on a single RTX 5090, with a single iteration operating in linear time with respect to the sequence length.

These results highlight that, under sparse-view and generative data constraints, explicit geometric stitching achieves higher geometric and perceptual consistency than radiance-based optimization.

Implications and Discussion

Theoretical and Practical Implications

By reframing 4D generation as region-wise geometric completion, the geometric stitching paradigm fundamentally alters the cost-accuracy tradeoff in scene synthesis. Unlike radiance-field models that require view redundancy and substantial optimization, this approach adapts sampling and computation to strictly necessary regions, controls error propagation, and maintains explicit geometric control.

Practically, this unlocks new downstream applications difficult for prior methods, including geometrically robust scene extrapolation and interactive 4D scene editing (e.g., object addition/removal, local modification) with explicit geometric consistency—critical for AR/VR, simulation, and digital content creation.

(Figure 4)

Figure 4: Downstream applications such as object editing and scene modifications, enabled by explicit, controllable 4D geometric assets.

Limitations and Open Problems

The pipeline remains bounded by the reliability of NVS backbones and the feed-forward geometry estimator; inaccuracies here can still manifest in the stitched additions. Importantly, the method is most suitable for creative and simulation uses rather than for metrically accurate or safety-critical deployments, as unrevealed or occluded regions may be hallucinated or geometrically invalid.

Future Directions

Potential avenues include enhancing scalability to large-scale or dynamic scenes, end-to-end differentiable optimization incorporating human feedback on geometric plausibility, generalization to arbitrary video sources and capture conditions, and fusion with radiance-based methods for optimal tradeoffs between explicitness and photorealism. Integration with emerging 4D vision transformers or more powerful scene priors will further amplify the utility and fidelity of geometric stitching.

Conclusion

Geometric 4D Stitching introduces a principled, geometry-driven alternative to 4D scene generation, where explicit region-wise geometric completion supplants dense generative supervision and costly radiance-based reconstruction. The resulting assets exhibit robust spatial and temporal coherence, support efficient scene editing and expansion, and empirically outperform radiance-field pipelines under sparse and generative view regimes. This paradigm shift underlines the feasibility and advantages of explicit geometric modeling for future 4D scene generation tasks, offering a compelling foundation for controllable and consistent scene synthesis in both research and creative contexts.