NavCrafter: Exploring 3D Scenes from a Single Image

Abstract: Creating flexible 3D scenes from a single image is vital when direct 3D data acquisition is costly or impractical. We introduce NavCrafter, a novel framework that explores 3D scenes from a single image by synthesizing novel-view video sequences with camera controllability and temporal-spatial consistency. NavCrafter leverages video diffusion models to capture rich 3D priors and adopts a geometry-aware expansion strategy to progressively extend scene coverage. To enable controllable multi-view synthesis, we introduce a multi-stage camera control mechanism that conditions diffusion models with diverse trajectories via dual-branch camera injection and attention modulation. We further propose a collision-aware camera trajectory planner and an enhanced 3D Gaussian Splatting (3DGS) pipeline with depth-aligned supervision, structural regularization and refinement. Extensive experiments demonstrate that NavCrafter achieves state-of-the-art novel-view synthesis under large viewpoint shifts and substantially improves 3D reconstruction fidelity.

• The paper introduces a unified framework combining camera-controllable video diffusion, collision-aware trajectory planning, and enhanced 3D Gaussian Splatting for single-image 3D scene reconstruction.
• The method employs dual-branch feature injection and LoRA-based attention modulation to enforce strict viewpoint control and temporal consistency.
• Iterative view synthesis with collision detection and depth-aligned supervision significantly improves scene completeness and geometric fidelity over traditional approaches.

NavCrafter: Exploring 3D Scenes from a Single Image

Introduction and Motivation

NavCrafter addresses the challenge of reconstructing high-fidelity 3D scenes from a single image, a task of high practical significance where direct 3D data acquisition is expensive or infeasible. Traditional NeRF and 3DGS methods require dense, multi-view data and cannot generalize to this underconstrained regime. Recent advancements in video diffusion models provide strong generative priors; however, they historically lack controllability and spatiotemporal consistency necessary for robust 3D scene synthesis and subsequent reconstruction. NavCrafter introduces a unified framework that couples a camera-controllable video diffusion backbone with geometry-aware, collision-averse trajectory planning and enhanced 3DGS-based reconstruction, aiming to overcome the critical limitations of prior work in this domain (Figure 1).

Figure 1: The NavCrafter framework. (1) Controllable novel-view synthesis leverages video diffusion with explicit camera trajectory conditioning; (2) Iterative view synthesis with collision-aware trajectory planning; (3) Geometry-aware 3D reconstruction via enhanced 3D Gaussian Splatting with depth-aligned supervision and structural regularization.

Controllable Video Diffusion with Multi-Stage Camera Control

NavCrafter's backbone is a transformer-based video diffusion model augmented to provide explicit control of camera viewpoints and maintain multi-view temporal coherence essential for 3D consistency. The key innovation is a multi-stage camera control architecture:

• Dual-Branch Feature Injection: Camera pose signals are encoded (using Plücker embedding) and injected directly into video token streams both pre-transformer block and within self-attention outputs, initializing strong viewpoint constraints and reinforcing cross-view consistency.
• LoRA-Based Attention Modulation: Lightweight 3D convolutional encoders project pose embeddings as LoRA control tokens that modulate all attention operations, sharpening trajectory adherence without incurring full fine-tuning cost.
• Random Reference Frame Attention: Random sampling of reference frames in self-attention enhances temporal consistency beyond explicit pose conditioning.

This architecture explicitly enables synthesis of view sequences that adhere to prescribed camera trajectories, as evidenced by significant improvement in rotation and translation errors compared to all baselines (Table 3 in the paper).

Figure 2: Qualitative novel view synthesis comparison. Blue boxes denote reference regions; orange boxes denote failures of previous models. NavCrafter consistently preserves structure and fidelity.

Iterative View Synthesis with Collision-Aware Trajectory Planning

Typical video diffusion rollouts are susceptible to drift and collision with scene geometry, particularly under aggressive, long-horizon viewpoint changes. NavCrafter uses an iterative, NBV-driven synthesis paradigm wherein:

• Collision Awareness: Candidate camera poses are pruned with a geometric collision detector that leverages the expanding point cloud, ensuring all paths remain physically valid.
• Adaptive Trajectory Optimization: Smooth, collision-free camera trajectories are generated by minimizing both penetration risk and trajectory discontinuity, ensuring plausible motion and view coverage.
• Progressive Expansion: Each synthesized view with refined pose is back-projected to further enhance scene point clouds, bootstrapping more accurate coverage and geometry with each iteration.

This approach reliably avoids geometric inconsistencies prevalent in prior direct or utility-only NBV strategies, improving completeness (77.67% vs 66.20%), reducing noise, and yielding higher F-scores (Table 4).

Figure 3: Comparison of point cloud reconstruction quality between NavCrafter and ViewCrafter, demonstrating improved geometric consistency and scene coverage.

Geometry-Aware 3D Reconstruction with Enhanced 3DGS

NavCrafter introduces a specialized 3DGS pipeline optimized for the sparse-view, generative regime:

• Depth-Aligned Supervision: Monocular depth estimates (MoGe-2) are scale-calibrated via joint alignment with neural-matching depths, correcting for bias and producing edge-preserving supervision signals for initialization.
• Structural Regularization: Progressive Gaussian dropping (DropGaussian) with adaptive opacity scaling mitigates overfitting and regularizes structure in regions with limited multi-view constraints.
• Multi-View and Diffusion-Based Refinement: Rendered views are iteratively improved using a denoising diffusion module (Difix3D+), providing perceptual and photometric regularization even in ambiguous or occluded areas.

A multi-term objective balances L1 RGB, perceptual (LPIPS), and calibrated-depth losses, explicitly promoting both appearance fidelity and geometric accuracy. NavCrafter outperforms all baselines in single-image 3D scene fidelity and consistency (Table 2).

Figure 4: 3D reconstruction qualitative comparison. Blue boxes: visible input regions; yellow boxes: prior methods' failures. NavCrafter reconstructs both visible and occluded regions with less artifact.

Figure 5: Ablation study. Disabling any major component degrades performance, justifying the integrated design.

Experimental Evaluation

NavCrafter is empirically validated on RealEstate10K, DL3DV, and Tanks & Temples datasets. Across all metrics—LPIPS, PSNR, SSIM for frame similarity; FID, FVD for generative realism; rotation and translation error for guidance accuracy—the framework achieves superior performance, most notably under extreme and out-of-domain viewpoint conditions.

Ablation experiments confirm the critical impact of depth-aligned supervision, structural regularization, and iterative refinement: removing any yields substantial qualitative and quantitative drops.

Theoretical and Practical Implications

NavCrafter demonstrates that strong 3D geometric priors and explicit pose control can be transferred from video diffusion to high-fidelity, single-image scene reconstruction. The framework's progressive expansion and collision-aware planning solve key failure cases seen in prior 3DGS-from-video pipelines, particularly for wide-view, multi-room or outdoor settings with complex geometry.

Practically, this technique unlocks flexible 3D content creation for applications where only a single photograph is accessible—including AR/VR rapid prototyping, robotics spatial reasoning, and visual effects. The theoretical convergence of generative priors, explicit control, and geometry-aware optimization points the way toward fully controllable, scalable 3D scene generation from sparse or unposed imagery.

Conclusion

NavCrafter establishes a new level of quality and controllability for single-image 3D scene reconstruction by tightly integrating camera-guided video diffusion, collision-aware NBV trajectory optimization, and geometry-enhanced 3DGS pipelines. The approach advances both the practical reach and the robustness of generative 3D scene understanding, setting a new standard for future research in controllable neural scene synthesis and single-image-based scene reconstruction.