Geometry-Guided Progressive Fusion

• Geometry-guided progressive fusion is a method that integrates explicit geometric priors with appearance and semantic features through iterative, stage-wise fusion to improve accuracy and consistency.
• It employs information-theoretic measures like Expected Information Gain to localize and reduce local misalignment and hallucination, effectively addressing sensor degradation and viewpoint shifts.
• Multi-level fusion structures, including voxel/BEV duality, multi-scale registration, and structured prior injection, enable state-of-the-art performance across 3D object detection, image restoration, and molecular representation tasks.

Geometry-guided progressive fusion encompasses a family of techniques in which explicit geometric representations are fused progressively, often in cycles or stages, with appearance, semantic, or other modality-driven features. The strategy exploits geometric consistency to guide the fusion, regularization, or restoration processes in tasks such as 3D scene reconstruction, image registration, multimodal perception, molecular representation, and inpainting. Notable frameworks—including FaithFusion, ProFusion3D, C-MPDR, MuMo, and DDHFusion—deploy this paradigm to balance fidelity, flexibility, and robustness across demanding benchmarks. The following sections synthesize the principal methodologies and theoretical frameworks.

1. Unified Principles and Definitions

Geometry-guided progressive fusion is defined by the intertwined evolution of geometric and non-geometric (e.g., appearance, semantic, contextual) information streams, mediated by iterative, stage-wise, or uncertainty-weighted mechanisms. A core property is that geometry does not simply reside in a preprocessing or feature-extraction phase but continuously steers the fusion process via:

• Spatially explicit priors (e.g., depth, normals, structural constraints)
• Progressive correction or injection (e.g., information-gain maps, curriculum training, multi-scale field updates)
• Feedback or looped schemes where updated reconstructions or latent states recursively inform alignment or synthesis

In contrast to static fusion, progressive methods address local misalignment, hallucination, drift, and modality collapse by periodically re-aligning or re-weighting fusion with respect to geometry-aware signals (Wang et al., 26 Nov 2025, Mohan et al., 2024, Wang et al., 2023, Jing et al., 24 Oct 2025, Hu et al., 12 Mar 2025).

2. Information-theoretic and Uncertainty-based Guidance

A central methodological innovation is the use of information-theoretic constructs, such as pixel-wise Expected Information Gain (EIG), to localize and quantify geometric uncertainty for targeted fusion. In FaithFusion, EIG is defined as

$$\mathrm{EIG}(\Omega;Y^{\rm NVS}|X^{\rm NVS}) = \mathbb{H}[\Omega] - \mathbb{E}_{p(Y|X)}\bigl[\mathbb{H}[\Omega | Y^{\rm NVS}, X^{\rm NVS}]\bigr]$$

where $\Omega$ is the Laplace-approximated posterior over geometric parameters (e.g., 3D Gaussian Splatting), and $\mathbb{H}[\cdot]$ denotes differential entropy. Under Laplace and log-determinant approximations, EIG admits a trace-form upper bound per pixel, efficient for per-frame evaluation (Wang et al., 26 Nov 2025).

The EIG map acts as a spatial prior, injected into dual-branch diffusion (EIGent) and as a pixel-level weighting for distilling hallucinated content back into the geometric model via EIG-weighted loss. This approach restricts hallucination to genuinely under-constrained regions, maintaining geometric fidelity even under large viewpoint shifts.

Related strategies use adaptive uncertainty or error estimation to steer fusion—e.g., in C-MPDR, the residual misalignment at each multiscale stage is addressed by co-evolving deformation fields and feature refinements, with explicit channel and spatial attention (Wang et al., 2023).

3. Multi-level and Multi-stage Progressive Fusion Structures

Methods instantiate progressive geometry guidance at multiple architectural levels:

• Voxel/BEV Duality: In 3D object detection (DDHFusion, ProFusion3D), features are represented both in voxel (volumetric) and BEV (Bird’s Eye View) domains. Dual-stage fusion networks—Homogeneous Voxel Fusion (HVF) and BEV Fusion (HBF)—combine features intra- and cross-modally using Mamba blocks (structured state-space models), aligning fine geometric cues with dense semantic context (Hu et al., 12 Mar 2025, Mohan et al., 2024).
• Multi-scale Registration and Fusion: C-MPDR decomposes geometric alignment into a progressive series of deformation field (DF) sub-updates, with each added incrementally and adaptively weighted, alternating with feature refinement stages. This eliminates the dichotomy of hand-crafted coarse-to-fine registration and static fusion, embedding dense geometry correction within the fusion loop (Wang et al., 2023).
• Hierarchical Windowed Attention: ProFusion3D fuses multi-modal (LiDAR, camera) data at both intermediate (pixel/voxel) and query levels within both BEV and perspective view (PV) domains. Local windowed and global-dilated attention modules are stacked and combined with transformer decoders anchored by explicit 3D positional encodings (Mohan et al., 2024).
• Structured Prior Injection: In molecular representation (MuMo), topological and geometric descriptors are unified in a structured prior, which is injected into the sequence stream in a progressive, layer-wise (gated) fashion, preserving sequence/graph autonomy at early stages while enabling geometric enrichment later (Jing et al., 24 Oct 2025).

4. Task-specific Loss Formulations and Geometry-weighted Reduction

All leading frameworks derive their loss functions to directly encode geometric priorities or alignments:

• Weighted Reconstruction Losses: FaithFusion’s image loss for novel views is modulated by the normalized EIG map, preserving existing geometry in low-uncertainty regions and focusing training updates on high-uncertainty ones. The total trajectory loss further incorporates depth loss with a tunable $\lambda_d$ weighting (Wang et al., 26 Nov 2025).
• Multi-task and Attention-driven Losses: EGSA-PT fuses edge-guided spatial attention with a progressive curriculum over edge maps (first RGB, then predicted depth), with loss terms for per-pixel depth, gradient, and normal consistency; total loss combines depth and segmentation with empirically optimized weights (Omotara et al., 18 Nov 2025).
• Self-supervised Multi-modal Objectives: ProFusion3D applies mask modeling over both camera and LiDAR tokens (with asymmetric ratios), optimizing masked token reconstruction, denoising, and cross-modal attribute matching, which regularize both spatial structure and cross-modal consistency (Mohan et al., 2024).
• Joint Registration–Fusion Optimization: C-MPDR co-trains its deformation field and fusion components, combining bidirectional feature similarity with field smoothness and multi-scale SSIM, gradient, and saliency-weighted fidelity losses (Wang et al., 2023).

5. Algorithmic Summaries and Pseudocode Foundations

A hallmark of geometry-guided progressive fusion is explicit, iterative algorithms and loop structures. For instance, the FaithFusion training loop (Algorithm 2) expands the training trajectory iteratively (e.g., lane shifts in driving scenes), renders novel-view frames, computes per-pixel EIG, applies diffusion-based restoration constrained by EIG, and finally fine-tunes the 3DGS using EIG-weighted losses. The process is repeated until the augmented trajectory covers the application-specified spatial envelope (Wang et al., 26 Nov 2025).

In DDHFusion, query selection for detection proceeds in stages—first extracting easy queries via heatmap peaks with non-max suppression, then hard queries after masking out the easy ones and activating the remainder with cross-attended object relations (HIA). The progressive decoder then sequentially merges context-rich BEV and geometry-aware voxel features for precise classification and bounding box regression (Hu et al., 12 Mar 2025).

6. Empirical Validation and Benchmark Outcomes

State-of-the-art results are demonstrated across multiple domains:

• FaithFusion achieves NTA-IoU = 0.581 and FID = 71.51 at 3m, and FID = 107.47 at 6m lane shift on Waymo, outperforming architectures with additional priors or structural modifications (Wang et al., 26 Nov 2025).
• ProFusion3D achieves 71.1% mAP and 73.6% NDS on nuScenes, 37.7% mAP on Argoverse2. It demonstrates resilience to sensor loss and superior data efficiency under label scarcity (Mohan et al., 2024).
• C-MPDR consistently improves mean squared error and mutual information metrics in multi-modality image fusion, outperforming prior state-of-the-art in both alignment and fusion, and robust under severe simulated deformation (Wang et al., 2023).
• MuMo surpasses baseline molecular models by 2.7% on average across 29 tasks, with ablations confirming the necessity of progressive injection and structured geometric fusion (Jing et al., 24 Oct 2025).
• DDHFusion attains 71.6 mAP and 73.8 NDS on nuScenes, with ablations confirming the joint contribution of voxel and BEV domain fusion, stage-wise query induction, and progressive decoder design (Hu et al., 12 Mar 2025).

7. Implications, Limitations, and Outlook

Geometry-guided progressive fusion frameworks eliminate reliance on handcrafted priors, global optimization, or invasive architecture changes, often preserving modularity and plug-and-play deployment. Nevertheless, limitations persist in cases of non-diffeomorphic geometric distortions, severe sensor degradation, or extreme class imbalance. Future work may extend progressive injection schemes to additional modalities, further automate uncertainty quantification, or accelerate computation via model compression. The explicit coupling of geometry and feature fusion is now established as pivotal for achieving top-tier performance in 3D perception, cross-modal synthesis, and physically-grounded generative modeling (Wang et al., 26 Nov 2025, Mohan et al., 2024, Wang et al., 2023, Jing et al., 24 Oct 2025, Hu et al., 12 Mar 2025).