Advances in Global Solvers for 3D Vision

Abstract: Global solvers have emerged as a powerful paradigm for 3D vision, offering certifiable solutions to nonconvex geometric optimization problems traditionally addressed by local or heuristic methods. This survey presents the first systematic review of global solvers in geometric vision, unifying the field through a comprehensive taxonomy of three core paradigms: Branch-and-Bound (BnB), Convex Relaxation (CR), and Graduated Non-Convexity (GNC). We present their theoretical foundations, algorithmic designs, and practical enhancements for robustness and scalability, examining how each addresses the fundamental nonconvexity of geometric estimation problems. Our analysis spans ten core vision tasks, from Wahba problem to bundle adjustment, revealing the optimality-robustness-scalability trade-offs that govern solver selection. We identify critical future directions: scaling algorithms while maintaining guarantees, integrating data-driven priors with certifiable optimization, establishing standardized benchmarks, and addressing societal implications for safety-critical deployment. By consolidating theoretical foundations, practical advances, and broader impacts, this survey provides a unified perspective and roadmap toward certifiable, trustworthy perception for real-world applications. A continuously-updated literature summary and companion code tutorials are available at https://github.com/ericzzj1989/Awesome-Global-Solvers-for-3D-Vision.

• The paper introduces a unified taxonomy categorizing Branch-and-Bound, Convex Relaxation, and Graduated Non-Convexity methods for 3D vision, analyzing trade-offs in optimality, robustness, and scalability.
• It details theoretical principles and algorithmic techniques with comparative benchmarks across ten critical 3D vision tasks such as SLAM, structure-from-motion, and bundle adjustment.
• The survey identifies future challenges in scalability, integration with data-driven priors, and establishing standardized benchmarks for certifiable, robust 3D vision systems.

Advances in Global Solvers for 3D Vision: A Comprehensive Survey

Introduction and Scope

Global optimization frameworks have become fundamental in geometric computer vision, offering certifiable solutions to nonconvex estimation problems underlying applications such as SLAM, structure-from-motion (SfM), registration, and bundle adjustment. This survey introduces a unified taxonomy for global solvers, systematically categorizing them into three main paradigms: Branch-and-Bound (BnB), Convex Relaxation (CR), and Graduated Non-Convexity (GNC). The paper details the theoretical underpinnings, algorithmic techniques, and practical improvements associated with each paradigm, and conducts a comparative analysis across ten critical 3D vision tasks, emphasizing the trade-offs in optimality, robustness, and scalability that shape solver applicability.

Theoretical Principles and Algorithms

Branch-and-Bound (BnB)

BnB is a deterministic method that explores the parameter search space via recursive partitioning and pruning based on provable bounds—guaranteeing global optimality under sufficient refinement. The BnB approach requires careful domain parameterization (e.g., SO(3), SE(3)), tight bounding strategies (Lipschitz, interval arithmetic, convex relaxations), and effective branching and node selection heuristics. For consensus maximization problems (e.g., robustly maximizing inlier sets under outliers), BnB can provide globally optimal solutions up to moderate scales. However, its exponential complexity in parameter dimension fundamentally limits its deployment beyond small- to mid-scale settings, even when advanced bounding or hybrid refinement strategies are employed.

Convex Relaxation (CR)

The two dominant CR frameworks are Shor's relaxation (lifting QCQPs to semidefinite programs, SDPs) and Moment/SOS (sum-of-squares) hierarchies (Lasserre's hierarchy), which address polynomial optimization via a sequence of SDP relaxations. Shor's relaxation, when tight, yields global optima with duality-based optimality certificates. Moment-SOS approaches provide a convergent sequence of lower bounds; for many geometric vision problems, low-order relaxations suffice for tightness, allowing practical global optimality guarantees. Nevertheless, cubic or higher complexity in the matrix variable dimension, and combinatorial explosion with order in the case of SOS, restrict their effective use to moderate-sized instances. Problem-specific convex reformulations (quasi-convex, SOCP, LP, QP) proffer scalability and exactness for certain task-specific loss functions (e.g., $L_\infty$ norms), but lack the modeling generality of SDP-based methods.

Graduated Non-Convexity (GNC)

GNC methods use deterministic continuation strategies to track a solution from an initial convex surrogate to the original nonconvex robust objective via a homotopy parameter. At each stage, an iteratively reweighted least-squares (IRLS) problem is solved with progressively stronger outlier suppression. While lacking global optimality certificates, GNC enables robust, large-scale estimation (outlier ratios of $30$–$50\%$) with near-global empirical performance over diverse tasks. Recent advances include adaptive annealing, minimally tuned GNC, and integration with neural or data-driven schedules.

Comparative Analysis: Trade-Offs and Deployment

Method	Optimality	Outlier Handling	Complexity/Memory	Scalability	Certificates
Branch-and-Bound	Global (ε-optimal)	CM	Exponential	Low ($n<20$)	Yes
Shor's Relaxation	Global (if tight)	M-est	$O(n^{3.5})$	Medium ($N<1\text{k}$)	Yes (tight)
Moment-SOS	Global (if tight)	M-est, CM	$O(r^{3.5}), r\sim\binom{n+d}{d}$	Low/Medium ($n<100$)	Yes (tight)
Task-Specific Relaxations	Global/Near-global	Variable	$O(n^3)\sim O(n)$	High (problem class)	Problem
GNC	Near-global (empirical)	Built-in	$O(Kn^2)$	Excellent ($N\gg1\text{k}$)	No

BnB provides the strongest guarantees but is limited by exponential scaling; CR methods balance certifiability and practical scalability, with tightness dependent on instance structure. Task-specific relaxations achieve high scalability, but require fit-to-structure assumptions. GNC offers best-in-class robustness and scalability absent certificates.

Applications Across 3D Vision Tasks

The survey systematically reviews the application of global solvers to ten canonical tasks: Wahba problem, vanishing point estimation, absolute/relative pose estimation, 3D registration, rotation/translation averaging, triangulation, pose graph optimization, and bundle adjustment. For each, the interplay between algebraic structure (e.g., rotations, graph, manifold, bilinear constraints), the prevalence of noise/outliers, and the required optimality/robustness/scalability trade-off determines the optimal solver paradigm.

For example, modern SDP-based relaxations attain certifiable global optima in rotation averaging and pose graph optimization on moderate scales under good graph connectivity and bounded noise (tight relaxation). GNC methods match and often surpass RANSAC in robust pose estimation and registration when dealing with severe outlier contamination, enabling scalable, dataset-wide optimization.

Future Directions and Open Challenges

The paper identifies four critical areas for future inquiry:

• Scalability under guarantees: Algorithmic innovations to address the exponential or combinatorial blow-ups in BnB and SDP-based approaches—such as problem structure exploitation (e.g., chordal decomposition), distributed solvers, and first-order approaches for very large SDPs.
• Integration with data-driven priors: Bridging learned priors and certifiable optimization—e.g., warm-starting global solvers with neural estimators, or incorporating deep representations as constraints within relaxation frameworks.
• Standardized benchmarks and reproducibility: Need for rigorous, community-adopted benchmarks evaluating different global solvers on certifiability, scalability, and robustness axes, plus open-source baseline implementations and code tutorials.
• Societal and safety impacts: Addressing the deployment of global solvers in safety-critical, real-world systems, ensuring transparent, explainable modeling, and robust handling of adversarial or non-stationary input regimes.

Conclusion

This survey unifies global solvers for 3D vision under a principled taxonomy, explicating their theoretical properties, algorithmic realizations, trade-offs, and empirical performance across foundational geometric estimation problems. The authors emphasize that bridging the gap between global optimality guarantees and scalable, robust deployment—especially in safety- or mission-critical perception—remains a compelling and evolving challenge. By consolidating theoretical, algorithmic, and practical advances, the survey provides an essential roadmap and reference for certifiable, trustworthy computer vision under uncertainty and nonconvexity.

---

Reference:

"Advances in Global Solvers for 3D Vision" (2602.14662)