Stochastic Bundle Adjustment for Efficient and Scalable 3D Reconstruction (2008.00446v1)

Abstract: Current bundle adjustment solvers such as the Levenberg-Marquardt (LM) algorithm are limited by the bottleneck in solving the Reduced Camera System (RCS) whose dimension is proportional to the camera number. When the problem is scaled up, this step is neither efficient in computation nor manageable for a single compute node. In this work, we propose a stochastic bundle adjustment algorithm which seeks to decompose the RCS approximately inside the LM iterations to improve the efficiency and scalability. It first reformulates the quadratic programming problem of an LM iteration based on the clustering of the visibility graph by introducing the equality constraints across clusters. Then, we propose to relax it into a chance constrained problem and solve it through sampled convex program. The relaxation is intended to eliminate the interdependence between clusters embodied by the constraints, so that a large RCS can be decomposed into independent linear sub-problems. Numerical experiments on unordered Internet image sets and sequential SLAM image sets, as well as distributed experiments on large-scale datasets, have demonstrated the high efficiency and scalability of the proposed approach. Codes are released at https://github.com/zlthinker/STBA.

• The paper introduces a stochastic optimization method that reformulates bundle adjustment using clustering and chance constraints to reduce computational complexity.
• It integrates a steepest descent correction to enhance convergence and maintain accuracy in smaller trust regions.
• The approach demonstrates significant scalability and efficiency improvements over traditional LM-based methods in large 3D reconstruction tasks.

Stochastic Bundle Adjustment for Efficient and Scalable 3D Reconstruction

The academic paper presents a novel approach to optimizing the bundle adjustment (BA) process in the domain of 3D reconstruction. It introduces Stochastic Bundle Adjustment (STBA) designed to improve efficiency and scalability, particularly when dealing with large datasets in which current methods face computational bottlenecks.

Summary of Proposed Method

Traditional bundle adjustment algorithms, such as the Levenberg-Marquardt (LM) method, encounter significant computational challenges when scaling up due to the solving of the Reduced Camera System (RCS). The dimension of the RCS is correlated with the number of cameras involved, leading to increased computation and memory overhead. To address these issues, this paper proposes a stochastic approach that breaks down the RCS into manageable sub-problems.

The key innovations in this research include:

1. Reformulation Using Clustering: Instead of addressing the entire RCS, the authors propose a reformulation that employs clustering. By decomposing the visibility graph into clusters, the method introduces equality constraints to manage inter-cluster dependencies, thereby simplifying the large problem into smaller, solvable sub-problems.
2. Chance Constrained Relaxation: The reformulation introduces excess equality constraints. The novel insight is to convert these into chance constraints, allowing for a probabilistic relaxation using a sampled convex program. This approach permits the approximation of the RCS' solution by treating it as a stochastic process, thereby reducing computation complexity.
3. Steepest Descent Correction: To improve convergence, especially in smaller trust regions where deviation from the steepest descent direction could impede progress, a correction step is integrated. This aims to adjust the approximate solutions back towards the steepest descent, enhancing overall algorithm performance.
4. Stochastic Graph Clustering: A stochastic clustering mechanism is introduced to sample chance constraints efficiently, thereby influencing the splitting strategy. This randomization avoids fixed partition pitfalls and allows for iterative improvement of the cluster-based approximation of the RCS.

Numerical Results and Comparison

Extensive numerical experiments are performed, demonstrating the effectiveness of STBA on unordered Internet image sets and sequential SLAM datasets. The results indicate significant improvements in efficiency and scalability compared to traditional LM-based methods. The authors illustrate that STBA can handle larger numbers of cameras by distributing computations across compute nodes, achieving notable time savings and maintaining competitive accuracy in terms of reprojection errors.

Implications and Future Directions

The proposed STBA method provides a practical solution for large-scale 3D reconstruction tasks, where handling massive numbers of images and associated data could otherwise become infeasible. By enabling parallel and distributed computation, the method sets a precedent for future advancements in distributed optimization frameworks within the field of 3D vision.

This research has implications not only for enhancing current 3D reconstruction pipelines and systems but also for other computer vision problems that rely heavily on bundle adjustment. Future development could explore extending stochastic approaches to other components of computer vision algorithms, potentially improving scalability and efficiency in various applications.

In summary, STBA proposes a sophisticated, yet computationally efficient, mechanism for tackling BA challenges in 3D reconstruction, setting a promising course for research in scalable computer vision methods. This method's reliance on stochastic processes and modular problem-solving strategies offers a robust alternative to deterministic approaches, encouraging further exploration in this domain.