Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction (2308.10694v1)

Abstract: We tackle the problem of estimating a Manhattan frame, i.e. three orthogonal vanishing points, and the unknown focal length of the camera, leveraging a prior vertical direction. The direction can come from an Inertial Measurement Unit that is a standard component of recent consumer devices, e.g., smartphones. We provide an exhaustive analysis of minimal line configurations and derive two new 2-line solvers, one of which does not suffer from singularities affecting existing solvers. Additionally, we design a new non-minimal method, running on an arbitrary number of lines, to boost the performance in local optimization. Combining all solvers in a hybrid robust estimator, our method achieves increased accuracy even with a rough prior. Experiments on synthetic and real-world datasets demonstrate the superior accuracy of our method compared to the state of the art, while having comparable runtimes. We further demonstrate the applicability of our solvers for relative rotation estimation. The code is available at https://github.com/cvg/VP-Estimation-with-Prior-Gravity.

• The paper introduces two new minimal solvers for vanishing point estimation in uncalibrated images, requiring only two lines and leveraging a prior gravity direction.
• A hybrid RANSAC framework integrates these new two-line solvers with existing ones to enhance robustness and adaptability, especially with noisy gravity priors.
• Extensive experiments demonstrate that the new solvers and framework improve accuracy and robustness across various datasets, with implications for autonomous systems and consumer electronics.

Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction

The paper presents a novel approach to vanishing point (VP) estimation in uncalibrated images, focusing on scenarios with a known gravity direction. The authors introduce two new minimal solvers for VP estimation that utilize only two lines along with a prior gravity direction, specified via Inertial Measurement Units (IMUs) now common in consumer devices. This advancement addresses some singularities and deficiencies faced by pre-existing methods.

Minimal Solvers and Problem Formulation

The challenge tackled involves determining a Manhattan frame—with three orthogonal vanishing points—alongside the camera's unknown focal length, all predicated on a known vertical direction. Traditional methods in uncalibrated settings typically rely on four lines for calibration. The contribution of two new solvers, which require only two input lines, represents a significant refinement in problem-solving methodology, increasing both computational efficiency and practicality in real-world applications.

Outlined as pivotal, the paper's two-line solvers avoid the singularities inherent in earlier models. These solvers leverage minimal line configurations from which the camera rotation and focal length can be derived accurately. Beyond the minimal approach, a non-minimal solution is introduced to refine the results, enhancing local optimization even when applied to a larger set of lines.

Hybrid RANSAC Framework

A hybrid RANSAC framework is proposed, integrating the newly formulated solvers with pre-existing ones to achieve robustness and adaptability across a spectrum of scenarios with varying prior information accuracy. This hybrid approach is particularly adept at dealing with rough priors, thereby broadening the scope of applications where it can be effectively deployed.

Experimental Results and Implications

The utility of these new solvers is demonstrated through extensive experimental evaluations on both synthetic and real-world datasets, including York Urban and ScanNet. The results underscore enhanced accuracy in VP detection, even when the input gravity direction carries inherent noise, validating the solvers' robustness. The hybrid RANSAC framework further affirms its superiority, ensuring better performance outcomes through seamless solver integration.

This research has practical implications across several applications, notably in autonomous systems and consumer electronics involving cameras such as smartphones and augmented reality devices, where continuous recalibration is infeasible, and traditional camera parameters unavailable. The approach is particularly aligned with use cases involving upright images or where achieving exact camera calibration is challenging or impossible.

Future Directions

For future explorations, extending this approach to more generalized scenarios, such as integrating additional known orientation constraints like horizon lines or addressing unavailability of gravity directions, will likely extend its applicability further. Moreover, as computational processing continues to advance, further refinement of these solvers towards real-time applications remains an intriguing prospect.

In conclusion, the paper contributes valuable advancements in the domain of VP estimation for uncalibrated images by introducing efficient solvers that reduce the required input and leverage accessible device sensors like IMUs, all within an adaptable and robust framework capable of delivering enhanced precision across various datasets and practical scenarios.