PARSAC: Accelerating Robust Multi-Model Fitting with Parallel Sample Consensus (2401.14919v1)

Abstract: We present a real-time method for robust estimation of multiple instances of geometric models from noisy data. Geometric models such as vanishing points, planar homographies or fundamental matrices are essential for 3D scene analysis. Previous approaches discover distinct model instances in an iterative manner, thus limiting their potential for speedup via parallel computation. In contrast, our method detects all model instances independently and in parallel. A neural network segments the input data into clusters representing potential model instances by predicting multiple sets of sample and inlier weights. Using the predicted weights, we determine the model parameters for each potential instance separately in a RANSAC-like fashion. We train the neural network via task-specific loss functions, i.e. we do not require a ground-truth segmentation of the input data. As suitable training data for homography and fundamental matrix fitting is scarce, we additionally present two new synthetic datasets. We demonstrate state-of-the-art performance on these as well as multiple established datasets, with inference times as small as five milliseconds per image.

• The paper introduces PARSAC, a neural network-augmented method that predicts sample and inlier weights for parallel model recovery.
• It leverages two synthetic datasets, HOPE-F and SMH, to train and evaluate performance on homography and fundamental matrix fitting tasks.
• The method achieves a fivefold speed increase over traditional RANSAC variants, processing images in as little as five milliseconds.

Introduction

In computer vision, robustly estimating geometric models from observational data contaminated with outliers is a cornerstone for numerous applications, including 3D scene analysis, Structure-from-Motion (SfM), and Simultaneous Localization and Mapping (SLAM). Conventional methods, such as Random Sample Consensus (RANSAC) and its derivatives, have been the go-to approaches, despite the computational complexity associated with handling multiple geometric models. In recent developments, the integration of machine learning has opened pathways to more efficient and robust strategies for multi-model fitting.

Parallel Sample Consensus (PARSAC)

A paradigm shift brought forward by the latest research leverages neural networks to segment input data, paving the way for the parallelisation of geometric model discovery. The PARSAC method diverges from its predecessors by predicting sample and inlier weights for all model instances simultaneously. This neural network-augmented approach allows for RANSAC-like model recovery, albeit independently and in parallel for each potential instance. The neural network employed is trained through task-specific loss functions, negating the necessity for extensive ground truth, which can be particularly scarce for certain tasks like homography and fundamental matrix fitting.

Datasets: HOPE-F and SMH

Recognising the limitations imposed by the dearth of suitable training data, the researchers behind PARSAC contribute two synthetic extensive datasets, HOPE-F and Synthetic Metropolis Homographies (SMH), constructed specifically for training in fundamental matrix and homography fitting tasks. These datasets surpass existing ones like AdelaideRMF in volume and variety, offering a significant quantity of annotated instances and a broad range of noise and outlier scenarios—ideal conditions for developing sophisticated multi-model estimators.

Performance and Efficiency

PARSAC is tested against several state-of-the-art techniques across various datasets, including the newly introduced synthetic datasets, as well as standard benchmarks like SU3 and NYU-VP. Notably, PARSAC consistently achieves dominant performance while setting a new standard in computational efficiency. On tasks such as vanishing point estimation, it markedly outperforms competitive methods, clocking inference times as short as five milliseconds per image. This represents a fivefold speed increase over the second-fastest technique and is magnitudes faster than models with comparable accuracy.

Conclusion

The introduction of PARSAC marks a substantial advancement in robust multi-model fitting techniques. By facilitating the parallel discovery of geometric models, PARSAC considerably speeds up the process without sacrificing accuracy. The proposed approach demonstrates superior performance over existing methods and introduces substantial datasets for the research community. Such contributions are poised to significantly influence future developments in 3D scene analysis and related fields.