Multi-camera calibration with pattern rigs, including for non-overlapping cameras: CALICO (1903.06811v3)

Abstract: This paper describes CALICO, a method for multi-camera calibration suitable for challenging contexts: stationary and mobile multi-camera systems, cameras without overlapping fields of view, and non-synchronized cameras. Recent approaches are roughly divided into infrastructure- and pattern-based. Infrastructure-based approaches use the scene's features to calibrate, while pattern-based approaches use calibration patterns. Infrastructure-based approaches are not suitable for stationary camera systems, and pattern-based approaches may constrain camera placement because shared fields of view or extremely large patterns are required. CALICO is a pattern-based approach, where the multi-calibration problem is formulated using rigidity constraints between patterns and cameras. We use a {\it pattern rig}: several patterns rigidly attached to each other or some structure. We express the calibration problem as that of algebraic and reprojection error minimization problems. Simulated and real experiments demonstrate the method in a variety of settings. CALICO compared favorably to Kalibr. Mean reconstruction accuracy error was $\le 0.71$ mm for real camera rigs, and $\le 1.11$ for simulated camera rigs. Code and data releases are available at \cite{tabb_amy_2019_3520866} and \url{https://github.com/amy-tabb/calico}.

• The paper introduces CALICO, a robust method using pattern rigs to calibrate multi-camera systems, even for non-overlapping and asynchronous cameras.
• The methodology leverages rigidity constraints and minimization techniques to compute calibration parameters through iterative initialization and refinement.
• Experimental results demonstrate that CALICO achieves efficient calibration with lower computational overhead and superior performance compared to state-of-the-art methods.

Overview of CALICO: Multi-camera Calibration Method

The research paper introduces CALICO, a robust method for the calibration of multi-camera systems, tailored to various configurations including stationary, mobile, and non-overlapping fields of view and non-synchronized camera setups. This calibration method is particularly useful in contexts such as human activity detection and complex reconstruction tasks where camera fields may not overlap, and synchronization is not always feasible.

Methodology

CALICO is a pattern-based multi-camera calibration approach that addresses the limitations of both infrastructure and existing pattern-based methodologies. Infrastructure-based solutions rely on utilizing features in real-world scenes, which are unsuitable for stationary cameras, while traditional pattern-based methods necessitate overlapping fields of view or large calibration patterns that restrict camera placement. CALICO innovatively employs a pattern rig with multiple patterns rigidly connected, allowing the calibration of diverse camera arrangements through rigidity constraints and algebraic error minimization combined with reprojection error minimization.

The methodology involves several key steps:

1. Individual Camera Calibration: Utilizing Zhang's algorithm to acquire internal parameters by waving a calibration target and estimating extrinsic parameters through PnP pose computation for each camera.
2. Constraint Set Formulation: The multi-camera calibration problem is cast as a set of rigidity constraints between cameras, patterns, and time labels to solve algebraic and reprojection error minimization problems.
3. Initialization and Refinement: CALICO iterates through the constraints to apply closed-form solutions for initializing unknown variables when only one or two remain unassigned per constraint. Subsequently, the algorithm refines these by minimizing reprojection error.
4. Data Handling and Testing: The overall method is supported by simulated datasets and real experiments, where cases ranged from intricate camera network setups (stationary or mobile) to straightforward environments like stereo rigs.

Experimental Evaluation

The paper evaluates CALICO on several datasets including simulations and real-world camera systems (both stationary and mobile) with charuco and April tag patterns. The evaluation metrics include algebraic error, reprojection root mean squared error, and reconstruction accuracy error, providing a comprehensive assessment of CALICO’s performance. The results demonstrate effective calibration with reproducible accuracy across various configurations. Crucially, CALICO's run time is efficient; for large datasets, multi-camera calibration takes merely seconds post individual camera calibration and data loading.

Comparative Analysis

CALICO's proficiency is benchmarked against the Kalibr toolbox, a recognized state-of-the-art method. Tests indicate that CALICO often yields equivalent or superior performance, especially in scenarios where Kalibr faces challenges, such as when the dataset configurations involve non-overlapping views or asynchronous captures. Simultaneously, CALICO provides robust solutions with minimal computational overhead.

Implications and Future Directions

The implications of CALICO are multifaceted, extending reliability and flexibility in multi-camera calibration tasks across disciplines, fostering advancements in areas requiring precise reconstruction from multiple viewpoints such as autonomous systems, motion capture, and surveillance. The method solidifies its niche where conventional approaches encounter limitations, promising enhanced robustness and handling diversity in camera arrangements. Future research could delve into scaling CALICO for higher-dimensional datasets, harnessing advanced optimization strategies, and exploring applications in real-time systems without pre-configured patterns.

This paper lays foundational work for pushing the boundaries of multi-camera calibration, providing both theoretical and practical strides beneficial to the scientific community and applicable to contemporary technological challenges in automated and intelligent systems.