A Pose-only Geometric Constraint for Multi-Camera Pose Adjustment

Abstract: Multi-camera systems offer rich observation capabilities for visual navigation and 3D scene reconstruction; however, the resulting feature redundancy often compromises computational efficiency. This challenge is particularly pronounced during bundle adjustment, where the non-linear optimization of both system poses and scene points incurs substantial computational overhead. To address this challenge, this paper introduces a pose-only geometric constraint for multi-camera systems and proposes a corresponding pose adjustment algorithm. Specifically, we use generalized camera model to establish a unified representation of the multi-camera system. Building upon this model, we formulate the multi-camera pose-only constraint, which implicitly represents a 3D scene point using two base observations and their associated poses, thereby achieving a pose-only representation of the projection geometry. Subsequently, we introduce a multi-camera pose adjustment algorithm that eliminates 3D points from the parameter space, thereby achieving efficient and focused pose optimization. Experimental results on both synthetic and real-world datasets demonstrate that the proposed algorithm outperforms baseline bundle adjustment methods in computational efficiency, while maintaining or even improving pose estimation accuracy.

• The paper introduces a novel pose-only constraint that eliminates explicit 3D point parameters from traditional multi-camera bundle adjustment.
• It employs uncertainty-ellipsoid-driven base observation pair selection to minimize reconstruction error and computational complexity.
• Experimental results on synthetic and real-world datasets confirm improved accuracy, scalability, and efficiency in multi-camera pose estimation.

Pose-only Geometric Constraint for Multi-Camera Pose Adjustment

Introduction and Motivation

The presented work systematically addresses the high computational overhead in multi-camera bundle adjustment caused by redundant features and high-dimensional parameter spaces. The core contribution is the introduction of a multi-camera pose-only geometric constraint, formulated within a unified generalized camera model framework. By implicitly representing 3D structure via pairs of base observations and associated poses, the approach eliminates explicit 3D point variables from the optimization. This contrasts with conventional bundle adjustment, which jointly optimizes poses and high-cardinality 3D points, resulting in considerable efficiency improvements without sacrificing accuracy.

Generalized Camera Model and Pose-only Constraint

The generalized camera model offers a coherent mathematical foundation to represent any multi-camera configuration, including those with non-overlapping or overlapping fields of view. It encapsulates each observation as a 6D tuple $(\mathbf{f}, \mathbf{v})$, where $\mathbf{f}$ encodes the normalized observation direction and $\mathbf{v}$ the origin of the ray. Using this model, the paper constructs a pose-only constraint: for any scene point, there exists a pair of base observations such that all other observations of the point are deterministic functions of these bases and their associated poses, omitting the 3D point coordinate from the parameter space.

Figure 1: Schematic diagram of multi-camera pose-only constraint in the generalized camera model; each observation maps uniquely into a 6D vector, allowing projection geometry to be formulated in pose-only space.

This representation leads to a set of tightly coupled geometric constraints over multi-camera system poses, substantially reducing the problem's dimensionality. Furthermore, it provides an analytic mechanism to eliminate scene point parameters and facilitates construction of efficient pose optimization.

Base Observation Pair Selection

Selection of the optimal base observation pair is formalized as a critical component in minimizing reconstruction uncertainty. The authors propose a strategy leveraging the roundness of the uncertainty ellipsoid—estimated via first-order error propagation in ray intersection through covariance analysis. The observed point pair yielding maximal ellipsoid roundness (defined as $\sqrt{\lambda_{3}/\lambda_{1}}$ on the eigenvalue spectrum) is selected, maximizing geometric conditioning of the underlying linear system.

Figure 2: Visualization of 3D point uncertainty as an ellipsoid; roundness quantifies pair suitability for base observation selection.

This approach is empirically shown to outperform alternative heuristics (random selection, max-disparity, max-angle) and joint multi-view strategies, the latter being susceptible to error amplification with noisy or redundant observations.

Statistically Optimal Reconstruction

Given optimized poses, scene reconstruction is achieved by minimizing the Euclidean or Mahalanobis distances between the reconstructed point and all observation rays, projected into the nullspace of their directions. The resulting estimator accounts for the direction's full covariance, producing a statistically optimal (in the sense of minimum Mahalanobis cost) closed-form solution for generalized multi-view triangulation.

Algorithmic Properties and Implementation

The multi-camera pose adjustment algorithm is built around the previously defined constraint set. The optimization is performed using Levenberg-Marquardt or Gauss-Newton, with closed-form analytic Jacobians supplied for efficient computation. The authors also discuss practical computation strategies for the normal equations, reducing unnecessary matrix operations via blockwise reuse.

A cost function is derived that depends solely on pose variables, optionally leveraging both left- and right-base observations to construct doubly-constrained systems (offering a speed-accuracy trade-off). This removes the Schur complement's dependence on the large 3D point blocks, dramatically reducing memory usage.

Experimental Evaluation

Synthetic Data

Comprehensive evaluation on synthetic datasets (varying both camera arrangements and motion/trajectory regimes) demonstrates strong robustness and consistent improvements over MultiCol, BACS, and other common bundle adjustment frameworks. Notably, the pose-only approach attains lower median triangulation errors even compared to multi-view triangulation using all available observations, particularly under increasing measurement noise.

Figure 3: Two representative extrinsic configurations (forward, omnidirectional) for the tested synthetic multi-camera systems.

Figure 4: Four synthetic datasets derived from combinations of camera configuration and system trajectory, providing diverse test environments.

Base observation selection studies show that the uncertainty-ellipsoid-driven method consistently outperforms baselines, and that carefully chosen base pairs actually yield better-conditioned reconstruction than using large, redundant image sets.

Figure 5: Median triangulation error as a function of observation selection strategy; roundness-based selection (proposed) achieves lowest error.

Further, strong numerical results are presented on pose estimation accuracy (rotation, translation, and reconstruction error), indicating that the pose-only adjustment is consistently more robust to outlier and noise levels.

Figure 6: Comparative accuracy of different triangulation methods under synthetic noise; the proposed statistically optimal triangulation is superior for all tested noise regimes.

Runtime and Memory Analysis

Tabulated results across multiple problem scales confirm substantial runtime and memory savings: for large-scale problems, the memory footprint sees a reduction of up to 19x, and runtime improves by factors exceeding 2.5x in comparison with bundle-adjustment-based competitors. This efficiency arises directly from the reduced optimization parameter space enabled by the pose-only parametrization.

Real-world Data (ETH3D & KITTI)

Experiments on the ETH3D and KITTI Odometry datasets validate the practical efficacy of the proposed algorithms for demanding real scenarios. The methods demonstrate superior or competitive accuracy (in rotation error, translation error, and reprojection error) while scaling efficiently to data volumes in the millions of observations.

For example, in challenging scenes (low texture, complex illumination), the approach yields smoother, better-aligned trajectories and visibly improved point cloud reconstructions.

Figure 7: Visual and numerical comparison on ETH3D sequences; the proposed pose-only adjustment more accurately aligns trajectories to ground truth and improves scene fidelity across diverse environments.

Figure 8: KITTI trajectory and reconstruction results pre- and post-pose-only adjustment, evidencing major gains in geometric clarity and drift reduction.

Ablation over initialization strategies, including realistic pipeline-inferred initial poses with significant accumulated drift, confirms that the pose-only method robustly improves both global consistency and accuracy, outperforming traditional methods in all metrics.

Implications and Future Directions

The introduced pose-only adjustment has both theoretical and practical implications. The main theoretical insight is that, in multi-camera settings, explicit parameterization of 3D points can be dispensed with entirely in favor of implicitly pose-anchored, observation-driven geometries, without loss of information or accuracy. This opens the door to scalable, robust real-time visual SLAM and multi-view reconstruction pipelines that can incorporate richer, redundant sensor configurations without being bottlenecked by combinatoric parameter growth.

Practically, the efficiency and scalability gains will be directly beneficial in SLAM, large-scale SFM, and calibration tasks where multi-camera rigs are common. The modular nature of base observation selection also suggests future work on robust, dynamic selection under data association uncertainty or partial calibration.

An open research direction is joint optimization of calibration and pose parameters, especially for large, time-varying multi-camera rigs where extrinsics may drift or be unknown.

Conclusion

The pose-only geometric constraint for multi-camera pose adjustment presents a unifying framework for efficient, accurate, and scalable pose optimization. By leveraging the generalized camera model, carefully conditioned base observation selection, and statistically optimal reconstruction, the approach outperforms bundle adjustment baselines in computational and accuracy metrics. As multi-sensor systems and visual mapping pipelines continue to expand, this methodology offers a foundation for future research in robust, large-scale visual geometry estimation.