LF-VIO: A Visual-Inertial-Odometry Framework for Large Field-of-View Cameras with Negative Plane (2202.12613v3)

Abstract: Visual-inertial-odometry has attracted extensive attention in the field of autonomous driving and robotics. The size of Field of View (FoV) plays an important role in Visual-Odometry (VO) and Visual-Inertial-Odometry (VIO), as a large FoV enables to perceive a wide range of surrounding scene elements and features. However, when the field of the camera reaches the negative half plane, one cannot simply use [u,v,1]^T to represent the image feature points anymore. To tackle this issue, we propose LF-VIO, a real-time VIO framework for cameras with extremely large FoV. We leverage a three-dimensional vector with unit length to represent feature points, and design a series of algorithms to overcome this challenge. To address the scarcity of panoramic visual odometry datasets with ground-truth location and pose, we present the PALVIO dataset, collected with a Panoramic Annular Lens (PAL) system with an entire FoV of 360{\deg}x(40{\deg}-120{\deg}) and an IMU sensor. With a comprehensive variety of experiments, the proposed LF-VIO is verified on both the established PALVIO benchmark and a public fisheye camera dataset with a FoV of 360{\deg}x(0{\deg}-93.5{\deg}). LF-VIO outperforms state-of-the-art visual-inertial-odometry methods. Our dataset and code are made publicly available at https://github.com/flysoaryun/LF-VIO

• The paper introduces LF-VIO, a framework that fully leverages negative plane features to enhance visual-inertial odometry performance in large FoV cameras.
• The paper presents novel 3D feature representation and algorithmic adaptations, along with the PALVIO dataset, to enable robust real-time processing.
• The paper demonstrates superior performance over state-of-the-art methods, achieving high precision and stability even in dynamically changing environments.

Overview of LF-VIO: A Visual-Inertial-Odometry Framework for Large Field-of-View Cameras with Negative Plane

The paper presents LF-VIO, a real-time Visual-Inertial-Odometry (VIO) framework specifically designed for cameras with a large Field of View (FoV) that encompasses the negative plane. This research addresses the prevalent issue in existing VIO systems where the negative plane, frequently encountered in ultra-wide FoV cameras, cannot be properly utilized, resulting in significant loss of valuable spatial information.

Key Contributions

1. Representation of Feature Points: LF-VIO introduces a novel method for representing image feature points as three-dimensional vectors with unit length, which crucially allows for the exploitation of features across both positive and negative planes of the FoV.
2. Algorithmic Adjustments: The project adapts and designs algorithms to operate effectively with large-FoV camera data, addressing challenges such as rapid increases in $u$ and $v$ values near $180^\circ$ and ensuring stability across key computational tasks like epipolar constraint solving and feature triangulation.
3. PALVIO Dataset: The research introduces the PALVIO dataset, captured using a Panoramic Annular Lens (PAL) system with a FoV of $360^\circ{\times}(40^\circ{\sim}120^\circ)$, complemented by an IMU sensor. This dataset serves as a rich test bed for evaluating and advancing panoramic visual odometry methods.
4. System Performance: LF-VIO demonstrates superior performance compared to state-of-the-art methods on several datasets, including the PALVIO benchmark and a fisheye camera dataset, underscoring its efficacy in handling large-FoV inputs.

Experimental Insights

• Feature Utilization: Analysis reveals the negative plane features are critical for achieving high precision in VIO tasks. When both positive and negative plane features are leveraged, the system's performance in terms of RPEt, RPEr, and ATE significantly improves.
• Robustness Across Conditions: The LF-VIO framework shows robustness and accuracy even in high angular velocity scenarios, where competing methods often struggle. This robustness is particularly important for real-time operations in dynamic environments.
• Efficiency: The implementation achieves real-time processing speeds of around 10Hz on a standard laptop setup, making it feasible for on-the-fly applications in mobile robotics.

Practical and Theoretical Implications

The paper's framework and resulting findings have several implications:

• Practical Applications: LF-VIO's ability to accurately model and interpret scenes using high-FoV cameras positions it well for applications in autonomous driving and mobile robotics, specifically in scenarios requiring comprehensive environmental understanding.
• Advancement of VIO and SLAM Systems: This work pioneers methodologies for enhanced feature utilization in wide-angle imaging, setting a new standard for VIO systems that can potentially be extended to complementary SLAM systems.
• Open Data and Tools: The provision of the PALVIO dataset alongside the LF-VIO framework invites further research and development in the domain, encouraging replication and enhancement of results.

Future Directions

Future research could expand on this foundation by integrating advanced loop closure techniques, enhancing the adaptability of LF-VIO to systems with complex multi-sensor configurations, and exploring real-world challenges such as varying lighting conditions and sensor noise. The framework could also be optimized further to handle more expansive datasets and operational settings, maintaining its robustness across even larger environmental scales.

In summary, LF-VIO marks a significant progression in visual-inertial odometry by proficiently integrating negative plane features, thus offering a robust and scalable solution for applications requiring a holistic view of the environment via large-FoV cameras.