Adaptive Sampling-based Particle Filter for Visual-inertial Gimbal in the Wild
Abstract: In this paper, we present a Computer Vision (CV) based tracking and fusion algorithm, dedicated to a 3D printed gimbal system on drones operating in nature. The whole gimbal system can stabilize the camera orientation robustly in a challenging nature scenario by using skyline and ground plane as references. Our main contributions are the following: a) a light-weight Resnet-18 backbone network model was trained from scratch, and deployed onto the Jetson Nano platform to segment the image into binary parts (ground and sky); b) our geometry assumption from nature cues delivers the potential for robust visual tracking by using the skyline and ground plane as a reference; c) a spherical surface-based adaptive particle sampling, can fuse orientation from multiple sensor sources flexibly. The whole algorithm pipeline is tested on our customized gimbal module including Jetson and other hardware components. The experiments were performed on top of a building in the real landscape.
- Adrian Carrio, Hriday Bavle, and Pascual Campoy. “Attitude estimation using horizon detection in thermal images.” International Journal of Micro Air Vehicles 10.4 (2018): 352-361.
- La Place, Cecilia, Aisha Urooj, and Ali Borji. “Segmenting sky pixels in images: Analysis and comparison.” 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2019.
- Damien Dusha, Wageeh Boles, and Rodney Walker. “Fixed-wing attitude estimation using computer vision based horizon detection.” Proceedings of AIAC12: 2nd Australasian Unmanned Air Vehicles Conference. Waldron Smith Management, 2007.
- David Nistér. “An efficient solution to the five-point relative pose problem.” IEEE transactions on pattern analysis and machine intelligence 26.6 (2004): 756-770.
- Madgwick, Sebastian. “An efficient orientation filter for inertial and inertial/magnetic sensor arrays.” Report x-io and University of Bristol (UK) 25 (2010): 113-118.
- Tong Qin, Peiliang Li, and Shaojie Shen. “Vins-mono: A robust and versatile monocular visual-inertial state estimator.” IEEE Transactions on Robotics 34.4 (2018): 1004-1020.
- Zheng Huai, and Guoquan Huang. “Robocentric visual-inertial odometry.” 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018.
- Lukas von Stumberg, and Daniel Cremers. “DM-VIO: Delayed Marginalization Visual-Inertial Odometry.” IEEE Robotics and Automation Letters 7.2 (2022): 1408-1415.
- Paul Furgale, Joern Rehder, Roland Siegwart (2013). “Unified Temporal and Spatial Calibration for Multi-Sensor Systems.” In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan.
- Aytaç Altan, and Rıfat Hacıoğlu. “Model predictive control of three-axis gimbal system mounted on UAV for real-time target tracking under external disturbances.” Mechanical Systems and Signal Processing 138 (2020): 106548.
- Shamsundar Kulkarni, M. D. S. Bormane, and S. L. Nalbalwar. “RANSAC algorithm for matching inlier correspondences in video stabilisation.” International Journal of Signal and Imaging Systems Engineering 10.4 (2017): 178-184.
- Binoy Pinto, and P. R. Anurenjan. “Video stabilization using speeded up robust features.” 2011 International Conference on Communications and Signal Processing. IEEE, 2011.
- Wilbert G. Aguilar, and Cecilio Angulo. “Real-time model-based video stabilization for microaerial vehicles.” Neural processing letters 43.2 (2016): 459-477.
- Junlan Yang, Dan Schonfeld, and Magdi Mohamed. “Robust video stabilization based on particle filter tracking of projected camera motion.” IEEE Transactions on Circuits and Systems for Video Technology 19.7 (2009): 945-954.
- Hany Farid, and Jeffrey B. Woodward. “Video stabilization and enhancement.” (2007).
- Paresh Rawat, and Jyoti Singhai. “Review of motion estimation and video stabilization techniques for hand held mobile video.” Signal & Image Processing: An International Journal (SIPIJ) Vol 2 (2011).
- Jiyang Yu, and Ravi Ramamoorthi. “Learning video stabilization using optical flow.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.
- Oswualdo Alquisiris-Quecha, and Jose Martinez-Carranza. “Video Stabilization of the NAO Robot Using IMU Data.” In Robot Operating System (ROS), pp. 147-162. Springer, Cham, 2020.
- Xueyang, K. A. N. G., and Shunying Yuan. “Integrated visual-inertial odometry and image stabilization for image processing.” U.S. Patent Application 18/035,479, filed December 28, 2023.
- Kang, Xueyang, and Shunying Yuan. “INTEGRATED VISUAL-INERTIAL ODOMETRY AND IMAGE STABILIZATION FOR IMAGE PROCESSING.” U.S. Patent Application 18/035,479, filed December 28, 2023.
- Jutamanee Auysakul, He Xu, and Vishwanath Pooneeth. “A hybrid motion estimation for video stabilization based on an IMU sensor.” Sensors 18.8 (2018): 2708.
- Ahlem Walha, Ali Wali, and Adel M. Alimi. “Video stabilization for aerial video surveillance.” Aasri Procedia 4 (2013): 72-77.
- Ahlem Walha, Ali Wali, and Adel M. Alimi. “Video stabilization with moving object detecting and tracking for aerial video surveillance.” Multimedia Tools and Applications 74.17 (2015): 6745-6767.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
