Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Visual Inertial Odometry using Focal Plane Binary Features (BIT-VIO) (2403.09882v1)

Published 14 Mar 2024 in cs.RO

Abstract: Focal-Plane Sensor-Processor Arrays (FPSP)s are an emerging technology that can execute vision algorithms directly on the image sensor. Unlike conventional cameras, FPSPs perform computation on the image plane -- at individual pixels -- enabling high frame rate image processing while consuming low power, making them ideal for mobile robotics. FPSPs, such as the SCAMP-5, use parallel processing and are based on the Single Instruction Multiple Data (SIMD) paradigm. In this paper, we present BIT-VIO, the first Visual Inertial Odometry (VIO) which utilises SCAMP-5.BIT-VIO is a loosely-coupled iterated Extended Kalman Filter (iEKF) which fuses together the visual odometry running fast at 300 FPS with predictions from 400 Hz IMU measurements to provide accurate and smooth trajectories.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Ceres Solver, 2023. https://github.com/ceres-solver/ceres-solver.
  2. Robust visual inertial odometry using a direct EKF-based approach. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 298–304. IEEE, 2015.
  3. Visual Odometry for Pixel Processor Arrays. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 4614–4622. IEEE, October 2017.
  4. A 100,000 FPS vision sensor with embedded 535gops/w 256×\times× 256 SIMD processor array. In 2013 symposium on VLSI circuits, pages C182–C183. IEEE, 2013.
  5. Weighted node mapping and localisation on a pixel processor array. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 6702–6708. IEEE, 2021.
  6. Feature extraction using a portable vision system. 2017.
  7. Scamp5d vision system and development framework. In Proceedings of the 12th International Conference on Distributed Smart Cameras, pages 1–2, 2018.
  8. Proximity estimation using vision features computed on sensor. In 2020 IEEE international conference on robotics and automation (ICRA), pages 2689–2695. IEEE, 2020.
  9. Camera tracking on focal-plane sensor-processor arrays. In High Performance and Embedded Architecture and Compilation (HiPEAC), Workshop on Programmability and Architectures for Heterogeneous Multicores (MULTIPROG), 2019.
  10. AUKE: Automatic Kernel Code Generation for an Analogue SIMD Focal-Plane Sensor-Processor Array. ACM Transactions on Architecture and Code Optimization, 15:1–26, 01 2019.
  11. Piotr Dudek. SCAMP-3: A vision chip with SIMD current-mode analogue processor array. Focal-plane sensor-processor chips, pages 17–43, 2011.
  12. On-manifold preintegration for real-time visual–inertial odometry. IEEE Transactions on Robotics, 33(1):1–21, 2016.
  13. SVO: Semidirect visual odometry for monocular and multicamera systems. IEEE Transactions on Robotics, 33(2):249–265, 2016.
  14. Continuous-time batch estimation using temporal basis functions. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 2088–2095, St. Paul, MN, 2012.
  15. Unified temporal and spatial calibration for multi-sensor systems. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan, 2013.
  16. Tracking control of a UAV with a parallel visual processor. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4248–4254. IEEE, 2017.
  17. Perspective correcting visual odometry for agile MAVs using a pixel processor array. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 987–994. IEEE, 2018.
  18. Towards drone racing with a pixel processor array. In Proceeding of 11th International Micro Air Vehicle Competition and Conference, IMAV 2019, pages 76–82, 2019.
  19. M. Grupp. evo: Python package for the evaluation of odometry and slam, 2017. https://github.com/MichaelGrupp/evo.
  20. iSAM2: Incremental smoothing and mapping using the bayes tree. The International Journal of Robotics Research, 31(2):216–235, 2012.
  21. J. Kannala and S. Brandt. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8):1335–1340, 2006.
  22. Parallel tracking and mapping for small ar workspaces. In 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pages 225–234, 2007.
  23. Keyframe-based visual–inertial odometry using nonlinear optimization. The International Journal of Robotics Research, 34(3):314–334, 2015.
  24. A 128×\times×128 120 db 15μ𝜇\muitalic_μs latency asynchronous temporal contrast vision sensor. IEEE journal of solid-state circuits, 43(2):566–576, 2008.
  25. Direct servo control from in-sensor CNN inference with a pixel processor array. arXiv preprint arXiv:2106.07561, 2021.
  26. Globally consistent range scan alignment for environment mapping. Autonomous robots, 4:333–349, 1997.
  27. A robust and modular multi-sensor fusion approach applied to MAV navigation. In 2013 IEEE/RSJ international conference on intelligent robots and systems, pages 3923–3929. IEEE, 2013.
  28. Self-supervised calibration for robotic systems. In Proc. of the IEEE Intelligent Vehicles Symposium (IVS), 2013.
  29. Visual odometry using pixel processor arrays for unmanned aerial systems in GPS denied environments. Frontiers in Robotics and AI, 7:126, 2020.
  30. A multi-state constraint Kalman filter for vision-aided inertial navigation. In Proceedings 2007 IEEE international conference on robotics and automation, pages 3565–3572. IEEE, 2007.
  31. Continuous-time visual-inertial odometry for event cameras. IEEE Transactions on Robotics, 34(6):1425–1440, 2018.
  32. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE transactions on robotics, 31(5):1147–1163, 2015.
  33. High-frame rate homography and visual odometry by tracking binary features from the focal plane. Autonomous Robots, Jul 2023.
  34. BIT-VO: Visual odometry at 300 FPS using binary features from the focal plane. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8579–8586. IEEE, 2020.
  35. David Nistér. An efficient solution to the five-point relative pose problem. IEEE transactions on pattern analysis and machine intelligence, 26(6):756–770, 2004.
  36. Vins-mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics, 34(4):1004–1020, 2018.
  37. Real-time visual-inertial odometry for event cameras using keyframe-based nonlinear optimization. 2017.
  38. Extending kalibr: Calibrating the extrinsics of multiple imus and of individual axes. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 2016.
  39. Machine learning for high-speed corner detection. In Computer Vision–ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings, Part I 9, pages 430–443. Springer, 2006.
  40. Navigating the landscape for real-time localization and mapping for robotics and virtual and augmented reality. Proceedings of the IEEE, 106(11):2020–2039, 2018.
  41. Aerial robots, visual-inertial odometry of. 2020.
  42. Compiling CNNs with Cain: focal-plane processing for robot navigation. Autonomous Robots, 46(8):893–910, 2022.
  43. Cain: Automatic code generation for simultaneous convolutional kernels on focal-plane sensor-processors. In International Workshop on Languages and Compilers for Parallel Computing, pages 181–197. Springer, 2020.
  44. A benchmark for the evaluation of RGB-D SLAM systems. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 573–580. IEEE, 2012.
  45. Indirect Kalman filter for 3D attitude estimation. University of Minnesota, Dept. of Comp. Sci. & Eng., Tech. Rep, 2:2005, 2005.
  46. Ultimate SLAM? Combining events, images, and IMU for robust visual SLAM in HDR and high-speed scenarios. IEEE Robotics and Automation Letters, 3(2):994–1001, 2018.
  47. Real-time metric state estimation for modular vision-inertial systems. In 2011 IEEE international conference on robotics and automation, pages 4531–4537. IEEE, 2011.
  48. AnalogNet: Convolutional neural network inference on analog focal plane sensor processors. ArXiv, abs/2006.01765, 2020.
  49. Clock synchronization algorithms for network measurements. In Proceedings of the IEEE Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies, 2002.
  50. Event-based visual inertial odometry. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5391–5399, 2017.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now