Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Image-based Visual Servo Control for Aerial Manipulation Using a Fully-Actuated UAV (2306.16530v1)

Published 28 Jun 2023 in cs.RO, cs.SY, and eess.SY

Abstract: Using Unmanned Aerial Vehicles (UAVs) to perform high-altitude manipulation tasks beyond just passive visual application can reduce the time, cost, and risk of human workers. Prior research on aerial manipulation has relied on either ground truth state estimate or GPS/total station with some Simultaneous Localization and Mapping (SLAM) algorithms, which may not be practical for many applications close to infrastructure with degraded GPS signal or featureless environments. Visual servo can avoid the need to estimate robot pose. Existing works on visual servo for aerial manipulation either address solely end-effector position control or rely on precise velocity measurement and pre-defined visual visual marker with known pattern. Furthermore, most of previous work used under-actuated UAVs, resulting in complicated mechanical and hence control design for the end-effector. This paper develops an image-based visual servo control strategy for bridge maintenance using a fully-actuated UAV. The main components are (1) a visual line detection and tracking system, (2) a hybrid impedance force and motion control system. Our approach does not rely on either robot pose/velocity estimation from an external localization system or pre-defined visual markers. The complexity of the mechanical system and controller architecture is also minimized due to the fully-actuated nature. Experiments show that the system can effectively execute motion tracking and force holding using only the visual guidance for the bridge painting. To the best of our knowledge, this is one of the first studies on aerial manipulation using visual servo that is capable of achieving both motion and force control without the need of external pose/velocity information or pre-defined visual guidance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. S. Zhao, H. Zhang, P. Wang, L. Nogueira, and S. Scherer, “Super odometry: Imu-centric lidar-visual-inertial estimator for challenging environments,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 8729–8736.
  2. R. Bonatti, W. Wang, C. Ho, A. Ahuja, M. Gschwindt, E. Camci, E. Kayacan, S. Choudhury, and S. Scherer, “Autonomous aerial cinematography in unstructured environments with learned artistic decision-making,” Journal of Field Robotics, vol. 37, no. 4, pp. 606–641, 2020.
  3. B. Moon, S. Chatterjee, and S. Scherer, “Tigris: An informed sampling-based algorithm for informative path planning,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022.
  4. J. Geng and J. W. Langelaan, “Cooperative transport of a slung load using load-leading control,” Journal of Guidance, Control, and Dynamics, vol. 43, no. 7, pp. 1313–1331, 2020.
  5. M. Mousaei, J. Geng, A. Keipour, D. Bai, and S. Scherer, “Design, modeling and control for a tilt-rotor vtol uav in the presence of actuator failure,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 4310–4317.
  6. A. Keipour, “Physical interaction and manipulation of the environment using aerial robots,” Ph.D. dissertation, Carnegie Mellon University, 2022.
  7. F. Ruggiero, V. Lippiello, and A. Ollero, “Aerial manipulation: A literature review,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1957–1964, 2018.
  8. A. Suarez, V. M. Vega, M. Fernandez, G. Heredia, and A. Ollero, “Benchmarks for aerial manipulation,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2650–2657, 2020.
  9. D. Tzoumanikas, F. Graule, Q. Yan, D. Shah, M. Popovic, and S. Leutenegger, “Aerial manipulation using hybrid force and position nmpc applied to aerial writing,” arXiv preprint arXiv:2006.02116, 2020.
  10. C. Lanegger, M. Ruggia, M. Tognon, L. Ott, and R. Siegwart, “Aerial layouting: Design and control of a compliant and actuated end-effector for precise in-flight marking on ceilings,” Proceedings of Robotics Robotics: Science and System XVIII, 2022.
  11. A. J. Choi, J. Park, and J.-H. Han, “Automated aerial docking system using onboard vision-based deep learning,” Journal of Aerospace Information Systems, vol. 19, no. 6, pp. 421–436, 2022.
  12. D. Lee, H. Seo, D. Kim, and H. J. Kim, “Aerial manipulation using model predictive control for opening a hinged door,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 1237–1242.
  13. M. Brunner, L. Giacomini, R. Siegwart, and M. Tognon, “Energy tank-based policies for robust aerial physical interaction with moving objects,” arXiv preprint arXiv:2202.06755, 2022.
  14. S. Hamaza, I. Georgilas, G. Heredia, A. Ollero, and T. Richardson, “Design, modeling, and control of an aerial manipulator for placement and retrieval of sensors in the environment,” Journal of Field Robotics, vol. 37, no. 7, pp. 1224–1245, 2020.
  15. A. E. Jimenez-Cano, P. J. Sanchez-Cuevas, P. Grau, A. Ollero, and G. Heredia, “Contact-based bridge inspection multirotors: Design, modeling, and control considering the ceiling effect,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 3561–3568, 2019.
  16. P. J. Sanchez-Cuevas, A. Gonzalez-Morgado, N. Cortes, D. B. Gayango, A. E. Jimenez-Cano, A. Ollero, and G. Heredia, “Fully-actuated aerial manipulator for infrastructure contact inspection: Design, modeling, localization, and control,” Sensors, vol. 20, no. 17, p. 4708, 2020.
  17. S. Hutchinson, G. D. Hager, and P. I. Corke, “A tutorial on visual servo control,” IEEE transactions on robotics and automation, vol. 12, no. 5, pp. 651–670, 1996.
  18. J. Thomas, G. Loianno, K. Sreenath, and V. Kumar, “Toward image based visual servoing for aerial grasping and perching,” in 2014 IEEE international conference on robotics and automation (ICRA).   IEEE, 2014, pp. 2113–2118.
  19. B. Espiau, F. Chaumette, and P. Rives, “A new approach to visual servoing in robotics,” ieee Transactions on Robotics and Automation, vol. 8, no. 3, pp. 313–326, 1992.
  20. N. Andreff, B. Espiau, and R. Horaud, “Visual servoing from lines,” The International Journal of Robotics Research, vol. 21, no. 8, pp. 679–699, 2002.
  21. C. Yu, Z. Cai, H. Pham, and Q.-C. Pham, “Siamese convolutional neural network for sub-millimeter-accurate camera pose estimation and visual servoing,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2019, pp. 935–941.
  22. Y. Ma, X. Liu, J. Zhang, D. Xu, D. Zhang, and W. Wu, “Robotic grasping and alignment for small size components assembly based on visual servoing,” The International Journal of Advanced Manufacturing Technology, vol. 106, no. 11, pp. 4827–4843, 2020.
  23. A. Keipour, G. A. Pereira, R. Bonatti, R. Garg, P. Rastogi, G. Dubey, and S. Scherer, “Visual servoing approach to autonomous uav landing on a moving vehicle,” Sensors, vol. 22, no. 17, p. 6549, 2022.
  24. D. Zheng, H. Wang, W. Chen, and Y. Wang, “Planning and tracking in image space for image-based visual servoing of a quadrotor,” IEEE Transactions on Industrial Electronics, vol. 65, no. 4, pp. 3376–3385, 2017.
  25. B. Luo, H. Chen, F. Quan, S. Zhang, and Y. Liu, “Natural feature-based visual servoing for grasping target with an aerial manipulator,” Journal of Bionic Engineering, vol. 17, no. 2, pp. 215–228, 2020.
  26. N. Lai, Y. Chen, J. Liang, B. He, H. Zhong, H. Zhang, and Y. Wang, “Image dynamics-based visual servo control for unmanned aerial manipulatorl with a virtual camera,” IEEE/ASME Transactions on Mechatronics, 2022.
  27. M. Xu, A. Hu, and H. Wang, “Image-based visual impedance force control for contact aerial manipulation,” IEEE Transactions on Automation Science and Engineering, 2022.
  28. H. Zhong, Z. Miao, Y. Wang, J. Mao, L. Li, H. Zhang, Y. Chen, and R. Fierro, “A practical visual servo control for aerial manipulation using a spherical projection model,” IEEE Transactions on Industrial Electronics, vol. 67, no. 12, pp. 10 564–10 574, 2019.
  29. A. Santamaria-Navarro, P. Grosch, V. Lippiello, J. Solà, and J. Andrade-Cetto, “Uncalibrated visual servo for unmanned aerial manipulation,” IEEE/ASME Transactions on Mechatronics, vol. 22, no. 4, pp. 1610–1621, 2017.
  30. A. Ollero, M. Tognon, A. Suarez, D. Lee, and A. Franchi, “Past, present, and future of aerial robotic manipulators,” IEEE Transactions on Robotics, vol. 38, no. 1, pp. 626–645, 2021.
  31. A. Keipour, M. Mousaei, A. T. Ashley, and S. Scherer, “Integration of fully-actuated multirotors into real-world applications,” 2020. [Online]. Available: https://arxiv.org/abs/2011.06666
  32. K. Bodie, M. Brunner, M. Pantic, S. Walser, P. Pfändler, U. Angst, R. Siegwart, and J. Nieto, “Active interaction force control for contact-based inspection with a fully actuated aerial vehicle,” IEEE Transactions on Robotics, vol. 37, no. 3, pp. 709–722, 2021.
  33. F. Chaumette, P. Rives, and B. Espiau, “Classification and realization of the different vision-based tasks,” in Visual Servoing: Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback.   World Scientific, 1993, pp. 199–228.
  34. M. Marey and F. Chaumette, “New strategies for avoiding robot joint limits: Application to visual servoing using a large projection operator,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010, pp. 6222–6227.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now