Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exploring Deep Reinforcement Learning for Robust Target Tracking using Micro Aerial Vehicles (2312.17552v2)

Published 29 Dec 2023 in cs.RO

Abstract: The capability to autonomously track a non-cooperative target is a key technological requirement for micro aerial vehicles. In this paper, we propose an output feedback control scheme based on deep reinforcement learning for controlling a micro aerial vehicle to persistently track a flying target while maintaining visual contact. The proposed method leverages relative position data for control, relaxing the assumption of having access to full state information which is typical of related approaches in literature. Moreover, we exploit classical robustness indicators in the learning process through domain randomization to increase the robustness of the learned policy. Experimental results validate the proposed approach for target tracking, demonstrating high performance and robustness with respect to mass mismatches and control delays. The resulting nonlinear controller significantly outperforms a standard model-based design in numerous off-nominal scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. B. J. Emran and H. Najjaran, “A review of quadrotor: An underactuated mechanical system,” Annual Reviews in Control, vol. 46, pp. 165–180, 2018.
  2. A. Saviolo, G. Li, and G. Loianno, “Physics-inspired temporal learning of quadrotor dynamics for accurate model predictive trajectory tracking,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 256–10 263, 2022.
  3. V. Mistler, A. Benallegue, and N. M’sirdi, “Exact linearization and noninteracting control of a 4 rotors helicopter via dynamic feedback,” in Proceedings of the 10th IEEE International Workshop on Robot and Human Interactive Communication, 2001, pp. 586–593.
  4. P. Casau, C. G. Mayhew, R. G. Sanfelice, and C. Silvestre, “Robust global exponential stabilization on the n-dimensional sphere with applications to trajectory tracking for quadrotors,” Automatica, vol. 110, p. 108534, 2019.
  5. D. Invernizzi, M. Lovera, and L. Zaccarian, “Global robust attitude tracking with torque disturbance rejection via dynamic hybrid feedback,” Automatica, vol. 144, p. 110462, 2022.
  6. M. Giurato and M. Lovera, “Quadrotor attitude determination: a comparison study,” in 2016 IEEE Conference on Control Applications (CCA), 2016, pp. 21–26.
  7. D. Bicego, J. Mazzetto, R. Carli, M. Farina, and A. Franchi, “Nonlinear model predictive control with enhanced actuator model for multi-rotor aerial vehicles with generic designs,” Journal of Intelligent & Robotic Systems, vol. 100, no. 3, pp. 1213–1247, 2020.
  8. W. Zhao, H. Liu, F. L. Lewis, K. P. Valavanis, and X. Wang, “Robust visual servoing control for ground target tracking of quadrotors,” IEEE Transactions on Control Systems Technology, vol. 28, no. 5, pp. 1980–1987, 2020.
  9. J. Thomas, G. Loianno, K. Sreenath, and V. Kumar, “Toward image based visual servoing for aerial grasping and perching,” in IEEE International Conference on Robotics and Automation, 2014, pp. 2113–2118.
  10. C. Sampedro, A. Rodriguez-Ramos, I. Gil, L. Mejias, and P. Campoy, “Image-based visual servoing controller for multirotor aerial robots using deep reinforcement learning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018, pp. 979–986.
  11. R. Polvara, M. Patacchiola, S. Sharma, J. Wan, A. Manning, R. Sutton, and A. Cangelosi, “Toward end-to-end control for UAV autonomous landing via deep reinforcement learning,” in 2018 International Conference on Unmanned Aircraft Systems, 2018, pp. 115–123.
  12. M. Xi, Y. Zhou, Z. Chen, W. Zhou, and H. Li, “Anti-distractor active object tracking in 3D environments,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 6, pp. 3697–3707, 2021.
  13. A. Molchanov, T. Chen, W. Hönig, J. A. Preiss, N. Ayanian, and G. S. Sukhatme, “Sim-to-(multi)-real: Transfer of low-level robust control policies to multiple quadrotors,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2019, pp. 59–66.
  14. A. M. Deshpande, A. A. Minai, and M. Kumar, “Robust deep reinforcement learning for quadcopter control,” IFAC-PapersOnLine, vol. 54, no. 20, pp. 90–95, 2021.
  15. M. Turchetta, A. Krause, and S. Trimpe, “Robust model-free reinforcement learning with multi-objective bayesian optimization,” in 2020 IEEE International Conference on Robotics and Automation, 2020, pp. 10 702–10 708.
  16. Y. Yu, S. Yang, M. Wang, C. Li, and Z. Li, “High performance full attitude control of a quadrotor on SO(3),” in 2015 IEEE International Conference on Robotics and Automation, 2015, pp. 1698–1703.
  17. N. Cao and A. F. Lynch, “Inner–outer loop control for quadrotor UAVs with input and state constraints,” IEEE Transactions on Control Systems Technology, vol. 24, no. 5, pp. 1797–1804, 2015.
  18. A. Mokhtari, A. Benallegue, and B. Daachi, “Robust feedback linearization and G⁢H∞𝐺subscript𝐻GH_{\infty}italic_G italic_H start_POSTSUBSCRIPT ∞ end_POSTSUBSCRIPT controller for a quadrotor unmanned aerial vehicle,” in 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005, pp. 1198–1203.
  19. M. A. Lotufo, L. Colangelo, and C. Novara, “Control design for UAV quadrotors via embedded model control,” IEEE Transactions on Control Systems Technology, vol. 28, no. 5, pp. 1741–1756, 2019.
  20. M. Leomanni, F. Ferrante, N. Cartocci, G. Costante, M. L. Fravolini, K. M. Dogan, and T. Yucelen, “Robust output feedback control of a quadrotor UAV for autonomous vision-based target tracking,” in AIAA SCITECH 2023 Forum, 2023.
  21. D. Cabecinhas, R. Cunha, and C. Silvestre, “A nonlinear quadrotor trajectory tracking controller with disturbance rejection,” Control Engineering Practice, vol. 26, pp. 1–10, 2014.
  22. D. Lee, H. Jin Kim, and S. Sastry, “Feedback linearization vs. adaptive sliding mode control for a quadrotor helicopter,” International Journal of Control, Automation and Systems, vol. 7, no. 3, pp. 419–428, 2009.
  23. T. Lee, M. Leok, and N. H. McClamroch, “Geometric tracking control of a quadrotor UAV on SE (3),” in 49th IEEE Conference on Decision and Control (CDC), 2010, pp. 5420–5425.
  24. S. Wu, R. Li, Y. Shi, and Q. Liu, “Vision-based target detection and tracking system for a quadcopter,” IEEE Access, vol. 9, pp. 62 043–62 054, 2021.
  25. J. Li, J. Xu, F. Zhong, X. Kong, Y. Qiao, and Y. Wang, “Pose-assisted multi-camera collaboration for active object tracking,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, 2020, pp. 759–766.
  26. A. Devo, A. Dionigi, and G. Costante, “Enhancing continuous control of mobile robots for end-to-end visual active tracking,” Robotics and Autonomous Systems, vol. 142, p. 103799, 2021.
  27. W. Lei, H. Fu, and G. Sun, “Active object tracking of free floating space manipulators based on deep reinforcement learning,” Advances in Space Research, vol. 70, no. 11, pp. 3506–3519, 2022.
  28. J. Hwangbo, I. Sa, R. Siegwart, and M. Hutter, “Control of a quadrotor with reinforcement learning,” IEEE Robotics and Automation Letters, vol. 2, no. 4, pp. 2096–2103, 2017.
  29. X. Lin, Y. Yu, and C. Sun, “Supplementary reinforcement learning controller designed for quadrotor UAVs,” IEEE Access, vol. 7, pp. 26 422–26 431, 2019.
  30. H. Han, J. Cheng, Z. Xi, and B. Yao, “Cascade flight control of quadrotors based on deep reinforcement learning,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11 134–11 141, 2022.
  31. Q. Sun, J. Fang, W. X. Zheng, and Y. Tang, “Aggressive quadrotor flight using curiosity-driven reinforcement learning,” IEEE Transactions on Industrial Electronics, vol. 69, no. 12, pp. 13 838–13 848, 2022.
  32. S. Batra, Z. Huang, A. Petrenko, T. Kumar, A. Molchanov, and G. S. Sukhatme, “Decentralized control of quadrotor swarms with end-to-end deep reinforcement learning,” in Conference on Robot Learning, 2022, pp. 576–586.
  33. M. W. Mueller, M. Hehn, and R. D’Andrea, “A computationally efficient motion primitive for quadrocopter trajectory generation,” IEEE Transactions on Robotics, vol. 31, no. 6, pp. 1294–1310, 2015.
  34. E. Kaufmann, L. Bauersfeld, and D. Scaramuzza, “A benchmark comparison of learned control policies for agile quadrotor flight,” in 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 10 504–10 510.
  35. J. Thomas, J. Welde, G. Loianno, K. Daniilidis, and V. Kumar, “Autonomous flight for detection, localization, and tracking of moving targets with a small quadrotor,” IEEE Robotics and Automation Letters, vol. 2, no. 3, pp. 1762–1769, 2017.
  36. L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, “Asymmetric actor critic for image-based robot learning,” in 14th Robotics: Science and Systems, 2018.
  37. A. Dionigi, A. Devo, L. Guiducci, and G. Costante, “E-vat: An asymmetric end-to-end approach to visual active exploration and tracking,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4259–4266, 2022.
  38. T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in International Conference on Machine Learning, 2018, pp. 1861–1870.
  39. J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017, pp. 23–30.
  40. A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforcement learning implementations,” Journal of Machine Learning Research, vol. 22, no. 268, pp. 1–8, 2021.
  41. Epic Games, “Unreal engine.” [Online]. Available: https://www.unrealengine.com
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now