Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Recent Advances in Path Integral Control for Trajectory Optimization: An Overview in Theoretical and Algorithmic Perspectives (2309.12566v2)

Published 22 Sep 2023 in cs.RO, cs.SY, eess.SY, and math.OC

Abstract: This paper presents a tutorial overview of path integral (PI) control approaches for stochastic optimal control and trajectory optimization. We concisely summarize the theoretical development of path integral control to compute a solution for stochastic optimal control and provide algorithmic descriptions of the cross-entropy (CE) method, an open-loop controller using the receding horizon scheme known as the model predictive path integral (MPPI), and a parameterized state feedback controller based on the path integral control theory. We discuss policy search methods based on path integral control, efficient and stable sampling strategies, extensions to multi-agent decision-making, and MPPI for the trajectory optimization on manifolds. For tutorial demonstrations, some PI-based controllers are implemented in Python, MATLAB and ROS2/Gazebo simulations for trajectory optimization. The simulation frameworks and source codes are publicly available at https://github.com/INHA-Autonomous-Systems-Laboratory-ASL/An-Overview-on-Recent-Advances-in-Path-Integral-Control.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (172)
  1. O. Von Stryk and R. Bulirsch, “Direct and indirect methods for trajectory optimization,” Annals of operations research, vol. 37, no. 1, pp. 357–373, 1992.
  2. J. T. Betts, “Survey of numerical methods for trajectory optimization,” Journal of guidance, control, and dynamics, vol. 21, no. 2, pp. 193–207, 1998.
  3. A. V. Rao, “Trajectory optimization: A survey,” Optimization and optimal control in automotive systems, pp. 3–21, 2014.
  4. B. Paden, M. Čáp, S. Z. Yong, D. Yershov, and E. Frazzoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Transactions on Intelligent Vehicles, vol. 1, no. 1, pp. 33–55, 2016.
  5. L. Claussmann, M. Revilloud, D. Gruyer, and S. Glaser, “A review of motion planning for highway autonomous driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 5, pp. 1826–1848, 2019.
  6. S. Teng, X. Hu, P. Deng, B. Li, Y. Li, Y. Ai, D. Yang, L. Li, Z. Xuanyuan, F. Zhu et al., “Motion planning for autonomous driving: The state of the art and future perspectives,” IEEE Transactions on Intelligent Vehicles, 2023.
  7. Y. Song, M. Steinweg, E. Kaufmann, and D. Scaramuzza, “Autonomous drone racing with deep reinforcement learning,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 1205–1212.
  8. Z. Han, Z. Wang, N. Pan, Y. Lin, C. Xu, and F. Gao, “Fast-Racing: An open-source strong baseline for s⁢es𝑒\mathrm{s}eroman_s italic_e(3) planning in autonomous drone racing,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 8631–8638, 2021.
  9. D. Hanover, A. Loquercio, L. Bauersfeld, A. Romero, R. Penicka, Y. Song, G. Cioffi, E. Kaufmann, and D. Scaramuzza, “Autonomous drone racing: A survey,” arXiv e-prints, pp. arXiv–2301, 2023.
  10. M. Lan, S. Lai, T. H. Lee, and B. M. Chen, “A survey of motion and task planning techniques for unmanned multicopter systems,” Unmanned Systems, vol. 9, no. 02, pp. 165–198, 2021.
  11. M. Wang, J. Diepolder, S. Zhang, M. Söpper, and F. Holzapfel, “Trajectory optimization-based maneuverability assessment of evtol aircraft,” Aerospace Science and Technology, vol. 117, p. 106903, 2021.
  12. J. Park, I. Kim, J. Suk, and S. Kim, “Trajectory optimization for takeoff and landing phase of uam considering energy and safety,” Aerospace Science and Technology, vol. 140, p. 108489, 2023.
  13. P. Pradeep, T. A. Lauderdale, G. B. Chatterji, K. Sheth, C. F. Lai, B. Sridhar, K.-M. Edholm, and H. Erzberger, “Wind-optimal trajectories for multirotor evtol aircraft on uam missions,” in Aiaa Aviation 2020 Forum, 2020, p. 3271.
  14. H.-H. Kwon and H.-L. Choi, “A convex programming approach to mid-course trajectory optimization for air-to-ground missiles,” International Journal of Aeronautical and Space Sciences, vol. 21, pp. 479–492, 2020.
  15. H. Roh, Y.-J. Oh, M.-J. Tahk, K.-J. Kwon, and H.-H. Kwon, “L1 penalized sequential convex programming for fast trajectory optimization: With application to optimal missile guidance,” International Journal of Aeronautical and Space Sciences, vol. 21, pp. 493–503, 2020.
  16. I. Garcia and J. P. How, “Trajectory optimization for satellite reconfiguration maneuvers with position and attitude constraints,” in Proceedings of the 2005, American Control Conference, 2005.   IEEE, 2005, pp. 889–894.
  17. A. Weiss, F. Leve, M. Baldwin, J. R. Forbes, and I. Kolmanovsky, “Spacecraft constrained attitude control using positively invariant constraint admissible sets on SO(3)×ℝ3absentsuperscriptℝ3\times\mathbb{R}^{3}× blackboard_R start_POSTSUPERSCRIPT 3 end_POSTSUPERSCRIPT,” in 2014 American Control Conference.   IEEE, 2014, pp. 4955–4960.
  18. A. Gatherer and Z. Manchester, “Magnetorquer-only attitude control of small satellites using trajectory optimization,” in Proceedings of AAS/AIAA Astrodynamics Specialist Conference, 2019.
  19. D. Malyuta, Y. Yu, P. Elango, and B. Açıkmeşe, “Advances in trajectory optimization for space vehicle control,” Annual Reviews in Control, vol. 52, pp. 282–315, 2021.
  20. T. L. Dearing, J. Hauser, X. Chen, M. M. Nicotra, and C. Petersen, “Efficient trajectory optimization for constrained spacecraft attitude maneuvers,” Journal of Guidance, Control, and Dynamics, vol. 45, no. 4, pp. 638–650, 2022.
  21. M. Elbanhawi and M. Simic, “Sampling-based robot motion planning: A review,” IEEE Access, vol. 2, pp. 56–77, 2014.
  22. Z. Kingston, M. Moll, and L. E. Kavraki, “Sampling-based methods for motion planning with constraints,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 1, no. 1, pp. 159–185, 2018.
  23. M. Kelly, “An introduction to trajectory optimization: How to do your own direct collocation,” SIAM Review, vol. 59, no. 4, pp. 849–904, 2017.
  24. R. Bonalli, A. Cauligi, A. Bylard, and M. Pavone, “GuSTO: Guaranteed sequential trajectory optimization via sequential convex programming,” in 2019 International conference on robotics and automation (ICRA).   IEEE, 2019, pp. 6741–6747.
  25. T. A. Howell, B. E. Jackson, and Z. Manchester, “ALTRO: A fast solver for constrained trajectory optimization,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2019, pp. 7674–7679.
  26. S. G. Manyam, D. W. Casbeer, I. E. Weintraub, and C. Taylor, “Trajectory optimization for rendezvous planning using quadratic Bézier curves,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 1405–1412.
  27. D. Malyuta, T. P. Reynolds, M. Szmuk, T. Lew, R. Bonalli, M. Pavone, and B. Açıkmeşe, “¡ convex optimization for trajectory generation: A tutorial on generating dynamically feasible trajectories reliably and efficiently,” IEEE Control Systems Magazine, vol. 42, no. 5, pp. 40–113, 2022.
  28. D. Q. Mayne, “Differential dynamic programming–a unified approach to the optimization of dynamic systems,” in Control and dynamic systems.   Elsevier, 1973, vol. 10, pp. 179–254.
  29. Z. Xie, C. K. Liu, and K. Hauser, “Differential dynamic programming with nonlinear constraints,” in 2017 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2017, pp. 695–702.
  30. J. Chen, W. Zhan, and M. Tomizuka, “Autonomous driving motion planning with constrained iterative LQR,” IEEE Transactions on Intelligent Vehicles, vol. 4, no. 2, pp. 244–254, 2019.
  31. A. Pavlov, I. Shames, and C. Manzie, “Interior point differential dynamic programming,” IEEE Transactions on Control Systems Technology, vol. 29, no. 6, pp. 2720–2727, 2021.
  32. K. Cao, M. Cao, S. Yuan, and L. Xie, “DIRECT: A differential dynamic programming based framework for trajectory generation,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 2439–2446, 2022.
  33. I. Chatzinikolaidis and Z. Li, “Trajectory optimization of contact-rich motions using implicit differential dynamic programming,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 2626–2633, 2021.
  34. M.-G. Kim and K.-K. K. Kim, “An extension of interior point differential dynamic programming for optimal control problems with second-order conic constraints,” Transactions of the Korean Institute of Electrical Engineers, vol. 71, no. 11, pp. 1666–1672, 2022.
  35. X. Zhong, J. Tian, H. Hu, and X. Peng, “Hybrid path planning based on safe a* algorithm and adaptive window approach for mobile robot in large-scale dynamic environment,” Journal of Intelligent & Robotic Systems, vol. 99, no. 1, pp. 65–77, 2020.
  36. S. Sun, A. Romero, P. Foehn, E. Kaufmann, and D. Scaramuzza, “A comparative study of nonlinear MPC and differential-flatness-based control for quadrotor agile flight,” IEEE Transactions on Robotics, vol. 38, no. 6, pp. 3357–3373, 2022.
  37. M. Faessler, A. Franchi, and D. Scaramuzza, “Differential flatness of quadrotor dynamics subject to rotor drag for accurate tracking of high-speed trajectories,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 620–626, 2017.
  38. A. Domahidi and J. Jerez, “FORCES Professional,” EmbotechAG, url=https://embotech.com/FORCES-Pro, 2014–2023.
  39. J. D. Gammell and M. P. Strub, “Asymptotically optimal sampling-based motion planning methods,” arXiv preprint arXiv:2009.10484, 2020.
  40. J. J. Kuffner and S. M. LaValle, “RRT-connect: An efficient approach to single-query path planning,” in 2000 IEEE International Conference on Robotics and Automation (ICRA), vol. 2.   IEEE, 2000, pp. 995–1001.
  41. L. Janson, B. Ichter, and M. Pavone, “Deterministic sampling-based motion planning: Optimality, complexity, and performance,” The International Journal of Robotics Research, vol. 37, no. 1, pp. 46–61, 2018.
  42. L. Campos-Macías, D. Gómez-Gutiérrez, R. Aldana-López, R. de la Guardia, and J. I. Parra-Vilchis, “A hybrid method for online trajectory planning of mobile robots in cluttered environments,” IEEE Robotics and Automation Letters, vol. 2, no. 2, pp. 935–942, 2017.
  43. A. A. Ravankar, A. Ravankar, T. Emaru, and Y. Kobayashi, “HPPRM: Hybrid potential based probabilistic roadmap algorithm for improved dynamic path planning of mobile robots,” IEEE Access, vol. 8, pp. 221 743–221 766, 2020.
  44. F. Kiani, A. Seyyedabbasi, R. Aliyev, M. U. Gulle, H. Basyildiz, and M. A. Shah, “Adapted-RRT: Novel hybrid method to solve three-dimensional path planning problem using sampling and metaheuristic-based algorithms,” Neural Computing and Applications, vol. 33, no. 22, pp. 15 569–15 599, 2021.
  45. Z. Yu, Z. Si, X. Li, D. Wang, and H. Song, “A novel hybrid particle swarm optimization algorithm for path planning of uavs,” IEEE Internet of Things Journal, vol. 9, no. 22, pp. 22 547–22 558, 2022.
  46. I. A. Sucan, M. Moll, and L. E. Kavraki, “The open motion planning library,” IEEE Robotics & Automation Magazine, vol. 19, no. 4, pp. 72–82, 2012.
  47. M. Kalakrishnan, S. Chitta, E. Theodorou, P. Pastor, and S. Schaal, “STOMP: Stochastic trajectory optimization for motion planning,” in IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2011, pp. 4569–4574.
  48. M. Likhachev, “Search-based planning lab,” 2010. [Online]. Available: http://sbpl.net/Home
  49. M. Zucker, N. Ratliff, A. D. Dragan, M. Pivtoraiko, M. Klingensmith, C. M. Dellin, J. A. Bagnell, and S. S. Srinivasa, “Chomp: Covariant hamiltonian optimization for motion planning,” The International Journal of Robotics Research, vol. 32, no. 9-10, pp. 1164–1193, 2013.
  50. H. J. Kappen, “Linear theory for control of nonlinear stochastic systems,” Physical review letters, vol. 95, no. 20, p. 200201, 2005.
  51. V. Gómez, H. J. Kappen, J. Peters, and G. Neumann, “Policy search for path integral control,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases.   Springer, 2014, pp. 482–497.
  52. G. Williams, A. Aldrich, and E. A. Theodorou, “Model predictive path integral control: From theory to parallel computation,” Journal of Guidance, Control, and Dynamics, vol. 40, no. 2, pp. 344–357, 2017.
  53. S. Thijssen and H. Kappen, “Consistent adaptive multiple importance sampling and controlled diffusions,” arXiv preprint arXiv:1803.07966, 2018.
  54. G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive driving with model predictive path integral control,” in IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2016, pp. 1433–1440.
  55. ——, “Information-theoretic model predictive control: Theory and applications to autonomous driving,” IEEE Transactions on Robotics, vol. 34, no. 6, pp. 1603–1622, 2018.
  56. G. R. Williams, “Model predictive path integral control: Theoretical foundations and applications to autonomous driving,” Ph.D. dissertation, Georgia Institute of Techonology, 2019.
  57. Y. Pan, E. Theodorou, and M. Kontitsis, “Sample efficient path integral control under uncertainty,” Advances in Neural Information Processing Systems, vol. 28, 2015.
  58. I. Abraham, A. Handa, N. Ratliff, K. Lowrey, T. D. Murphey, and D. Fox, “Model-based generalization under parameter uncertainty using path integral control,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2864–2871, 2020.
  59. J. Pravitra, K. A. Ackerman, C. Cao, N. Hovakimyan, and E. A. Theodorou, “L1-Adaptive MPPI architecture for robust and agile control of multirotors,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 7661–7666.
  60. G. Williams, E. Rombokas, and T. Daniel, “GPU based path integral control with learned dynamics,” arXiv preprint arXiv:1503.00330, 2015.
  61. M. Okada, T. Aoshima, and L. Rigazio, “Path integral networks: End-to-end differentiable optimal control,” arXiv preprint arXiv:1706.09597, 2017.
  62. I. S. Mohamed, M. Ali, and L. Liu, “GP-guided MPPI for efficient navigation in complex unknown cluttered environments,” arXiv preprint arXiv:2307.04019, 2023.
  63. M. S. Gandhi, B. Vlahov, J. Gibson, G. Williams, and E. A. Theodorou, “Robust model predictive path integral control: Analysis and performance guarantees,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1423–1430, 2021.
  64. E. Arruda, M. J. Mathew, M. Kopicki, M. Mistry, M. Azad, and J. L. Wyatt, “Uncertainty averse pushing with model predictive path integral control,” in 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).   IEEE, 2017, pp. 497–502.
  65. M. Raisi, A. Noohian, and S. Fallah, “A fault-tolerant and robust controller using model predictive path integral control for free-flying space robots,” Frontiers in Robotics and AI, vol. 9, p. 1027918, 2022.
  66. J. Zeng, B. Zhang, and K. Sreenath, “Safety-critical model predictive control with discrete-time control barrier function,” in 2021 American Control Conference (ACC).   IEEE, 2021, pp. 3882–3889.
  67. C. Tao, H.-J. Yoon, H. Kim, N. Hovakimyan, and P. Voulgaris, “Path integral methods with stochastic control barrier functions,” in 2022 IEEE 61st Conference on Decision and Control (CDC).   IEEE, 2022, pp. 1654–1659.
  68. J. Yin, C. Dawson, C. Fan, and P. Tsiotras, “Shield model predictive path integral: A computationally efficient robust MPC approach using control barrier functions,” arXiv preprint arXiv:2302.11719, 2023.
  69. J. Yin, Z. Zhang, E. Theodorou, and P. Tsiotras, “Trajectory distribution control for model predictive path integral control using covariance steering,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 1478–1484.
  70. F. S. Barbosa, B. Lacerda, P. Duckworth, J. Tumova, and N. Hawes, “Risk-aware motion planning in partially known environments,” in 2021 60th IEEE Conference on Decision and Control (CDC).   IEEE, 2021, pp. 5220–5226.
  71. Z. Wang, O. So, K. Lee, and E. A. Theodorou, “Adaptive risk sensitive model predictive control with stochastic search,” in Learning for Dynamics and Control.   PMLR, 2021, pp. 510–522.
  72. X. Cai, M. Everett, L. Sharma, P. R. Osteen, and J. P. How, “Probabilistic traversability model for risk-aware motion planning in off-road environments,” arXiv preprint arXiv:2210.00153, 2022.
  73. J. Yin, Z. Zhang, and P. Tsiotras, “Risk-aware model predictive path integral control using conditional value-at-risk,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 7937–7943.
  74. C. Tao, H. Kim, and N. Hovakimyan, “RRT guided model predictive path integral method,” arXiv preprint arXiv:2301.13143, 2023.
  75. E. A. Theodorou and E. Todorov, “Relative entropy and free energy dualities: Connections to path integral and KL control,” in 2012 51st IEEE Conference on Decision and Control (CDC).   IEEE, 2012, pp. 1466–1473.
  76. A. Ijspeert, J. Nakanishi, and S. Schaal, “Learning attractor landscapes for learning motor primitives,” Advances in neural information processing systems, vol. 15, 2002.
  77. E. Theodorou, J. Buchli, and S. Schaal, “A generalized path integral control approach to reinforcement learning,” The Journal of Machine Learning Research, vol. 11, pp. 3137–3181, 2010.
  78. S. Levine and V. Koltun, “Guided policy search,” in International Conference on Machine Learning.   PMLR, 2013, pp. 1–9.
  79. W. H. Montgomery and S. Levine, “Guided policy search via approximate mirror descent,” Advances in Neural Information Processing Systems (NIPS), vol. 29, 2016.
  80. M.-G. Kim and K.-K. K. Kim, “MPPI-IPDDP: Hybrid method of collision-free smooth trajectory generation for autonomous robots,” arXiv preprint arXiv:2208.02439, 2022.
  81. D. Thalmeier, H. J. Kappen, S. Totaro, and V. Gómez, “Adaptive smoothing for path integral control,” The Journal of Machine Learning Research, vol. 21, no. 1, pp. 7814–7850, 2020.
  82. S. Särkkä, “Unscented rauch–tung–striebel smoother,” IEEE transactions on automatic control, vol. 53, no. 3, pp. 845–849, 2008.
  83. H.-C. Ruiz and H. J. Kappen, “Particle smoothing for hidden diffusion processes: Adaptive path integral smoother,” IEEE Transactions on Signal Processing, vol. 65, no. 12, pp. 3191–3203, 2017.
  84. T. Neve, T. Lefebvre, and G. Crevecoeur, “Comparative study of sample based model predictive control with application to autonomous racing,” in 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).   IEEE, 2022, pp. 1632–1638.
  85. T. Kim, G. Park, K. Kwak, J. Bae, and W. Lee, “Smooth model predictive path integral control without smoothing,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 406–10 413, 2022.
  86. T. Lefebvre and G. Crevecoeur, “Entropy regularised deterministic optimal control: From path integral solution to sample-based trajectory optimisation,” in 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).   IEEE, 2022, pp. 401–408.
  87. A. Lambert, A. Fishman, D. Fox, B. Boots, and F. Ramos, “Stein variational model predictive control,” arXiv preprint arXiv:2011.07641, 2020.
  88. B. Eysenbach and S. Levine, “If MaxEnt RL is the answer, what is the question?” arXiv preprint arXiv:1910.01913, 2019.
  89. P. Whittle, “Likelihood and cost as path integrals,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 53, no. 3, pp. 505–529, 1991.
  90. H. J. Kappen, V. Gómez, and M. Opper, “Optimal control as a graphical model inference problem,” Machine learning, vol. 87, no. 2, pp. 159–182, 2012.
  91. J. Watson, H. Abdulsamad, and J. Peters, “Stochastic optimal control as approximate input inference,” in Conference on Robot Learning.   PMLR, 2020, pp. 697–716.
  92. T. Haarnoja, H. Tang, P. Abbeel, and S. Levine, “Reinforcement learning with deep energy-based policies,” in International conference on machine learning.   PMLR, 2017, pp. 1352–1361.
  93. S. Levine, “Reinforcement learning and control as probabilistic inference: Tutorial and review,” arXiv preprint arXiv:1805.00909, 2018.
  94. L. Martino, V. Elvira, D. Luengo, and J. Corander, “An adaptive population importance sampler: Learning from uncertainty,” IEEE Transactions on Signal Processing, vol. 63, no. 16, pp. 4422–4437, 2015.
  95. S. U. Stich, A. Raj, and M. Jaggi, “Safe adaptive importance sampling,” Advances in Neural Information Processing Systems, vol. 30, 2017.
  96. M. F. Bugallo, V. Elvira, L. Martino, D. Luengo, J. Miguez, and P. M. Djuric, “Adaptive importance sampling: The past, the present, and the future,” IEEE Signal Processing Magazine, vol. 34, no. 4, pp. 60–79, 2017.
  97. H. J. Kappen and H. C. Ruiz, “Adaptive importance sampling for control and inference,” Journal of Statistical Physics, vol. 162, no. 5, pp. 1244–1266, 2016.
  98. D. M. Asmar, R. Senanayake, S. Manuel, and M. J. Kochenderfer, “Model predictive optimized path integral strategies,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 3182–3188.
  99. J. Carius, R. Ranftl, F. Farshidian, and M. Hutter, “Constrained stochastic optimal control with learned importance sampling: A path integral approach,” The International Journal of Robotics Research, vol. 41, no. 2, pp. 189–209, 2022.
  100. B. Arouna, “Adaptative Monte Carlo method, a variance reduction technique,” Monte Carlo Methods and Applications, vol. 10, no. 1, pp. 1–24, 2004.
  101. P.-T. De Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein, “A tutorial on the cross-entropy method,” Annals of operations research, vol. 134, no. 1, pp. 19–67, 2005.
  102. M. Kobilarov, “Cross-entropy motion planning,” The International Journal of Robotics Research, vol. 31, no. 7, pp. 855–871, 2012.
  103. W. Zhang, H. Wang, C. Hartmann, M. Weber, and C. Schute, “Applications of the cross-entropy method to importance sampling and optimal control of diffusions,” SIAM Journal on Scientific Computing, vol. 36, no. 6, pp. A2654–A2672, 2014.
  104. M. Testouri, G. Elghazaly, and R. Frank, “Towards a safe real-time motion planning framework for autonomous driving systems: An MPPI approach,” arXiv preprint arXiv:2308.01654, 2023.
  105. I. S. Mohamed, K. Yin, and L. Liu, “Autonomous navigation of AGVs in unknown cluttered environments: log-MPPI control strategy,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 240–10 247, 2022.
  106. J.-S. Ha, S.-S. Park, and H.-L. Choi, “Topology-guided path integral approach for stochastic optimal control in cluttered environment,” Robotics and Autonomous Systems, vol. 113, pp. 81–93, 2019.
  107. I. S. Mohamed, G. Allibert, and P. Martinet, “Model predictive path integral control framework for partially observable navigation: A quadrotor case study,” in 2020 16th International Conference on Control, Automation, Robotics and Vision (ICARCV).   IEEE, 2020, pp. 196–203.
  108. J. Pravitra, E. Theodorou, and E. N. Johnson, “Flying complex maneuvers with model predictive path integral control,” in AIAA Scitech 2021 Forum, 2021, p. 1957.
  109. M. D. Houghton, A. B. Oshin, M. J. Acheson, E. A. Theodorou, and I. M. Gregory, “Path planning: Differential dynamic programming and model predictive path integral control on VTOL aircraft,” in AIAA SCITECH 2022 Forum, 2022, p. 0624.
  110. J. Higgins, N. Mohammad, and N. Bezzo, “A model predictive path integral method for fast, proactive, and uncertainty-aware UAV planning in cluttered environments,” arXiv preprint arXiv:2308.00914, 2023.
  111. P. Nicolay, Y. Petillot, M. Marfeychuk, S. Wang, and I. Carlucho, “Enhancing AUV autonomy with model predictive path integral control,” arXiv preprint arXiv:2308.05547, 2023.
  112. L. Hou, H. Wang, H. Zou, and Y. Zhou, “Robotic manipulation planning for automatic peeling of glass substrate based on online learning model predictive path integral,” Sensors, vol. 22, no. 3, p. 1292, 2022.
  113. K. Yamamoto, R. Ariizumi, T. Hayakawa, and F. Matsuno, “Path integral policy improvement with population adaptation,” IEEE Transactions on Cybernetics, vol. 52, no. 1, pp. 312–322, 2020.
  114. I. S. Mohamed, G. Allibert, and P. Martinet, “Sampling-based MPC for constrained vision based control,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 3753–3758.
  115. I. S. Mohamed, “MPPI-VS: Sampling-based model predictive control strategy for constrained image-based and position-based visual servoing,” arXiv preprint arXiv:2104.04925, 2021.
  116. M. Costanzo, G. De Maria, C. Natale, and A. Russo, “Modeling and control of sampled-data image-based visual servoing with three-dimensional features,” IEEE Transactions on Control Systems Technology, 2023, (Early Access).
  117. S. Macenski, T. Moore, D. V. Lu, A. Merzlyakov, and M. Ferguson, “From the desks of ROS maintainers: A survey of modern & capable mobile robotics algorithms in the robot operating system 2,” Robotics and Autonomous Systems, p. 104493, 2023.
  118. S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall, “Robot operating system 2: Design, architecture, and uses in the wild,” Science Robotics, vol. 7, no. 66, p. eabm6074, 2022.
  119. B. Van Den Broek, W. Wiegerinck, and B. Kappen, “Graphical model inference in optimal control of stochastic multi-agent systems,” Journal of Artificial Intelligence Research, vol. 32, pp. 95–122, 2008.
  120. S. A. Thijssen, “Path integral control,” Ph.D. dissertation, Radboud University, 2016.
  121. V. Gómez, S. Thijssen, A. Symington, S. Hailes, and H. J. Kappen, “Real-time stochastic optimal control for multi-agent quadrotor systems,” in Proceedings of the Twenty-Sixth International Conference on Automated Planning and Scheduling (ICAPS), 2016, pp. 468–476.
  122. N. Wan, A. Gahlawat, N. Hovakimyan, E. A. Theodorou, and P. G. Voulgaris, “Cooperative path integral control for stochastic multi-agent systems,” in American Control Conference (ACC).   IEEE, 2021, pp. 1262–1267.
  123. P. Varnai and D. V. Dimarogonas, “Multi-agent stochastic control using path integral policy improvement,” in American Control Conference (ACC).   IEEE, 2022, pp. 3406–3411.
  124. A. Pourchot and O. Sigaud, “CEM-RL: Combining evolutionary and gradient-based methods for policy search,” arXiv preprint arXiv:1810.01222, 2018.
  125. M. Wen and U. Topcu, “Constrained cross-entropy method for safe reinforcement learning,” Advances in Neural Information Processing Systems, vol. 31, 2018.
  126. B. Amos and D. Yarats, “The differentiable cross-entropy method,” in International Conference on Machine Learning.   PMLR, 2020, pp. 291–302.
  127. Z. Zhang, J. Jin, M. Jagersand, J. Luo, and D. Schuurmans, “A simple decentralized cross-entropy method,” Advances in Neural Information Processing Systems, vol. 35, pp. 36 495–36 506, 2022.
  128. I. M. Balci, E. Bakolas, B. Vlahov, and E. A. Theodorou, “Constrained covariance steering based Tube-MPPI,” in 2022 American Control Conference (ACC).   IEEE, 2022, pp. 4197–4202.
  129. F. Stulp and O. Sigaud, “Path integral policy improvement with covariance matrix adaptation,” arXiv preprint arXiv:1206.4621, 2012.
  130. ——, “Policy improvement methods: Between black-box optimization and episodic reinforcement learning,” HAL Open Science, 2012.
  131. T. Lefebvre and G. Crevecoeur, “Path integral policy improvement with differential dynamic programming,” in 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).   IEEE, 2019, pp. 739–745.
  132. P. Varnai and D. V. Dimarogonas, “Path integral policy improvement: An information-geometric optimization approach,” 2020. [Online]. Available: 10.13140/RG.2.2.13969.76645.
  133. J. Fu, C. Li, X. Teng, F. Luo, and B. Li, “Compound heuristic information guided policy improvement for robot motor skill acquisition,” Applied Sciences, vol. 10, no. 15, p. 5346, 2020.
  134. H. Ba, J. Fan, X. Guo, and J. Hao, “Critic PI2: Master continuous planning via policy improvement with path integrals and deep actor-critic reinforcement learning,” in 2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM).   IEEE, 2021, pp. 716–722.
  135. H. J. Kappen, “An introduction to stochastic control theory, path integrals and reinforcement learning,” in AIP Conference Proceedings, vol. 887, no. 1.   American Institute of Physics, 2007, pp. 149–181.
  136. H. Kappen, “Optimal control theory and the linear bellman equation,” Barber, D.; Cemgil, AT; Chiappa, S.(ed.), Bayesian time series models, pp. 363–387, 2011.
  137. E. A. Theodorou, “Iterative path integral stochastic optimal control: Theory and applications to motor control,” Ph.D. dissertation, University of Southern California, 2011.
  138. S. Thijssen and H. Kappen, “Path integral control and state-dependent feedback,” Physical Review E, vol. 91, no. 3, p. 032104, 2015.
  139. W. H. Fleming and W. M. McEneaney, “Risk-sensitive control on an infinite time horizon,” SIAM Journal on Control and Optimization, vol. 33, no. 6, pp. 1881–1915, 1995.
  140. E. A. Theodorou, “Nonlinear stochastic control and information theoretic dualities: Connections, interdependencies and thermodynamic interpretations,” Entropy, vol. 17, no. 5, pp. 3352–3375, 2015.
  141. A. Kupcsik, M. Deisenroth, J. Peters, and G. Neumann, “Data-efficient contextual policy search for robot movement skills,” in Proceedings of the National Conference on Artificial Intelligence (AAAI).   Bellevue, 2013.
  142. N. Hansen and A. Ostermeier, “Completely derandomized self-adaptation in evolution strategies,” Evolutionary Computation, vol. 9, no. 2, pp. 159–195, 2001.
  143. N. Hansen, “The CMA evolution strategy: A tutorial,” arXiv preprint arXiv:1604.00772, 2016.
  144. M. P. Deisenroth, G. Neumann, J. Peters et al., “A survey on policy search for robotics,” Foundations and Trends® in Robotics, vol. 2, no. 1–2, pp. 1–142, 2013.
  145. J. Vinogradska, B. Bischoff, J. Achterhold, T. Koller, and J. Peters, “Numerical quadrature for probabilistic policy search,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 1, pp. 164–175, 2020.
  146. M. P. Deisenroth, D. Fox, and C. E. Rasmussen, “Gaussian processes for data-efficient learning in robotics and control,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 2, pp. 408–423, 2013.
  147. M. Thor, T. Kulvicius, and P. Manoonpong, “Generic neural locomotion control framework for legged robots,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 9, pp. 4013–4025, 2020.
  148. S. Schaal, J. Peters, J. Nakanishi, and A. Ijspeert, “Learning movement primitives,” in Robotics research. the eleventh international symposium.   Springer, 2005, pp. 561–572.
  149. A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, “Dynamical movement primitives: Learning attractor models for motor behaviors,” Neural computation, vol. 25, no. 2, pp. 328–373, 2013.
  150. S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” The Journal of Machine Learning Research, vol. 17, no. 1, pp. 1334–1373, 2016.
  151. A. S. Polydoros and L. Nalpantidis, “Survey of model-based reinforcement learning: Applications on robotics,” Journal of Intelligent & Robotic Systems, vol. 86, no. 2, pp. 153–173, 2017.
  152. L. C. Garaffa, M. Basso, A. A. Konzen, and E. P. de Freitas, “Reinforcement learning for mobile robotics exploration: A survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 8, pp. 3796–3810, 2021.
  153. T. M. Moerland, J. Broekens, A. Plaat, C. M. Jonker et al., “Model-based reinforcement learning: A survey,” Foundations and Trends® in Machine Learning, vol. 16, no. 1, pp. 1–118, 2023.
  154. R. S. Sutton, “Integrated architectures for learning, planning, and reacting based on approximating dynamic programming,” in Machine Learning Proceedings 1990.   Elsevier, 1990, pp. 216–224.
  155. M. Deisenroth and C. E. Rasmussen, “PILCO: A model-based and data-efficient approach to policy search,” in Proceedings of the 28th International Conference on machine learning (ICML-11), 2011, pp. 465–472.
  156. M. Janner, J. Fu, M. Zhang, and S. Levine, “When to trust your model: Model-based policy optimization,” Advances in neural information processing systems, vol. 32, pp. 1–9, 2019.
  157. T. Yu, G. Thomas, L. Yu, S. Ermon, J. Y. Zou, S. Levine, C. Finn, and T. Ma, “MOPO: Model-based offline policy optimization,” Advances in Neural Information Processing Systems, vol. 33, pp. 14 129–14 142, 2020.
  158. J. H. Kotecha and P. M. Djuric, “Gaussian sum particle filtering,” IEEE Transactions on signal processing, vol. 51, no. 10, pp. 2602–2612, 2003.
  159. Q. Zhang and Y. Chen, “Path integral sampler: A stochastic control approach for sampling,” in International Conference on Learning Representations, 2022.
  160. A. Patil, Y. Zhou, D. Fridovich-Keil, and T. Tanaka, “Risk-minimizing two-player zero-sum stochastic differential game via path integral control,” arXiv preprint arXiv:2308.11546, 2023.
  161. N. Wan, A. Gahlawat, N. Hovakimyan, E. A. Theodorou, and P. G. Voulgaris, “Distributed algorithms for linearly-solvable optimal control in networked multi-agent systems,” arXiv preprint arXiv:2102.09104, 2021.
  162. L. Song, N. Wan, A. Gahlawat, C. Tao, N. Hovakimyan, and E. A. Theodorou, “Generalization of safe optimal control actions on networked multiagent systems,” IEEE Transactions on Control of Network Systems, vol. 10, no. 1, pp. 491–502, 2022.
  163. L. Song, P. Zhao, N. Wan, and N. Hovakimyan, “Safety embedded stochastic optimal control of networked multi-agent systems via barrier states,” in 2023 American Control Conference (ACC).   IEEE, 2023, pp. 2554–2559.
  164. M. Watterson, S. Liu, K. Sun, T. Smith, and V. Kumar, “Trajectory optimization on manifolds with applications to SO(3) and ℝ3×𝕊2superscriptℝ3superscript𝕊2\mathbb{R}^{3}\times\mathbb{S}^{2}blackboard_R start_POSTSUPERSCRIPT 3 end_POSTSUPERSCRIPT × blackboard_S start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT,” in Robotics: Science and Systems, 2018, p. 9.
  165. R. Bonalli, A. Bylard, A. Cauligi, T. Lew, and M. Pavone, “Trajectory optimization on manifolds: A theoretically-guaranteed embedded sequential convex programming approach,” arXiv preprint arXiv:1905.07654, 2019.
  166. M. Watterson, S. Liu, K. Sun, T. Smith, and V. Kumar, “Trajectory optimization on manifolds with applications to quadrotor systems,” The International Journal of Robotics Research, vol. 39, no. 2-3, pp. 303–320, 2020.
  167. T. Osa, “Motion planning by learning the solution manifold in trajectory optimization,” The International Journal of Robotics Research, vol. 41, no. 3, pp. 281–311, 2022.
  168. N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre, “Manopt, a Matlab toolbox for optimization on manifolds,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 1455–1459, 2014.
  169. H. M. Menegaz, J. Y. Ishihara, and H. T. Kussaba, “Unscented Kalman filters for Riemannian state-space systems,” IEEE Transactions on Automatic Control, vol. 64, no. 4, pp. 1487–1502, 2018.
  170. M. Brossard, A. Barrau, and S. Bonnabel, “A code for unscented Kalman filtering on manifolds (UKF-M),” in IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 5701–5708.
  171. K. Li, F. Pfaff, and U. D. Hanebeck, “Unscented dual quaternion particle filter for SE(3) estimation,” IEEE Control Systems Letters, vol. 5, no. 2, pp. 647–652, 2020.
  172. T. Cantelobre, C. Chahbazian, A. Croux, and S. Bonnabel, “A real-time unscented Kalman filter on manifolds for challenging AUV navigation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 2309–2316.
Citations (11)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now