RETRO: Reactive Trajectory Optimization for Real-Time Robot Motion Planning in Dynamic Environments (2310.01738v1)
Abstract: Reactive trajectory optimization for robotics presents formidable challenges, demanding the rapid generation of purposeful robot motion in complex and swiftly changing dynamic environments. While much existing research predominantly addresses robotic motion planning with predefined objectives, emerging problems in robotic trajectory optimization frequently involve dynamically evolving objectives and stochastic motion dynamics. However, effectively addressing such reactive trajectory optimization challenges for robot manipulators proves difficult due to inefficient, high-dimensional trajectory representations and a lack of consideration for time optimization. In response, we introduce a novel trajectory optimization framework called RETRO. RETRO employs adaptive optimization techniques that span both spatial and temporal dimensions. As a result, it achieves a remarkable computing complexity of $O(T{2.4}) + O(Tn{2})$, a significant improvement over the traditional application of DDP, which leads to a complexity of $O(n{4})$ when reasonable time step sizes are used. To evaluate RETRO's performance in terms of error, we conducted a comprehensive analysis of its regret bounds, comparing it to an Oracle value function obtained through an Oracle trajectory optimization algorithm. Our analytical findings demonstrate that RETRO's total regret can be upper-bounded by a function of the chosen time step size. Moreover, our approach delivers smoothly optimized robot trajectories within the joint space, offering flexibility and adaptability for various tasks. It can seamlessly integrate task-specific requirements such as collision avoidance while maintaining real-time control rates. We validate the effectiveness of our framework through extensive simulations and real-world robot experiments in closed-loop manipulation scenarios.
- G. Alcan and V. Kyrki, “Differential dynamic programming with nonlinear safety constraints under system uncertainties,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1760–1767, 2022.
- Z. Xie, C. K. Liu, and K. Hauser, “Differential dynamic programming with nonlinear constraints,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 695–702.
- D. Carneiro, F. Silva, and P. Georgieva, “Robot anticipation learning system for ball catching,” Robotics, vol. 10, no. 4, 2021. [Online]. Available: https://www.mdpi.com/2218-6581/10/4/113
- A. A. Oliva, E. Aertbeliën, J. De Schutter, P. R. Giordano, and F. Chaumette, “Towards dynamic visual servoing for interaction control and moving targets,” in 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 150–156.
- A. Dastider and M. Lin, “Non-parametric stochastic policy gradient with strategic retreat for non-stationary environment,” in 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), 2022, pp. 1377–1384.
- S. Kim, A. Shukla, and A. Billard, “Catching objects in flight,” IEEE Transactions on Robotics, vol. 30, no. 5, pp. 1049–1065, 2014.
- P. Zhao, Y.-F. Xie, L. Zhang, and Z.-H. Zhou, “Efficient methods for non-stationary online learning,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 11 573–11 585. [Online]. Available: https://proceedings.neurips.cc/paper˙files/paper/2022/file/4b70484ebef62484e0c8cdd269e482fd-Paper-Conference.pdf
- C. Mastalli, W. Merkt, J. Marti-Saumell, H. Ferrolho, J. Solà, N. Mansard, and S. Vijayakumar, “A feasibility-driven approach to control-limited ddp,” Autonomous Robots, vol. 46, no. 8, pp. 985–1005, Dec. 2022.
- Y. Pan and E. Theodorou, “Probabilistic differential dynamic programming,” in Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger, Eds., vol. 27. Curran Associates, Inc., 2014.
- Y. Aoyama, A. D. Saravanos, and E. A. Theodorou, “Receding horizon differential dynamic programming under parametric uncertainty,” in 2021 60th IEEE Conference on Decision and Control (CDC), 2021, pp. 3761–3767.
- Y. Pan and E. A. Theodorou, “Data-driven differential dynamic programming using gaussian processes,” in 2015 American Control Conference (ACC), 2015, pp. 4467–4472.
- A. Oshin, M. D. Houghton, M. J. Acheson, I. M. Gregory, and E. A. Theodorou, “Parameterized differential dynamic programming,” arXiv preprint arXiv:2204.03727, 2022.
- W. Jallet, N. Mansard, and J. Carpentier, “Implicit differential dynamic programming,” in 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 1455–1461.
- B. Light, “The Principle of Optimality in Dynamic Programming: A Pedagogical Note,” arXiv e-prints, p. arXiv:2302.08467, Feb. 2023.
- L.-Z. Liao and C. A. Shoemaker, “Convergence in unconstrained discrete-time differential dynamic programming,” IEEE Transactions on Automatic Control, vol. 36, no. 6, pp. 692–706, 1991.
- E. Eirola and A. Lendasse, “Gaussian mixture models for time series modelling, forecasting, and interpolation,” in Proceedings of the 12th International Symposium on Advances in Intelligent Data Analysis XII - Volume 8207, ser. IDA 2013. Berlin, Heidelberg: Springer-Verlag, 2013, p. 162–173. [Online]. Available: https://doi.org/10.1007/978-3-642-41398-8˙15
- M. F. Steel, “Bayesian time series analysis,” in Macroeconometrics and time series analysis. Springer, 2010, pp. 35–45.
- B. Plancher, “Parallel and constrained differential dynamic programming for model predictive control,” Master’s thesis, Harvard University, Cambridge, MA, USA, May. 2018.
- E. Todorov, “Efficient computation of optimal actions,” Proceedings of the National Academy of Sciences, vol. 106, no. 28, pp. 11 478–11 483, July 2009. [Online]. Available: https://doi.org/10.1073/pnas.0710743106
- G. Neu and V. Gómez, “Fast rates for online learning in Linearly Solvable Markov Decision Processes,” in Proceedings of the 2017 Conference on Learning Theory, ser. Proceedings of Machine Learning Research, S. Kale and O. Shamir, Eds., vol. 65. PMLR, 07–10 Jul 2017, pp. 1567–1588. [Online]. Available: https://proceedings.mlr.press/v65/neu17a.html
- R. Tedrake and the Drake Development Team, “Drake: Model-based design and verification for robotics,” 2019. [Online]. Available: https://drake.mit.edu