Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models (2308.01557v2)
Abstract: Learning priors on trajectory distributions can help accelerate robot motion planning optimization. Given previously successful plans, learning trajectory generative models as priors for a new planning problem is highly desirable. Prior works propose several ways on utilizing this prior to bootstrapping the motion planning problem. Either sampling the prior for initializations or using the prior distribution in a maximum-a-posterior formulation for trajectory optimization. In this work, we propose learning diffusion models as priors. We then can sample directly from the posterior trajectory distribution conditioned on task goals, by leveraging the inverse denoising process of diffusion models. Furthermore, diffusion has been recently shown to effectively encode data multimodality in high-dimensional settings, which is particularly well-suited for large trajectory dataset. To demonstrate our method efficacy, we compare our proposed method - Motion Planning Diffusion - against several baselines in simulated planar robot and 7-dof robot arm manipulator environments. To assess the generalization capabilities of our method, we test it in environments with previously unseen obstacles. Our experiments show that diffusion models are strong priors to encode high-dimensional trajectory distributions of robot motions.
- N. Ratliff, M. Zucker, J. A. Bagnell, and S. Srinivasa, “Chomp: Gradient optimization techniques for efficient motion planning,” in IEEE International Conference on Robotics and Automation, 2009.
- M. Kalakrishnan, S. Chitta, E. Theodorou, P. Pastor, and S. Schaal, “Stomp: Stochastic trajectory optimization for motion planning,” in IEEE International Conference on Robotics and Automation, 2011.
- M. Elbanhawi and M. Simic, “Sampling-based robot motion planning: A review,” IEEE Access, vol. 2, pp. 56–77, 2014.
- L. Kavraki, P. Svestka, J.-C. Latombe, and M. Overmars, “Probabilistic roadmaps for path planning in high-dimensional configuration spaces,” IEEE Transactions on Robotics and Automation, 1996.
- S. M. Lavalle, “Rapidly-exploring random trees: A new tool for path planning,” 1998.
- J. Kuffner and S. LaValle, “Rrt-connect: An efficient approach to single-query path planning,” in IEEE ICRA, 2000.
- S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning,” IJRR, vol. 30, no. 7, pp. 846–894, 2011.
- J. Dong, M. Mukadam, F. Dellaert, and B. Boots, “Motion planning as probabilistic inference using Gaussian processes and factor graphs,” in RSS, 2016.
- M. Mukadam, J. Dong, X. Yan, F. Dellaert, and B. Boots, “Continuous-time gaussian process motion planning via probabilistic inference,” Int. J. Robotics Res., vol. 37, no. 11, 2018.
- K. K. Hauser and V. Ng-Thow-Hing, “Fast smoothing of manipulator trajectories using optimal bounded-acceleration shortcuts.” in ICRA. IEEE, 2010, pp. 2493–2498.
- J. Urain, A. Le, A. Lambert, G. Chalvatzaki, B. Boots, and J. Peters, “Learning implicit priors for motion optimization,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022.
- J. Wang et al., “A survey of learning-based robot motion planning,” IET Cyber-Systems and Robotics, vol. 3, no. 4, pp. 302–314, 2021.
- B. Ichter, J. Harrison, and M. Pavone, “Learning sampling distributions for robot motion planning,” in IEEE ICRA, 2018.
- A. H. Qureshi, A. Simeonov, M. J. Bency, and M. C. Yip, “Motion planning networks,” in IEEE ICRA, 2019.
- D. Koert, G. Maeda, R. Lioutikov, G. Neumann, and J. Peters, “Demonstration based trajectory optimization for generalizable robot motions,” in IEEE-RAS Humanoids, 2016, pp. 515–522.
- M. A. Rana, M. Mukadam, S. R. Ahmadzadeh, S. Chernova, and B. Boots, “Towards robust skill generalization: Unifying learning from demonstration and motion planning,” in CoRL. PMLR, 2017.
- A. T. Le et al., “Learning forceful manipulation skills from multi-modal human demonstrations,” in IEEE/RSJ IROS, 2021.
- J. Urain, N. Funk, J. Peters, and G. Chalvatzaki, “Se(3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion,” in IEEE ICRA, 2023.
- J. Ortiz-Haro, J.-S. Ha, D. Driess, and M. Toussaint, “Structured deep generative models for sampling on constraint manifolds in sequential manipulation,” in CoRL. PMLR, 2022.
- T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, and J. Peters, “An algorithmic perspective on imitation learning,” Found. Trends Robotics, vol. 7, no. 1-2, pp. 1–179, 2018.
- J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in ICML. JMLR.org, 2015.
- J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in NeurIPS. Curran Associates Inc., 2020.
- Y. Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” in NeurIPS, 2019.
- R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” 2021.
- G. Kim, T. Kwon, and J. C. Ye, “Diffusionclip: Text-guided diffusion models for robust image manipulation,” in IEEE/CVF CVPR, 2022.
- A. Ramesh et al., “Hierarchical text-conditional image generation with clip latents,” 2022. [Online]. Available: https://arxiv.org/abs/2204.06125
- P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” 2021.
- J. J. Johnson et al., “Motion planning transformers: A motion planning framework for mobile robots,” 2021. [Online]. Available: https://arxiv.org/abs/2106.02791
- A. H. Qureshi and M. C. Yip, “Deeply informed neural sampling for robot motion planning,” in IEEE/RSJ IROS, 2018.
- J. Wang, W. Chi, C. Li, C. Wang, and M. Q.-H. Meng, “Neural rrt*: Learning-based optimal path planning,” IEEE T-ASE, 2020.
- W. Grathwohl, J. Kelly, M. Hashemi, M. Norouzi, K. Swersky, and D. Duvenaud, “No mcmc for me: Amortized sampling for fast and stable training of energy-based models,” 2020.
- I. Kapelyukh, V. Vosylius, and E. Johns, “Dall-e-bot: Introducing web-scale diffusion models to robotics,” 2022.
- W. Liu, T. Hermans, S. Chernova, and C. Paxton, “Structdiffusion: Object-centric diffusion for semantic rearrangement of novel objects,” 2022. [Online]. Available: https://arxiv.org/abs/2211.04604
- E. J. C. Findlay, H. Zhang, Z. Chang, and H. P. H. Shum, “Denoising diffusion probabilistic models for styled walking synthesis,” 2022. [Online]. Available: https://arxiv.org/abs/2209.14828
- J. Carvalho, M. Baierl, J. Urain, and J. Peters, “Conditioned score-based models for learning collision-free trajectory generation,” in NeurIPS 2022 Workshop on Score-Based Methods, 2022.
- M. Janner, Y. Du, J. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” in ICML, 2022.
- A. Ajay et al., “Is conditional generative modeling all you need for decision-making?” 2022. [Online]. Available: https://arxiv.org/abs/2211.15657
- H. Attias, “Planning by probabilistic inference,” in Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, vol. R4. PMLR, 2003, pp. 9–16.
- M. Toussaint, “Robot trajectory optimization using approximate inference,” in ICML. Association for Computing Machinery, 2009.
- S. Levine, “Reinforcement learning and control as probabilistic inference: Tutorial and review,” 2018.
- J. Peters, K. Mülling, and Y. Altün, “Relative entropy policy search,” in AAAI. AAAI Press, 2010.
- J. Watson, H. Abdulsamad, and J. Peters, “Stochastic optimal control as approximate input inference,” in CoRL. PMLR, 2019.
- J. Watson and J. Peters, “Inferring smooth control: Monte carlo posterior policy iteration with gaussian processes,” in CoRL, 2022.
- A. T. Le, K. Hansel, J. Peters, and G. Chalvatzaki, “Hierarchical policy blending as optimal transport,” in arXiv preprint arXiv:2212.01938, 2022.
- J. Dong, M. Mukadam, B. Boots, and F. Dellaert, “Sparse Gaussian processes on matrix Lie groups: A unified framework for optimizing continuous-time trajectories,” in IEEE ICRA, 2018.
- A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in ICML. PMLR, 2021.
- P. Dhariwal and A. Q. Nichol, “Diffusion models beat gans on image synthesis,” in NeurIPS, 2021, pp. 8780–8794.
- A. Paszke et al., “Pytorch: An imperative style, high-performance deep learning library,” in NeurIPS. Curran Associates, Inc., 2019.
- J. Sola, J. Deray, and D. Atchuthan, “A micro lie theory for state estimation in robotics,” arXiv preprint arXiv:1812.01537, 2018.
- T. D. Barfoot, C. H. Tong, and S. Särkkä, “Batch continuous-time trajectory estimation as exactly sparse gaussian process regression.” in RSS, 2014.
- K. Sohn, H. Lee, and X. Yan, “Learning structured output representation using deep conditional generative models,” in NeurIPS, vol. 28. Curran Associates, Inc., 2015.
- M. Bhardwaj, B. Boots, and M. Mukadam, “Differentiable gaussian process motion planning,” in IEEE ICRA, 2020.