Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Subgoal Diffuser: Coarse-to-fine Subgoal Generation to Guide Model Predictive Control for Robot Manipulation (2403.13085v1)

Published 19 Mar 2024 in cs.RO

Abstract: Manipulation of articulated and deformable objects can be difficult due to their compliant and under-actuated nature. Unexpected disturbances can cause the object to deviate from a predicted state, making it necessary to use Model-Predictive Control (MPC) methods to plan motion. However, these methods need a short planning horizon to be practical. Thus, MPC is ill-suited for long-horizon manipulation tasks due to local minima. In this paper, we present a diffusion-based method that guides an MPC method to accomplish long-horizon manipulation tasks by dynamically specifying sequences of subgoals for the MPC to follow. Our method, called Subgoal Diffuser, generates subgoals in a coarse-to-fine manner, producing sparse subgoals when the task is easily accomplished by MPC and more dense subgoals when the MPC method needs more guidance. The density of subgoals is determined dynamically based on a learned estimate of reachability, and subgoals are distributed to focus on challenging parts of the task. We evaluate our method on two robot manipulation tasks and find it improves the planning performance of an MPC method, and also outperforms prior diffusion-based methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive driving with model predictive path integral control,” in 2016 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2016, pp. 1433–1440.
  2. J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020.
  3. M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” arXiv preprint arXiv:2205.09991, 2022.
  4. A. Ajay, Y. Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal, “Is conditional generative modeling all you need for decision-making?” arXiv preprint arXiv:2211.15657, 2022.
  5. C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” arXiv preprint arXiv:2303.04137, 2023.
  6. W. Li, X. Wang, B. Jin, and H. Zha, “Hierarchical diffusion for offline decision making,” 2023.
  7. C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, et al., “Photorealistic text-to-image diffusion models with deep language understanding,” Advances in Neural Information Processing Systems, vol. 35, pp. 36 479–36 494, 2022.
  8. J. Ho, W. Chan, C. Saharia, J. Whang, R. Gao, A. Gritsenko, D. P. Kingma, B. Poole, M. Norouzi, D. J. Fleet, et al., “Imagen video: High definition video generation with diffusion models,” arXiv preprint arXiv:2210.02303, 2022.
  9. T. Yu, T. Xiao, A. Stone, J. Tompson, A. Brohan, S. Wang, J. Singh, C. Tan, J. Peralta, B. Ichter, et al., “Scaling robot learning with semantically imagined experience,” arXiv preprint arXiv:2302.11550, 2023.
  10. H. Bharadhwaj, J. Vakil, M. Sharma, A. Gupta, S. Tulsiani, and V. Kumar, “Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking,” arXiv preprint arXiv:2309.01918, 2023.
  11. J. Urain, N. Funk, J. Peters, and G. Chalvatzaki, “Se (3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 5923–5930.
  12. I. Kapelyukh, V. Vosylius, and E. Johns, “Dall-e-bot: Introducing web-scale diffusion models to robotics,” IEEE Robotics and Automation Letters, 2023.
  13. W. Liu, T. Hermans, S. Chernova, and C. Paxton, “Structdiffusion: Object-centric diffusion for semantic rearrangement of novel objects,” arXiv preprint arXiv:2211.04604, 2022.
  14. T. Power, R. Soltani-Zarrin, S. Iba, and D. Berenson, “Sampling constrained trajectories using composable diffusion models,” in IROS 2023 Workshop on Differentiable Probabilistic Robotics: Emerging Perspectives on Robot Learning, 2023.
  15. J. Carvalho, A. T. Le, M. Baierl, D. Koert, and J. Peters, “Motion planning diffusion: Learning and planning of robot motions with diffusion models,” arXiv preprint arXiv:2308.01557, 2023.
  16. S. Huang, Z. Wang, P. Li, B. Jia, T. Liu, Y. Zhu, W. Liang, and S.-C. Zhu, “Diffusion-based generation, optimization, and planning in 3d scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 16 750–16 761.
  17. T. Pearce, T. Rashid, A. Kanervisto, D. Bignell, M. Sun, R. Georgescu, S. V. Macua, S. Z. Tan, I. Momennejad, K. Hofmann, et al., “Imitating human behaviour with diffusion models,” arXiv preprint arXiv:2301.10677, 2023.
  18. M. Reuss, M. Li, X. Jia, and R. Lioutikov, “Goal-conditioned imitation learning using score-based diffusion policies,” arXiv preprint arXiv:2304.02532, 2023.
  19. S. Nair and C. Finn, “Hierarchical foresight: Self-supervised learning of long-horizon tasks via visual subgoal generation,” arXiv preprint arXiv:1909.05829, 2019.
  20. B. Eysenbach, R. R. Salakhutdinov, and S. Levine, “Search on the replay buffer: Bridging planning and reinforcement learning,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  21. S. Nasiriany, V. Pong, S. Lin, and S. Levine, “Planning with goal-conditioned policies,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  22. K. Fang, P. Yin, A. Nair, and S. Levine, “Planning to practice: Efficient online fine-tuning by composing goals in latent space,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 4076–4083.
  23. X. Lin, Z. Huang, Y. Li, J. B. Tenenbaum, D. Held, and C. Gan, “Diffskill: Skill abstraction from differentiable physics for deformable object manipulations with tools,” arXiv preprint arXiv:2203.17275, 2022.
  24. T. Power and D. Berenson, “Variational inference mpc using normalizing flows and out-of-distribution projection,” arXiv preprint arXiv:2205.04667, 2022.
  25. T. Wang and J. Ba, “Exploring model-based planning with policy networks,” arXiv preprint arXiv:1906.08649, 2019.
  26. J. Sacks and B. Boots, “Learning to optimize in model predictive control,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 10 549–10 556.
  27. L. Li, Y. Miao, A. H. Qureshi, and M. C. Yip, “Mpc-mpnet: Model-predictive motion planning networks for fast, near-optimal planning under kinodynamic constraints,” IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 4496–4503, 2021.
  28. J. Carius, R. Ranftl, F. Farshidian, and M. Hutter, “Constrained stochastic optimal control with learned importance sampling: A path integral approach,” The International Journal of Robotics Research, vol. 41, no. 2, pp. 189–209, 2022.
  29. T. Lai, W. Zhi, T. Hermans, and F. Ramos, “Parallelised diffeomorphic sampling-based motion planning,” in Conference on Robot Learning.   PMLR, 2022, pp. 81–90.
  30. B. Ichter, J. Harrison, and M. Pavone, “Learning sampling distributions for robot motion planning,” in 2018 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2018, pp. 7087–7094.
  31. J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning.   PMLR, 2015, pp. 2256–2265.
  32. M. Welling and Y. W. Teh, “Bayesian learning via stochastic gradient langevin dynamics,” in Proceedings of the 28th international conference on machine learning (ICML-11).   Citeseer, 2011, pp. 681–688.
  33. C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” arXiv preprint arXiv:1706.02413, 2017.
  34. J. Hejna, J. Gao, and D. Sadigh, “Distance weighted supervised learning for offline interaction data,” arXiv preprint arXiv:2304.13774, 2023.
  35. E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ international conference on intelligent robots and systems.   IEEE, 2012, pp. 5026–5033.
  36. M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, O. Pieter Abbeel, and W. Zaremba, “Hindsight experience replay,” Advances in neural information processing systems, vol. 30, 2017.
  37. Y. Wang, D. McConachie, and D. Berenson, “Tracking partially-occluded deformable objects while enforcing geometric constraints,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 14 199–14 205.
Citations (1)
View on Semantic Scholar

Summary

  • The paper introduces a diffusion-based subgoal generation technique that dynamically adjusts subgoal density to guide MPC for complex tasks.
  • It employs a reachability-based assessment to adapt subgoal resolution, ensuring effective long-horizon planning and efficiency.
  • Experimental results reveal significantly lower mean distances to goals compared to state-of-the-art methods in rope and notebook manipulation tasks.

Subgoal Diffuser: Enhancing Long-Horizon Robot Manipulation with Subgoal Generation and Model Predictive Control

The paper "Subgoal Diffuser: Coarse-to-fine Subgoal Generation to Guide Model Predictive Control for Robot Manipulation" introduces a novel approach to address the challenges associated with long-horizon robotic manipulation tasks, particularly involving articulated and deformable objects. Model Predictive Control (MPC), although well-suited for adaptive control under disturbances, struggles with long-horizon planning due to its reliance on short planning horizons and the risk of becoming trapped in local minima. This paper presents an innovative method called Subgoal Diffuser, which dynamically specifies a sequence of subgoals to enhance the effectiveness of MPC across extended tasks.

Methodology

The Subgoal Diffuser employs a diffusion-based learning approach to generate subgoals in a coarse-to-fine manner. These subgoals serve as a guide for MPC, allowing the method to tackle complex tasks by dynamically determining subgoal density based on learned estimates of reachability. The method's strength lies in its ability to adjust subgoal resolution according to task difficulty, generating sparse subgoals for straightforward parts and dense subgoals for more challenging parts requiring careful guidance.

Key components of the Subgoal Diffuser include:

  1. Conditional Generative Model: The model predicts sequences of subgoals from the current to the goal state, helping MPC navigate through both straightforward and intricate aspects of a task.
  2. Reachability-Based Assessment: By learning a reachability metric, the method decides dynamically on the number of subgoals, ensuring they are dense enough to prevent the MPC from reaching local minima while being sparse enough to avoid inefficiency.
  3. Simultaneous Integration with MPC: The paper effectively integrates subgoal generation with sampling-based MPC, enabling low-level control and ensuring the MPC exploits the dynamically generated subgoal guidance.

Experimental Results

The authors validate their approach on two robot manipulation tasks: rope reconfiguration and notebook manipulation. The Subgoal Diffuser demonstrated superior performance compared to state-of-the-art diffusion-based decision-making systems, specifically Diffusion Policy and Decision Diffuser. It effectively prevented the MPC-controlled robot from falling into local minima, outperforming both modern learning-based tactics and traditional MPC solutions.

Noteworthy performance results include:

  • The method achieved a mean minimum distance to goal significantly lower than the compared baselines, consistently verifying the practical efficacy of the dynamic subgoal resolution adaptation.
  • Ablation studies underscored the importance of the proposed coarse-to-fine generation structure and the adaptive resolution mechanism, emphasizing their contribution to performance enhancement.

Implications and Future Directions

The introduction of the Subgoal Diffuser opens new avenues in robotic manipulation, particularly for tasks characterized by long time horizons and complex dynamics. By bridging the gap between high-level reasoning and low-level control, the method demonstrates the potential for more robust and adaptable robotic systems.

Future research could pioneer several aspects:

  • Exploration of more sophisticated reachability metrics to further enhance subgoal allocation strategies.
  • Extension of the Subgoal Diffuser's applications to broader robotic domains, incorporating various object types and dynamic environments.
  • Further investigation into real-world applicability, refining dynamics models for enhanced transferability from simulated to real-world experiments.

In conclusion, the Subgoal Diffuser offers a practical and efficient solution to long-horizon robot manipulation challenges. By leveraging diffusion-based subgoal generation in conjunction with MPC, it takes significant strides towards more autonomous and reliable robotic systems capable of overcoming intricate manipulation tasks.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets