Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
123 tokens/sec
GPT-4o
11 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
4 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Diff-Transfer: Model-based Robotic Manipulation Skill Transfer via Differentiable Physics Simulation (2310.04930v2)

Published 7 Oct 2023 in cs.RO, cs.AI, and cs.LG

Abstract: The capability to transfer mastered skills to accomplish a range of similar yet novel tasks is crucial for intelligent robots. In this work, we introduce $\textit{Diff-Transfer}$, a novel framework leveraging differentiable physics simulation to efficiently transfer robotic skills. Specifically, $\textit{Diff-Transfer}$ discovers a feasible path within the task space that brings the source task to the target task. At each pair of adjacent points along this task path, which is two sub-tasks, $\textit{Diff-Transfer}$ adapts known actions from one sub-task to tackle the other sub-task successfully. The adaptation is guided by the gradient information from differentiable physics simulations. We propose a novel path-planning method to generate sub-tasks, leveraging $Q$-learning with a task-level state and reward. We implement our framework in simulation experiments and execute four challenging transfer tasks on robotic manipulation, demonstrating the efficacy of $\textit{Diff-Transfer}$ through comprehensive experiments. Supplementary and Videos are on the website https://sites.google.com/view/difftransfer

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Ceres solver. http://ceres-solver.org.
  2. Rethinking optimization with differentiable simulation from a global perspective. In 6th Annual Conference on Robot Learning, 2022.
  3. Bradley Bell. Cppad: a package for c++ algorithmic differentiation. http://www.coin-or.org/CppAD, 2020.
  4. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.
  5. Trajectotree: Trajectory optimization meets tree search for planning multi-contact dexterous manipulation. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.  8262–8268, 2021. doi: 10.1109/IROS51168.2021.9636346.
  6. Contact mode guided sampling-based planning for quasistatic dexterous manipulation in 2d. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp.  6520–6526, 2021. doi: 10.1109/ICRA48506.2021.9560766.
  7. Iterative residual policy for goal-conditioned dynamic manipulation of deformable objects. In Proceedings of Robotics: Science and Systems (RSS), 2022.
  8. End-to-end differentiable physics for learning and control. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper/2018/file/842424a1d0595b76ec4fa03c46e8d755-Paper.pdf.
  9. A differentiable physics engine for deep learning in robotics. Frontiers in neurorobotics, pp.  6, 2019.
  10. Diffpd: Differentiable projective dynamics. ACM Trans. Graph., 41(2), nov 2021. ISSN 0730-0301. doi: 10.1145/3490168. URL https://doi.org/10.1145/3490168.
  11. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp.  1126–1135. PMLR, 2017.
  12. A Computational Framework for Environment-Aware Robotic Manipulation Planning, pp.  363–385. Springer International Publishing, 2018. ISBN 978-3-319-60916-4. doi: 10.1007/978-3-319-60916-4˙21.
  13. Add: Analytically differentiable dynamics for multi-body systems with frictional contact. ACM Transactions on Graphics (TOG), 39(6):1–15, 2020.
  14. Joint optimization of robot design and motion parameters using the implicit function theorem. In Siddhartha Srinivasa, Nora Ayanian, Nancy Amato, and Scott Kuindersma (eds.), Robotics, Robotics: Science and Systems, United States, 2017. MIT Press Journals. doi: 10.15607/rss.2017.xiii.003. Publisher Copyright: © 2017 MIT Press Journals. All rights reserved.; 2017 Robotics: Science and Systems, RSS 2017 ; Conference date: 12-07-2017 Through 16-07-2017.
  15. Learning to control pdes with differentiable physics. arXiv preprint arXiv:2001.07457, 2020.
  16. Taichi: a language for high-performance computation on spatially sparse data structures. ACM Transactions on Graphics (TOG), 38(6):201, 2019a.
  17. Chainqueen: A real-time differentiable physical simulator for soft robotics. In 2019 International conference on robotics and automation (ICRA), pp.  6265–6271. IEEE, 2019b.
  18. Reboot: Reuse data for bootstrapping efficient real-world dexterous manipulation. arXiv preprint arXiv:2309.03322, 2023.
  19. Human-oriented Representation Learning for Robotic Manipulation. arXiv e-prints, art. arXiv:2310.03023, October 2023.
  20. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2):3019–3026, 2020.
  21. gradsim: Differentiable simulation for system identification and visuomotor control. arXiv preprint arXiv:2104.02646, 2021.
  22. Adversarial skill learning for robust manipulation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp.  2555–2561. IEEE, 2021.
  23. Building portable options: Skill transfer in reinforcement learning. In Ijcai, volume 7, pp.  895–900, 2007.
  24. Transfer of samples in batch reinforcement learning. In Proceedings of the 25th international conference on Machine learning, pp.  544–551, 2008.
  25. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM Trans. Graph., mar 2022. ISSN 0730-0301. doi: 10.1145/3527660. URL https://doi.org/10.1145/3527660. Just Accepted.
  26. Differentiable cloth simulation for inverse problems. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper/2019/file/28f0b864598a1291557bed248a998d4e-Paper.pdf.
  27. Diffskill: Skill abstraction from differentiable physics for deformable object manipulations with tools. 2022a.
  28. Planning with spatial-temporal abstraction from point clouds for deformable object manipulation. In 6th Annual Conference on Robot Learning, 2022b. URL https://openreview.net/forum?id=tyxyBj2w4vw.
  29. Learning without knowing: Unobserved context in continuous transfer reinforcement learning. In Learning for Dynamics and Control, pp.  791–802. PMLR, 2021.
  30. Revolver: Continuous evolutionary models for robot-to-robot policy transfer. In International Conference on Machine Learning, pp.  13995–14007. PMLR, 2022a.
  31. Herd: Continuous human-to-robot evolution for learning from human demonstration. In 6th Annual Conference on Robot Learning, 2022b.
  32. Sagci-system: Towards sample-efficient, generalizable, compositional, and incremental robot learning. In 2022 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2022.
  33. A two-stage trajectory optimization strategy for articulated bodies with unscheduled contact sequences. IEEE Robotics and Automation Letters, 2(1):104–111, 2017. doi: 10.1109/LRA.2016.2547024.
  34. Contact-invariant optimization for hand manipulation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’12, pp.  137–144. Eurographics Association, 2012. ISBN 9783905674378.
  35. Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models, 2022.
  36. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32:8026–8037, 2019.
  37. Scalable differentiable physics for learning and control. arXiv preprint arXiv:2007.02168, 2020.
  38. Efficient differentiable simulation of articulated bodies. In International Conference on Machine Learning, pp.  8661–8671. PMLR, 2021.
  39. Do differentiable simulators give better policy gradients? In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp.  20668–20696. PMLR, 17–23 Jul 2022.
  40. Differentiable fluids with solid coupling for learning and control. Proceedings of the AAAI Conference on Artificial Intelligence, 35(7):6138–6146, May 2021. doi: 10.1609/aaai.v35i7.16764. URL https://ojs.aaai.org/index.php/AAAI/article/view/16764.
  41. Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10(7), 2009.
  42. Theano: A python framework for fast computation of mathematical expressions. arXiv preprint arXiv:1605.02688, 2016.
  43. Transfer of value functions via variational methods. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/file/9023effe3c16b0477df9b93e26d57e2c-Paper.pdf.
  44. Grasp’d: Differentiable contact-rich grasp synthesis for multi-fingered hands. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VI, pp.  201–221. Springer, 2022.
  45. Solver-in-the-loop: Learning from differentiable physics to interact with iterative pde-solvers. Advances in Neural Information Processing Systems, 33:6111–6122, 2020.
  46. Learning incompressible fluid dynamics from scratch–towards fast, differentiable fluid models that generalize. arXiv preprint arXiv:2006.08762, 2020.
  47. Fast and feature-complete differentiable physics for articulated rigid bodies with contact. In Proceedings of Robotics: Science and Systems (RSS), July 2021.
  48. Cocoi: Contact-aware online context inference for generalizable non-planar pushing. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.  176–182. IEEE, 2021.
  49. Jade: A differentiable physics engine for articulated rigid bodies with intersection-free frictional contact. arXiv preprint arXiv:2309.04710, 2023.
  50. Diffclothai: Differentiable cloth simulation with intersection-free frictional contact and differentiable two-way coupling with articulated rigid bodies. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023.
  51. Learning insertion primitives with discrete-continuous hybrid action space for robotic assembly tasks. In 2022 International Conference on Robotics and Automation (ICRA), pp.  9881–9887. IEEE, 2022.
  52. Learning generalizable pivoting skills. arXiv preprint arXiv:2305.02554, 2023a.
  53. Efficient sim-to-real transfer of contact-rich manipulation skills with online admittance residual learning. In 7th Annual Conference on Robot Learning, 2023b.
  54. Offline meta-reinforcement learning for industrial insertion. In 2022 International Conference on Robotics and Automation (ICRA), pp.  6386–6393, 2022. doi: 10.1109/ICRA46639.2022.9812312.
  55. Diff-lfd: Contact-aware model-based learning from visual demonstration for robotic manipulation via differentiable physics-based simulation and rendering. In Conference on Robot Learning. PMLR, 2023a.
  56. Allowing safe contact in robotic goal-reaching: Planning and tracking in operational and null spaces. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp.  8120–8126, 2023b. doi: 10.1109/ICRA48891.2023.10160649.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now