Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PUMA: Deep Metric Imitation Learning for Stable Motion Primitives (2310.12831v3)

Published 19 Oct 2023 in cs.RO

Abstract: Imitation Learning (IL) is a powerful technique for intuitive robotic programming. However, ensuring the reliability of learned behaviors remains a challenge. In the context of reaching motions, a robot should consistently reach its goal, regardless of its initial conditions. To meet this requirement, IL methods often employ specialized function approximators that guarantee this property by construction. Although effective, these approaches come with a set of limitations: 1) they are unable to fully exploit the capabilities of modern Deep Neural Network (DNN) architectures, 2) some are restricted in the family of motions they can model, resulting in suboptimal IL capabilities, and 3) they require explicit extensions to account for the geometry of motions that consider orientations. To address these challenges, we introduce a novel stability loss function, drawing inspiration from the triplet loss used in the deep metric learning literature. This loss does not constrain the DNN's architecture and enables learning policies that yield accurate results. Furthermore, it is not restricted to a specific state space geometry; therefore, it can easily incorporate the geometry of the robot's state space. We provide a proof of the stability properties induced by this loss and empirically validate our method in various settings. These settings include Euclidean and non-Euclidean state spaces, as well as first-order and second-order motions, both in simulation and with real robots. More details about the experimental results can be found in: https://youtu.be/ZWKLGntCI6w.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. N. Perrin and P. Schlehuber-Caissier, “Fast diffeomorphic matching to learn globally asymptotically stable nonlinear dynamical systems,” Systems & Control Letters, vol. 96, pp. 51–59, 2016.
  2. M. A. Rana, A. Li, D. Fox, B. Boots, F. Ramos, and N. Ratliff, “Euclideanizing flows: Diffeomorphic reduction for learning stable dynamical systems,” in 2nd Annual Conference on Learning for Dynamics and Control (L4DC), 2020.
  3. J. Urain, M. Ginesi, D. Tateo, and J. Peters, “ImitationFlow: Learning deep stable stochastic dynamic systems by normalizing flows,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 5231–5237.
  4. S. M. Khansari-Zadeh and A. Billard, “Learning stable nonlinear dynamical systems with gaussian mixture models,” IEEE Transactions on Robotics, vol. 27, no. 5, pp. 943–957, 2011.
  5. ——, “Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions,” Robotics and Autonomous Systems, vol. 62, no. 6, pp. 752–765, 2014.
  6. A. Lemme, K. Neumann, R. F. Reinhart, and J. J. Steil, “Neural learning of vector fields for encoding stable dynamical systems,” Neurocomputing, vol. 141, pp. 3–14, 2014.
  7. N. Figueroa and A. Billard, “Locally active globally stable dynamical systems: Theory, learning, and experiments,” The International Journal of Robotics Research, p. 02783649211030952, 2022.
  8. A. Ude, B. Nemec, T. Petrić, and J. Morimoto, “Orientation in cartesian space dynamic movement primitives,” in 2014 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2014, pp. 2997–3004.
  9. R. Pérez-Dattari and J. Kober, “Stable motion primitives via imitation and contrastive learning,” IEEE Transactions on Robotics, pp. 1–20, 2023.
  10. J. Zhang, H. B. Mohammadi, and L. Rozo, “Learning riemannian stable dynamical systems via diffeomorphisms,” in Conference on Robot Learning, 2022.
  11. J. Urain, D. Tateo, and J. Peters, “Learning stable vector fields on lie groups,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 12 569–12 576, 2022.
  12. M. Saveriano, F. J. Abu-Dakka, and V. Kyrki, “Learning stable robotic skills on riemannian manifolds,” Robotics and Autonomous Systems, p. 104510, 2023.
  13. F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 815–823.
  14. M. Kaya and H. Ş. Bilge, “Deep metric learning: A survey,” Symmetry, vol. 11, no. 9, p. 1066, 2019.
  15. S. Calinon, A. Pistillo, and D. G. Caldwell, “Encoding the time and space constraints of a task in explicit-duration hidden markov model,” in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.   IEEE, 2011, pp. 3413–3418.
  16. H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard, “Recent advances in robot learning from demonstration,” Annual review of control, robotics, and autonomous systems, vol. 3, pp. 297–330, 2020.
  17. A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, “Dynamical movement primitives: learning attractor models for motor behaviors,” Neural computation, vol. 25, no. 2, pp. 328–373, 2013.
  18. G. Li, Z. Jin, M. Volpp, F. Otto, R. Lioutikov, and G. Neumann, “Prodmp: A unified perspective on dynamic and probabilistic movement primitives,” IEEE Robotics and Automation Letters, vol. 8, no. 4, pp. 2325–2332, 2023.
  19. H. B. Amor, G. Neumann, S. Kamthe, O. Kroemer, and J. Peters, “Interaction primitives for human-robot cooperation tasks,” in 2014 IEEE international conference on robotics and automation (ICRA).   IEEE, 2014, pp. 2831–2837.
  20. A. Pervez, Y. Mao, and D. Lee, “Learning deep movement primitives using convolutional neural networks,” in 2017 IEEE-RAS 17th international conference on humanoid robotics (Humanoids).   IEEE, 2017, pp. 191–197.
  21. R. Pahič, B. Ridge, A. Gams, J. Morimoto, and A. Ude, “Training of deep neural networks for the generation of dynamic movement primitives,” Neural Networks, vol. 127, pp. 121–131, 2020.
  22. S. Bahl, M. Mukadam, A. Gupta, and D. Pathak, “Neural dynamic policies for end-to-end sensorimotor learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 5058–5069, 2020.
  23. N. Figueroa and A. Billard, “A physically-consistent Bayesian non-parametric mixture model for dynamical system learning,” in Conference on Robot Learning.   PMLR, 2018, pp. 927–946.
  24. H. Ravichandar, I. Salehi, and A. Dani, “Learning partially contracting dynamical systems from demonstrations,” in Conference on Robot Learning.   PMLR, 2017, pp. 369–378.
  25. K. Neumann and J. J. Steil, “Learning robot motions with stable dynamical systems under diffeomorphic transformations,” Robotics and Autonomous Systems, vol. 70, pp. 1–15, 2015.
  26. J. Duan, Y. Ou, J. Hu, Z. Wang, S. Jin, and C. Xu, “Fast and stable learning of dynamical systems based on extreme learning machine,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, no. 6, pp. 1175–1185, 2017.
  27. H. Ravichandar and A. Dani, “Learning contracting nonlinear dynamics from human demonstration for robot motion planning,” in Dynamic Systems and Control Conference, vol. 57250.   American Society of Mechanical Engineers, 2015, p. V002T27A008.
  28. C. Blocher, M. Saveriano, and D. Lee, “Learning stable dynamical systems using contraction theory,” in 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).   IEEE, 2017, pp. 124–129.
  29. W. Zhi, T. Lai, L. Ott, and F. Ramos, “Diffeomorphic transforms for generalised imitation learning.” in L4DC, 2022, pp. 508–519.
  30. P. Pastor, L. Righetti, M. Kalakrishnan, and S. Schaal, “Online movement adaptation based on previous sensor experiences,” in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.   IEEE, 2011, pp. 365–371.
  31. L. Koutras and Z. Doulgeri, “A correct formulation for the orientation dynamic movement primitives for robot control in the cartesian space,” in Conference on robot learning.   PMLR, 2020, pp. 293–302.
  32. M. Lang and S. Hirche, “Computationally efficient rigid-body gaussian process for motion dynamics,” IEEE Robotics and Automation Letters, vol. 2, no. 3, pp. 1601–1608, 2017.
  33. M. Arduengo, A. Colomé, J. Lobo-Prat, L. Sentis, and C. Torras, “Gaussian-process-based robot learning from demonstration,” Journal of Ambient Intelligence and Humanized Computing, pp. 1–14, 2023.
  34. J. Silvério, L. Rozo, S. Calinon, and D. G. Caldwell, “Learning bimanual end-effector poses from demonstrations using task-parameterized dynamical systems,” in 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS).   IEEE, 2015, pp. 464–470.
  35. I. Havoutis and S. Calinon, “Learning from demonstration for semi-autonomous teleoperation,” Autonomous Robots, vol. 43, pp. 713–726, 2019.
  36. M. J. Zeestraten, I. Havoutis, J. Silvério, S. Calinon, and D. G. Caldwell, “An approach for imitation learning on riemannian manifolds,” IEEE Robotics and Automation Letters, vol. 2, no. 3, pp. 1240–1247, 2017.
  37. S. Kim, R. Haschke, and H. Ritter, “Gaussian mixture model for 3-dof orientations,” Robotics and Autonomous Systems, vol. 87, pp. 28–37, 2017.
  38. Y. Huang, F. J. Abu-Dakka, J. Silvério, and D. G. Caldwell, “Toward orientation learning and adaptation in cartesian space,” IEEE Transactions on Robotics, vol. 37, no. 1, pp. 82–98, 2020.
  39. F. J. Abu-Dakka, Y. Huang, J. Silvério, and V. Kyrki, “A probabilistic framework for learning geometry-based robot manipulation skills,” Robotics and Autonomous Systems, vol. 141, p. 103761, 2021.
  40. H. C. Ravichandar and A. Dani, “Learning position and orientation dynamics from demonstrations via contraction analysis,” Autonomous Robots, vol. 43, no. 4, pp. 897–912, 2019.
  41. C. M. Kellett, “A compendium of comparison function results,” Mathematics of Control, Signals, and Systems, vol. 26, pp. 339–374, 2014.
  42. S. Sommer, T. Fletcher, and X. Pennec, “Introduction to differential and riemannian geometry,” in Riemannian Geometric Statistics in Medical Image Analysis.   Elsevier, 2020, pp. 3–37.
  43. M. Müller, “Dynamic time warping,” Information retrieval for music and motion, pp. 69–84, 2007.
  44. T. Eiter and H. Mannila, “Computing discrete Fréchet distance,” Technische Universitat Wien, Tech. Rep. CD-TR 94/64, 1994.
  45. P. Corke and J. Haviland, “Not your grandmother’s toolbox–the robotics toolbox reinvented for python,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 11 357–11 363.
  46. J. Jeong, M. Jun, and M. G. Genton, “Spherical process models for global spatial statistics,” Statistical science: a review journal of the Institute of Mathematical Statistics, vol. 32, no. 4, p. 501, 2017.
  47. J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, “Algorithms for hyper-parameter optimization,” Advances in neural information processing systems, vol. 24, 2011.
  48. T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 2623–2631.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now