Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Stable Motion Primitives via Imitation and Contrastive Learning (2302.10017v3)

Published 20 Feb 2023 in cs.RO

Abstract: Learning from humans allows non-experts to program robots with ease, lowering the resources required to build complex robotic solutions. Nevertheless, such data-driven approaches often lack the ability to provide guarantees regarding their learned behaviors, which is critical for avoiding failures and/or accidents. In this work, we focus on reaching/point-to-point motions, where robots must always reach their goal, independently of their initial state. This can be achieved by modeling motions as dynamical systems and ensuring that they are globally asymptotically stable. Hence, we introduce a novel Contrastive Learning loss for training Deep Neural Networks (DNN) that, when used together with an Imitation Learning loss, enforces the aforementioned stability in the learned motions. Differently from previous work, our method does not restrict the structure of its function approximator, enabling its use with arbitrary DNNs and allowing it to learn complex motions with high accuracy. We validate it using datasets and a real robot. In the former case, motions are 2 and 4 dimensional, modeled as first- and second-order dynamical systems. In the latter, motions are 3, 4, and 6 dimensional, of first and second order, and are used to control a 7DoF robot manipulator in its end effector space and joint space. More details regarding the real-world experiments are presented in: \url{https://youtu.be/OM-2edHBRfc}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, “Dynamical movement primitives: learning attractor models for motor behaviors,” Neural computation, vol. 25, no. 2, pp. 328–373, 2013.
  2. S. M. Khansari-Zadeh and A. Billard, “Learning stable nonlinear dynamical systems with gaussian mixture models,” IEEE Transactions on Robotics, vol. 27, no. 5, pp. 943–957, 2011.
  3. M. A. Rana, A. Li, D. Fox, B. Boots, F. Ramos, and N. Ratliff, “Euclideanizing flows: Diffeomorphic reduction for learning stable dynamical systems,” in 2nd Annual Conference on Learning for Dynamics and Control (L4DC), 2020.
  4. J. Kober, M. Glisson, and M. Mistry, “Playing catch and juggling with a humanoid robot,” in 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).   IEEE, 2012, pp. 875–881.
  5. V. Sindhwani, S. Tu, and M. Khansari, “Learning contracting vector fields for stable imitation learning,” arXiv preprint arXiv:1804.04878, 2018.
  6. N. Perrin and P. Schlehuber-Caissier, “Fast diffeomorphic matching to learn globally asymptotically stable nonlinear dynamical systems,” Systems & Control Letters, vol. 96, pp. 51–59, 2016.
  7. J. Urain, M. Ginesi, D. Tateo, and J. Peters, “ImitationFlow: Learning deep stable stochastic dynamic systems by normalizing flows,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 5231–5237.
  8. S. M. Khansari-Zadeh and A. Billard, “Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions,” Robotics and Autonomous Systems, vol. 62, no. 6, pp. 752–765, 2014.
  9. A. Lemme, K. Neumann, R. F. Reinhart, and J. J. Steil, “Neural learning of vector fields for encoding stable dynamical systems,” Neurocomputing, vol. 141, pp. 3–14, 2014.
  10. S. Chopra, R. Hadsell, and Y. LeCun, “Learning a similarity metric discriminatively, with application to face verification,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1.   IEEE, 2005, pp. 539–546.
  11. M. Kaya and H. Ş. Bilge, “Deep metric learning: A survey,” Symmetry, vol. 11, no. 9, p. 1066, 2019.
  12. G. Li, Z. Jin, M. Volpp, F. Otto, R. Lioutikov, and G. Neumann, “ProDMPs: A unified perspective on dynamic and probabilistic movement primitives,” arXiv preprint arXiv:2210.01531, 2022.
  13. H. B. Amor, G. Neumann, S. Kamthe, O. Kroemer, and J. Peters, “Interaction primitives for human-robot cooperation tasks,” in 2014 IEEE international conference on robotics and automation (ICRA).   IEEE, 2014, pp. 2831–2837.
  14. A. Pervez, Y. Mao, and D. Lee, “Learning deep movement primitives using convolutional neural networks,” in 2017 IEEE-RAS 17th international conference on humanoid robotics (Humanoids).   IEEE, 2017, pp. 191–197.
  15. B. Ridge, A. Gams, J. Morimoto, A. Ude et al., “Training of deep neural networks for the generation of dynamic movement primitives,” Neural Networks, vol. 127, pp. 121–131, 2020.
  16. S. Bahl, M. Mukadam, A. Gupta, and D. Pathak, “Neural dynamic policies for end-to-end sensorimotor learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 5058–5069, 2020.
  17. S. Calinon, A. Pistillo, and D. G. Caldwell, “Encoding the time and space constraints of a task in explicit-duration hidden markov model,” in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.   IEEE, 2011, pp. 3413–3418.
  18. H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard, “Recent advances in robot learning from demonstration,” Annual review of control, robotics, and autonomous systems, vol. 3, pp. 297–330, 2020.
  19. J. Duan, Y. Ou, J. Hu, Z. Wang, S. Jin, and C. Xu, “Fast and stable learning of dynamical systems based on extreme learning machine,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, no. 6, pp. 1175–1185, 2017.
  20. N. Figueroa and A. Billard, “A physically-consistent Bayesian non-parametric mixture model for dynamical system learning,” in Conference on Robot Learning.   PMLR, 2018, pp. 927–946.
  21. H. Ravichandar, I. Salehi, and A. Dani, “Learning partially contracting dynamical systems from demonstrations,” in Conference on Robot Learning.   PMLR, 2017, pp. 369–378.
  22. K. Neumann and J. J. Steil, “Learning robot motions with stable dynamical systems under diffeomorphic transformations,” Robotics and Autonomous Systems, vol. 70, pp. 1–15, 2015.
  23. H. Ravichandar and A. Dani, “Learning contracting nonlinear dynamics from human demonstration for robot motion planning,” in Dynamic Systems and Control Conference, vol. 57250.   American Society of Mechanical Engineers, 2015, p. V002T27A008.
  24. C. Blocher, M. Saveriano, and D. Lee, “Learning stable dynamical systems using contraction theory,” in 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).   IEEE, 2017, pp. 124–129.
  25. C. M. Bishop, “Pattern recognition,” Machine learning, vol. 128, no. 9, 2006.
  26. F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 815–823.
  27. J. K. Hunter, “Introduction to dynamical systems,” UCDavis Mathematics MAT A, vol. 207, p. 2011, 2011.
  28. J. Abramson, A. Ahuja, I. Barr, A. Brussee, F. Carnevale, M. Cassin, R. Chhaparia, S. Clark, B. Damoc, A. Dudzik et al., “Imitating interactive intelligence,” arXiv preprint arXiv:2012.05672, 2020.
  29. C. M. Legaard, T. Schranz, G. Schweiger, J. Drgoňa, B. Falay, C. Gomes, A. Iosifidis, M. Abkar, and P. G. Larsen, “Constructing neural network-based models for simulating dynamical systems,” arXiv preprint arXiv:2111.01495, 2021.
  30. T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, J. Peters et al., “An algorithmic perspective on imitation learning,” Foundations and Trends® in Robotics, vol. 7, no. 1-2, pp. 1–179, 2018.
  31. N. Lambert, K. Pister, and R. Calandra, “Investigating compounding prediction errors in learned dynamics models,” arXiv preprint arXiv:2203.09637, 2022.
  32. G. Bontempi, S. Ben Taieb, and Y.-A. L. Borgne, “Machine learning strategies for time series forecasting,” in European business intelligence summer school.   Springer, 2012, pp. 62–77.
  33. A. Venkatraman, M. Hebert, and J. A. Bagnell, “Improving multi-step prediction of learned time series models,” in Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
  34. P. J. Werbos, “Backpropagation through time: what it does and how to do it,” Proceedings of the IEEE, vol. 78, no. 10, pp. 1550–1560, 1990.
  35. J. Langford, R. Salakhutdinov, and T. Zhang, “Learning nonlinear dynamic models,” in Proceedings of the 26th Annual International Conference on Machine Learning, 2009, pp. 593–600.
  36. A. Graves, “Generating sequences with recurrent neural networks,” arXiv preprint arXiv:1308.0850, 2013.
  37. Y. Bengio, P. Simard, and P. Frasconi, “Learning long-term dependencies with gradient descent is difficult,” IEEE transactions on neural networks, vol. 5, no. 2, pp. 157–166, 1994.
  38. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  39. A. Veit, M. J. Wilber, and S. Belongie, “Residual networks behave like ensembles of relatively shallow networks,” Advances in neural information processing systems, vol. 29, 2016.
  40. M. Müller, “Dynamic time warping,” Information retrieval for music and motion, pp. 69–84, 2007.
  41. T. Eiter and H. Mannila, “Computing discrete Fréchet distance,” 1994.
  42. S. M. Khansari-Zadeh and A. Billard, “A dynamical system approach to realtime obstacle avoidance,” Autonomous Robots, vol. 32, no. 4, pp. 433–454, May 2012. [Online]. Available: https://doi.org/10.1007/s10514-012-9287-y
  43. M. Saveriano and D. Lee, “Distance based dynamical system modulation for reactive avoidance of moving obstacles,” in 2014 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2014, pp. 5618–5623.
  44. L. Huber, A. Billard, and J.-J. Slotine, “Avoidance of convex and concave obstacles with convergence ensured through contraction,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1462–1469, 2019.
  45. L. Huber, J.-J. Slotine, and A. Billard, “Avoiding dense and dynamic obstacles in enclosed spaces: Application to moving in crowds,” IEEE Transactions on Robotics, 2022.
  46. L. Van der Maaten and G. Hinton, “Visualizing data using t-SNE.” Journal of machine learning research, vol. 9, no. 11, 2008.
  47. A. Ude, B. Nemec, T. Petrić, and J. Morimoto, “Orientation in Cartesian space dynamic movement primitives,” in 2014 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2014, pp. 2997–3004.
  48. V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines,” in Icml, 2010.
  49. D. Hendrycks and K. Gimpel, “Gaussian error linear units (GELUs),” arXiv preprint arXiv:1606.08415, 2016.
  50. J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016.
  51. J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, “Algorithms for hyper-parameter optimization,” Advances in neural information processing systems, vol. 24, 2011.
  52. T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 2623–2631.
  53. Force Control.   Springer London, 2009.
  54. S. Kawamura, F. Miyazaki, and S. Arimoto, “Is a local linear pd feedback control law effective for trajectory tracking of robot motion?” in Proceedings. 1988 IEEE International Conference on Robotics and Automation.   IEEE, 1988, pp. 1335–1340.
  55. C. Della Santina, C. Duriez, and D. Rus, “Model based control of soft robots: A survey of the state of the art and open challenges,” arXiv preprint arXiv:2110.01358, 2021.
  56. P. Beeson and B. Ames, “TRAC-IK: An open-source library for improved solving of generic inverse kinematics,” in 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).   IEEE, 2015, pp. 928–935.
  57. P. Corke and J. Haviland, “Not your grandmother’s toolbox–the robotics toolbox reinvented for python,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 11 357–11 363.
  58. I. B. Love, “Levenberg-Marquardt algorithm in robotic controls,” Ph.D. dissertation, Ph. D. dissertation, University of Washington, 2020.
  59. M. Saveriano and D. Lee, “Point cloud based dynamical system modulation for reactive avoidance of convex and concave obstacles,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 5380–5387.
Citations (12)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now