Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Learning for Koopman-based Dynamic Movement Primitives (2312.03328v1)

Published 6 Dec 2023 in cs.RO and cs.LG

Abstract: The challenge of teaching robots to perform dexterous manipulation, dynamic locomotion, or whole--body manipulation from a small number of demonstrations is an important research field that has attracted interest from across the robotics community. In this work, we propose a novel approach by joining the theories of Koopman Operators and Dynamic Movement Primitives to Learning from Demonstration. Our approach, named \gls{admd}, projects nonlinear dynamical systems into linear latent spaces such that a solution reproduces the desired complex motion. Use of an autoencoder in our approach enables generalizability and scalability, while the constraint to a linear system attains interpretability. Our results are comparable to the Extended Dynamic Mode Decomposition on the LASA Handwriting dataset but with training on only a small fractions of the letters.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (20)
  1. A. Y. Ng, D. Harada, and S. Russell, “Policy invariance under reward transformations: Theory and application to reward shaping,” in Icml, vol. 99, pp. 278–287, 1999.
  2. A. D. Laud, Theory and application of reward shaping in reinforcement learning. University of Illinois at Urbana-Champaign, 2004.
  3. D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Mané, “Concrete problems in ai safety,” arXiv preprint arXiv:1606.06565, 2016.
  4. S. Schaal et al., “Learning from demonstration,” Advances in neural information processing systems, pp. 1040–1046, 1997.
  5. S. Schaal, J. Peters, J. Nakanishi, and A. Ijspeert, “Learning movement primitives,” in Robotics research. the eleventh international symposium, pp. 561–572, Springer, 2005.
  6. P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal, “Learning and generalization of motor skills by learning from demonstration,” in 2009 IEEE International Conference on Robotics and Automation, pp. 763–768, IEEE, 2009.
  7. J. Kober, A. Wilhelm, E. Oztop, and J. Peters, “Reinforcement learning to adjust parametrized motor primitives to new situations,” Autonomous Robots, vol. 33, no. 4, pp. 361–379, 2012.
  8. A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, “Dynamical movement primitives: learning attractor models for motor behaviors,” Neural computation, vol. 25, no. 2, pp. 328–373, 2013.
  9. T. Matsubara, S.-H. Hyon, and J. Morimoto, “Learning parametric dynamic movement primitives from multiple demonstrations,” Neural networks, vol. 24, no. 5, pp. 493–500, 2011.
  10. T. Davchev, K. S. Luck, M. Burke, F. Meier, S. Schaal, and S. Ramamoorthy, “Residual learning from demonstration: adapting dynamic movement primitives for contact-rich insertion tasks,” arXiv preprint arXiv:2008.07682, 2020.
  11. M. Saveriano, F. J. Abu-Dakka, A. Kramberger, and L. Peternel, “Dynamic movement primitives in robotics: A tutorial survey,” ArXiv, vol. abs/2102.03861, 2021.
  12. SIAM, 2016.
  13. S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proceedings of the National Academy of Sciences of the United States of America, 2016.
  14. B. Lusch, J. N. Kutz, and S. L. Brunton, “Deep learning for universal linear embeddings of nonlinear dynamics,” Nature communications, vol. 9, no. 1, pp. 1–10, 2018.
  15. Y. Lian and C. N. Jones, “Learning feature maps of the koopman operator: A subspace viewpoint,” in 2019 IEEE 58th Conference on Decision and Control (CDC), pp. 860–866, IEEE, 2019.
  16. S. L. Brunton, B. W. Brunton, J. L. Proctor, E. J. Kaiser, and N. Kutz, “Chaos as an intermittently forced linear system,” arXiv preprint arXiv:1608.05306v1, 2016.
  17. S. M. Khansari-Zadeh and A. Billard, “Learning stable nonlinear dynamical systems with gaussian mixture models,” IEEE Transactions on Robotics, vol. 27, no. 5, pp. 943–957, 2011.
  18. R. Saegusa, G. Metta, G. Sandini, and S. Sakka, “Active motor babbling for sensorimotor learning,” in 2008 IEEE International Conference on Robotics and Biomimetics, pp. 794–799, IEEE, 2009.
  19. I. G. K. Matthew O. Williams, Clarence W. Rowley, “A kernel-based method for data-driven koopman spectral analysis,” Journal of Computational Dynamics, vol. 2, no. 2, pp. 247–265, 2015.
  20. M. Innes, E. Saba, K. Fischer, D. Gandhi, M. C. Rudilosso, N. M. Joy, T. Karmali, A. Pal, and V. Shah, “Fashionable modelling with flux,” CoRR, vol. abs/1811.01457, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Tyler Han (5 papers)
  2. Carl Glen Henshaw (1 paper)

Summary

Overview

The paper presents a novel approach to robot learning using a combination of two theoretical frameworks: Koopman Operators and Dynamic Movement Primitives (DMPs). This fusion leads to the development of an Autoencoder Dynamic Mode Decomposition (aDMD), which aims to enable robots to learn complex movements from a limited set of demonstrations. The aDMD method projects nonlinear dynamical systems into a linear latent space, maintaining generalizability and scalability while retaining interpretability due to the linear nature of this space.

Learning from Demonstration (LfD)

Learning from Demonstration is a method of robot learning where an expert demonstrates a task to the robot, which then learns to mimic or replicate the task. Traditional methods require many demonstrations and work under constrained conditions, limiting their usefulness in unpredictable environments like disaster responses or domestic services. An alternative method uses DMPs, modeling the robot's motion as a dynamic system, which has proved successful but typically needs detailed hand-designing of system dynamics.

Advancements in Koopman Theory

Recent studies in dynamical systems have focused on the Koopman operators, which offer a linear perspective to the evolution of system states, contrasting with typical nonlinear differential equations. This theory inherently deals with an infinite number of observables, which are functions representing system states. Deep learning frameworks have become an intriguing possibility for approximating these observables and, correspondingly, the latent space representations.

The Proposed aDMD Framework

The proposed aDMD method integrates an autoencoder for discovering observable functions over multiple trajectories, aiming to identify a Koopman operator that can be used to extend and generalize to similar movements not explicitly trained. This is validated using the LASA Handwriting dataset, showing successful reconstruction of the motion with minimal training examples per trajectory. The memorability of aDMD allows for both the compression and simplification of complex dynamics into a linear, tractable system—a promising step for handling high-dimensional systems in robotics.

Reflection and Future Perspectives

The fusion of DMP and Koopman Theory through aDMD has shown potential for allowing robotic systems to learn from a single example per trajectory, projecting tasks into a linear latent space for ease of interpretation and manipulation. Future research aims to apply this framework to more complicated movements such as humanoid locomotion and to explore the integration of linear control theory within the latent space. The goal is to enhance the robot's ability to manage disruptions and adapt fluidly to new conditions in its operating environment.

By introducing a scalable and generalizable approach like aDMD, the field of robotic learning stands on the cusp of a significant breakthrough, where the dream of robots learning from limited demonstrations in unstructured environments moves closer to reality.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown