Papers
Topics
Authors
Recent
Search
2000 character limit reached

Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges

Published 29 Nov 2023 in cs.RO and cs.LG | (2311.18044v3)

Abstract: Transfer learning is a conceptually-enticing paradigm in pursuit of truly intelligent embodied agents. The core concept -- reusing prior knowledge to learn in and from novel situations -- is successfully leveraged by humans to handle novel situations. In recent years, transfer learning has received renewed interest from the community from different perspectives, including imitation learning, domain adaptation, and transfer of experience from simulation to the real world, among others. In this paper, we unify the concept of transfer learning in robotics and provide the first taxonomy of its kind considering the key concepts of robot, task, and environment. Through a review of the promises and challenges in the field, we identify the need of transferring at different abstraction levels, the need of quantifying the transfer gap and the quality of transfer, as well as the dangers of negative transfer. Via this position paper, we hope to channel the effort of the community towards the most significant roadblocks to realize the full potential of transfer learning in robotics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (144)
  1. ACM Trans. on Graphics 39(4). 10.1145/3386569.3392462.
  2. Ada SE, Ugur E and Akin HL (2022) Generalization in transfer learning: robust control of robot locomotion. Robotica 40(11): 3811–3836. 10.1017/S0263574722000625.
  3. arXiv preprint 2204.01691 URL https://arxiv.org/abs/2204.01691.
  4. In: IEEE/RAS Intl. Conf. on Humanoid Robots (Humanoids). pp. 169–175. 10.1109/ICHR.2006.321380.
  5. Autodesk, INC (2019) Maya. URL https:/autodesk.com/maya.
  6. Barnett SM and Ceci SJ (2002) When and where do we apply what we learn? A taxonomy for far transfer. Psychological Bulletin 128(4): 612–637. 10.1037/0033-2909.128.4.612.
  7. In: IEEE Intl. Conf. on Robotics and Automation (ICRA). pp. 8477–8483. 10.1109/ICRA.2019.8794273.
  8. Machine learning 79: 151–175. 10.1007/s10994-009-5152-4.
  9. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 1371–1394. 10.1007/978-3-540-30301-5_60.
  10. Blitzer J, McDonald R and Pereira F (2006) Domain adaptation with structural correspondence learning. In: Proc. of the conference on empirical methods in natural language processing. pp. 120–128. URL https://aclanthology.org/W06-1615.
  11. arXiv preprint 2108.07258 URL https://arxiv.org/abs/2108.07258.
  12. Bonilla EV, Chai K and Williams C (2007) Multi-task gaussian process prediction. In: Neural Information Processing Systems (NeurIPS), volume 20. URL https://papers.nips.cc/paper_files/paper/2007/hash/66368270ffd51418ec58bd793f2d9b1b-Abstract.html.
  13. In: IEEE Intl. Conf. on Robotics and Automation (ICRA). pp. 4243–4250. 10.1109/ICRA.2018.8460875.
  14. Bouzit M (1996) Design, implementation and testing of a data glove with force feedback for virtual and real objects telemanipulation. PhD Thesis, PhD Thesis, Laboratoire de Robotique de Paris, University of Pierre Et Marie Curie. URL https://cir.nii.ac.jp/crid/1571980075620850688.
  15. arXiv preprint 2307.15818 URL https://arxiv.org/abs/2307.15818.
  16. In: Robotics: Science and Systems (R:SS). URL https://www.roboticsproceedings.org/rss19/p025.pdf.
  17. Calinon S (2018) Robot Learning with Task-Parameterized Generative Models. Cham: Springer International Publishing, pp. 111–126. 10.1007/978-3-319-60916-4_7.
  18. In: 2015 international conference on advanced robotics (ICAR). IEEE, pp. 510–517. 10.1109/ICAR.2015.7251504.
  19. Cangelosi A (2010) Grounding language in action and perception: From cognitive agents to humanoid robots. Physics of Life Reviews 7(2): 139–151. https://doi.org/10.1016/j.plrev.2010.02.001.
  20. In: Neural Information Processing Systems (NeurIPS), volume 33. pp. 9912–9924. URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/70feb62b69f16e0238f741fab228fec2-Abstract.html.
  21. arXiv preprint 1803.11175 URL https://arxiv.org/abs/1803.11175.
  22. In: Intl. Conf. on Machine Learning (ICML), Proceedings of Machine Learning Research, volume 119. pp. 1597–1607. URL https://proceedings.mlr.press/v119/chen20j.html.
  23. Chen T, Murali A and Gupta A (2018) Hardware conditioned policies for multi-robot transfer learning. In: Neural Information Processing Systems (NeurIPS), volume 31. URL https://proceedings.neurips.cc/paper_files/paper/2018/hash/b8cfbf77a3d250a4523ba67a65a7d031-Abstract.html.
  24. arXiv preprint 2305.18565 URL https://arxiv.org/abs/2305.18565.
  25. Collins J, Howard D and Leitner J (2019a) Quantifying the reality gap in robotic manipulation tasks. In: IEEE Intl. Conf. on Robotics and Automation (ICRA). pp. 6706–6712. 10.1109/ICRA.2019.8793591.
  26. Collins J, Howard D and Leitner J (2019b) Quantifying the reality gap in robotic manipulation tasks. In: IEEE Intl. Conf. on Robotics and Automation (ICRA). pp. 6706–6712. 10.1109/ICRA.2019.8793591.
  27. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 3213–3223. 10.1109/CVPR.2016.350.
  28. URL https://pybullet.org/.
  29. http://pybullet.org.
  30. In: IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS). pp. 191–198. 10.1109/IROS.2008.4651104.
  31. Dautenhahn K and Nehaniv CL (eds.) (2002) Imitation in Animals and Artifacts. Cambridge, MA, USA: MIT Press. URL https://mitpress.mit.edu/9780262527750/imitation-in-animals-and-artifacts/.
  32. Deleu T and Bengio Y (2018) The effects of negative adaptation in model-agnostic meta-learning. arXiv preprint 1812.02159 URL https://arxiv.org/abs/1812.02159.
  33. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 248–255. 10.1109/CVPR.2009.5206848.
  34. IEEE/ASME Trans. on Mechatronics 21(5): 2581–2594. 10.1109/TMECH.2015.2510165.
  35. arXiv preprint 2303.03378 URL https://arxiv.org/abs/2303.03378.
  36. Egli P and Hutter M (2022) A general approach for the automation of hydraulic excavator arms using reinforcement learning. IEEE Robotics and Automation Letters 7(2): 5679–5686. 10.1109/LRA.2022.3152865.
  37. Evgeniou T and Pontil M (2004) Regularized multi–task learning. In: Proc. of the ACM SIGKDD Intl Conf. on Knowledge Discovery and Data Mining. pp. 109–117. 10.1145/1014052.1014067.
  38. Fernández F, García J and Veloso M (2010) Probabilistic policy reuse for inter-task transfer learning. Robotics and Autonomous Systems 58(7): 866–871. https://doi.org/10.1016/j.robot.2010.03.007.
  39. Finn C, Abbeel P and Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Intl. Conf. on Machine Learning (ICML), Proceedings of Machine Learning Research, volume 70. pp. 1126–1135. URL https://proceedings.mlr.press/v70/finn17a.html.
  40. arXiv preprint 2203.10421 URL https://arxiv.org/abs/2203.10421.
  41. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 6840–6849. 10.1109/CVPR52729.2023.00661.
  42. Geng C, Huang SJ and Chen S (2021) Recent advances in open set recognition: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 43(10): 3614–3631. 10.1109/TPAMI.2020.2981604.
  43. PloS one 12(9): e0183784.
  44. In: IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS). pp. 1274–1280. 10.1109/IROS51168.2021.9636628.
  45. Gielniak MJ, Liu CK and Thomaz AL (2013) Generating human-like motion for robots. Intl. Journal of Robotics Research 32(11): 1275–1301. 10.1177/0278364913490533.
  46. Journal of Machine Learning Research 13(1): 723–773. URL https://jmlr.csail.mit.edu/papers/v13/gretton12a.html.
  47. In: Intl. Conf. on Learning Representations (ICLR). URL https://openreview.net/pdf?id=HJxeWnCcF7.
  48. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 9729–9738. 10.1109/CVPR42600.2020.00975.
  49. Heyes C (2001) Causes and consequences of imitation. Trends in cognitive sciences 5(6): 253–261. 10.1016/s1364-6613(00)01661-2.
  50. IEEE Transactions on Automation Science and Engineering 18(2): 398–400. 10.1109/TASE.2021.3064065.
  51. In: Neural Information Processing Systems (NeurIPS), volume 19. URL https://papers.nips.cc/paper_files/paper/2006/hash/a2186aa7c086b46ad4e8bf81e2a3a19b-Abstract.html.
  52. In: Intl. Conf. on Machine Learning (ICML), Proceedings of Machine Learning Research, volume 162. pp. 9118–9147. URL https://proceedings.mlr.press/v162/huang22a.html.
  53. Ijspeert A, Nakanishi J and Schaal S (2002) Movement imitation with nonlinear dynamical systems in humanoid robots. In: IEEE Intl. Conf. on Robotics and Automation (ICRA), volume 2. pp. 1398–1403 vol.2. 10.1109/ROBOT.2002.1014739.
  54. Neural Computation 25(2): 328–373. 10.1162/NECO_a_00393.
  55. Jaquier N, Rozo L and Calinon S (2020) Analysis and transfer of human movement manipulability in industry-like activities. In: IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS). pp. 11131–11138. 10.1109/IROS45743.2020.9341353.
  56. arXiv preprint 2210.01672 URL https://arxiv.org/abs/2210.01672.
  57. URL https://github.com/thuml/Transfer-Learning-Library.
  58. arXiv preprint 2210.03094 URL https://arxiv.org/abs/2210.03094.
  59. The International Journal of Robotics Research 31(8): 927–934. 10.1177/0278364912445831.
  60. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 14829–14838. 10.1109/CVPR52688.2022.01441.
  61. Khansari-Zadeh SM and Billard A (2011) Learning stable nonlinear dynamical systems with Gaussian mixture models. IEEE Trans. on Robotics 27(5): 943–957. 10.1109/TRO.2011.2159412.
  62. In: Proceedings of the first international conference on Autonomous agents. pp. 340–347. 10.1145/267658.267738.
  63. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 5210–5217. 10.1109/IROS47612.2022.9982127.
  64. In: 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids). pp. 668–675. 10.1109/HUMANOIDS.2016.7803346.
  65. Kremelberg D (2019) Embodiment as a necessary a priori of general intelligence. In: Hammer P, Agrawal P, Goertzel B and Iklé M (eds.) Artificial General Intelligence. Springer International Publishing, pp. 132–136. 10.1007/978-3-030-27005-6_13.
  66. Krizhevsky A (2009) Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto URL https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
  67. Robotics and Autonomous Systems 59(10): 740–757. 10.1016/j.robot.2011.05.009.
  68. Kullback S and Leibler RA (1951) On information and sufficiency. The Annals of Mathematical Statistics 22(1): 79–86. URL http://www.jstor.org/stable/2236703.
  69. In: IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS). pp. 1161–1168. 10.1109/IROS47612.2022.9981091.
  70. IEEE Trans. on Systems, Man, and Cybernetics, Part B (Cybernetics) 27(1): 95–104. 10.1109/3477.552188.
  71. Lawrence ND and Platt JC (2004) Learning to learn with the informative vector machine. In: Intl. Conf. on Machine Learning (ICML). URL https://icml.cc/Conferences/2004/proceedings/abstracts/178.htm.
  72. Science Robotics 5(47): eabc5986. 10.1126/scirobotics.abc5986.
  73. Expert Systems with Applications 39(15): 12220–12228. 10.1016/j.eswa.2012.04.054.
  74. Li T and Figueroa N (2023) Task generalization with stability guarantees via elastic dynamical system motion policies. In: Conference on Robot Learning (CoRL). PMLR. URL https://openreview.net/pdf?id=8scj3Y0RLq.
  75. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 15396–15406. 10.1109/CVPR52688.2022.01496.
  76. In: European Conference on Computer Vision (ECCV). Springer International Publishing, pp. 740–755. URL https://link.springer.com/chapter/10.1007/978-3-319-10602-1_48.
  77. IEEE Trans. on Robotics 39(1): 57–75. 10.1109/TRO.2022.3188163.
  78. In: IEEE Intl. Conf. on Robotics and Automation (ICRA). pp. 1118–1125. 10.1109/ICRA.2018.8462901.
  79. Robotics and Autonomous Systems 128: 103515. 10.1016/j.robot.2020.103515.
  80. In: Neural Information Processing Systems (NeurIPS), volume 30. URL https://papers.nips.cc/paper_files/paper/2017/hash/03e0704b5690a2dee1861dc3ad3316c9-Abstract.html.
  81. arXiv preprint 2209.08996 URL https://arxiv.org/abs/2209.08996.
  82. In: IEEE/RAS Intl. Conf. on Humanoid Robots (Humanoids). pp. 285–290. 10.1109/Humanoids53995.2022.10000095.
  83. In: Intl. Conf. on Machine Learning (ICML), Proceedings of Machine Learning Research, volume 139. pp. 7090–7101. URL https://proceedings.mlr.press/v139/lopez21a.html.
  84. IEEE Robotics and Automation Letters 1(2): 784–791. 10.1109/LRA.2016.2525038.
  85. Science Robotics 4(26): eaau4984. 10.1126/scirobotics.aau4984.
  86. URL https://openreview.net/pdf?id=NJtSbIWmt2T.
  87. In: Robotics: Science and Systems (R:SS). URL https://www.roboticsproceedings.org/rss16/p061.pdf.
  88. arXiv preprint 2301.04195 URL https://arxiv.org/abs/2301.04195.
  89. Montanaro A, Valsesia D and Magli E (2022) Rethinking the compositionality of point clouds through regularization in the hyperbolic space. In: Neural Information Processing Systems (NeurIPS), volume 35. URL https://papers.nips.cc/paper_files/paper/2022/hash/da8f9fc2b555d122369f36a9684415c1-Abstract-Conference.html.
  90. IEEE Transactions on Robotics 24(1): 15–26. 10.1109/TRO.2007.914848.
  91. Frontiers in Robotics and AI 9. 10.3389/frobt.2022.799893.
  92. In: Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, volume 164. pp. 1303–1315. URL https://proceedings.mlr.press/v164/nair22a.html.
  93. arXiv preprint 2205.03532 URL https://arxiv.org/abs/2205.03532.
  94. Nickel M and Kiela D (2017) Poincaré Embeddings for Learning Hierarchical Representations. In: Neural Information Processing Systems (NeurIPS), volume 30. URL https://papers.nips.cc/paper_files/paper/2017/hash/59dfa2df42d9e3d41f5b02bfc32229dd-Abstract.html.
  95. Nvidia (2023) Universal scene description. URL https://developer.nvidia.com/usd.
  96. arXiv preprint 2310.08864 URL https://arxiv.org/abs/2310.08864.
  97. In: Proceedings of the Intl Conf. on Worldwide Web. pp. 751–760. 10.1145/1772690.1772767.
  98. Pan SJ and Yang Q (2010) A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22(10): 1345–1359. 10.1109/TKDE.2009.191.
  99. Advances in Neural Information Processing Systems 26. 10.5555/2999792.2999904.
  100. Robotics: Science and Systems (R:SS) URL https://www.roboticsproceedings.org/rss18/p010.pdf.
  101. In: Intl. Conf. on Machine Learning (ICML), volume 162. pp. 17359–17371. URL https://proceedings.mlr.press/v162/parisi22a.html.
  102. Peer A, Einenkel S and Buss M (2008) Multi-fingered telemanipulation-mapping of a human hand to a three finger gripper. In: IEEE Intl. Symposium on Robot and Human Interactive Communication (RO-MAN). pp. 465–470. 10.1109/ROMAN.2008.4600710.
  103. In: IEEE/RAS Intl. Conf. on Humanoid Robots (Humanoids). pp. 425–432. 10.1109/HUMANOIDS.2018.8624943.
  104. In: AAAI Conf. on Artificial Intelligence, volume 32. p. 3942–3951.
  105. Perkins DN and Salomon G (1992) Transfer of learning. In: Husén T and Postlethwaite TN (eds.) The International Encyclopedia of Education. pp. 47–79.
  106. Rakita D, Mutlu B and Gleicher M (2017) A motion retargeting method for effective mimicry-based teleoperation of robot arms. In: ACM/IEEE Intl. Conf. on Human-Robot Interaction (HRI). pp. 361–370. URL https://ieeexplore.ieee.org/document/8534763.
  107. Annual Review of Control, Robotics, and Autonomous Systems 3(1): 297–330. 10.1146/annurev-control-100819-063206.
  108. Reader SM, Morand-Ferron J and Flynn E (2016) Animal and human innovation: novel problems and novel solutions. Philosophical Transactions of the Royal Society B: Biological Sciences 371(1690): 20150182. 10.1098/rstb.2015.0182.
  109. In: Conference on Robot Learning (CoRL). PMLR, pp. 1531–1541. URL https://openreview.net/pdf?id=nPw7jaGBrCG.
  110. In: NIPS Workshop on Transfer Learning, volume 898. URL http://people.csail.mit.edu/mtr/papers/RosensteinM05c.pdf.
  111. IEEE Trans. on Robotics 32(3): 513–527. 10.1109/TRO.2016.2540623.
  112. Neural Information Processing Systems (NeurIPS) 34: 12786–12797. URL https://proceedings.neurips.cc/paper/2021/hash/6a30e32e56fce5cf381895dfe6ca7b6f-Abstract.html.
  113. In: European Conference on Computer Vision (ECCV). Springer, pp. 213–226. 10.1007/978-3-642-15561-1_16.
  114. Schaal S (1999) Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences 3(6): 233–242. https://doi.org/10.1016/S1364-6613(99)01327-3.
  115. Schmidt RA and Young DE (1987) Transfer of movement control in motor skill learning. In: Transfer of Learning: Contemporary Research and Applications. pp. 47–79. 10.1016/b978-0-12-188950-0.50009-6.
  116. In: Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, volume 205. pp. 44–54. URL https://proceedings.mlr.press/v205/shah23a.html.
  117. Shah R and Kumar V (2021) RRL: Resnet as representation for reinforcement learning. In: Intl. Conf. on Machine Learning (ICML), Proceedings of Machine Learning Research, volume 139. pp. 9465–9476. URL https://proceedings.mlr.press/v139/shah21a.html.
  118. SideFX (2022) Sidefx. URL https://www.sidefx.com/.
  119. arXiv preprint 2201.09565 URL https://arxiv.org/abs/2201.09565.
  120. Sorocky MJ, Zhou S and Schoellig AP (2020) Experience selection using dynamics similarity for efficient multi-source transfer learning between robots. In: IEEE Intl. Conf. on Robotics and Automation (ICRA). pp. 2739–2745. 10.1109/ICRA40945.2020.9196744.
  121. Sorocky MJ, Zhou S and Schoellig AP (2021) To share or not to share? Performance guarantees and the asymmetric nature of cross-robot experience transfer. IEEE Control Systems Letters 5(3): 923–928. 10.1109/lcsys.2020.3005886.
  122. In: Conference on Robot Learning (CoRL). URL https://openreview.net/pdf?id=9al6taqfTzr.
  123. Studio PU (2023) Universal scene description. URL https://github.com/PixarAnimationStudios/OpenUSD.
  124. Tan M and Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In: Intl. Conf. on Machine Learning (ICML), Proceedings of Machine Learning Research, volume 97. pp. 6105–6114. URL https://proceedings.mlr.press/v97/tan19a.html.
  125. Thrun S and Pratt L (1998) Learning to learn: Introduction and overview. Springer, pp. 3–17. 10.1007/978-1-4615-5529-2_1.
  126. Todorov E, Erez T and Tassa Y (2012) Mujoco: A physics engine for model-based control. In: IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS). pp. 5026–5033. 10.1109/IROS.2012.6386109.
  127. IEEE Transactions on Robotics 26(5): 800–815. 10.1109/TRO.2010.2065430.
  128. In: Neural Information Processing Systems (NeurIPS), volume 30. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
  129. Vosylius V and Johns E (2023) Where to start? Transferring simple skills to complex environments. In: Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, volume 205. pp. 471–481. URL https://proceedings.mlr.press/v205/vosylius23a.html.
  130. Walqui A (2000) Contextual factors in second language acquisition. ERIC Digest URL https://files.eric.ed.gov/fulltext/ED444381.pdf.
  131. Wang SJ and Johnson AM (2021) Domain adaptation using system invariant dynamics models. In: Conference on Learning for Dynamics and Control (L4DC), Proceedings of Machine Learning Research, volume 144. pp. 1130–1141. URL https://proceedings.mlr.press/v144/wang21c.html.
  132. Wang Z (2021) Mitigating negative transfer for better generalization and efficiency in transfer learning. PhD Thesis, Carnegie Mellon University. URL https://www.lti.cs.cmu.edu/sites/default/files/wang%2C%20zirui%20-%20final%20thesis.pdf.
  133. Whiten A and Ham R (1992) On the nature and evolution of imitation in the animal kingdom: Reappraisal of a century of research. Academic Press, pp. 239–283. https://doi.org/10.1016/S0065-3454(08)60146-1.
  134. Animal Learning & Behavior 32: 36–52. 10.3758/BF03196005.
  135. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 3733–3742. 10.1109/CVPR.2018.00393.
  136. Cambridge University Press. 10.1017/9781139061773.
  137. In: IEEE Intl. Conf. on Robotics and Automation (ICRA). pp. 7286–7293. 10.1109/ICRA40945.2020.9197331.
  138. Machine Vision and Applications 34(4): 48. 10.1007/s00138-023-01399-x.
  139. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 2715–2724. 10.1109/CVPR.2019.00283.
  140. In: Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, volume 164. pp. 537–546. URL https://proceedings.mlr.press/v164/zakka22a.html.
  141. Zhang LM, Plappert M and Zaremba W (2020) Predicting sim-to-real transfer with probabilistic dynamics models. arXiv preprint 2009.12864 URL https://arxiv.org/abs/2009.12864.
  142. Zhao W, Queralta JP and Westerlund T (2020) Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In: IEEE Symposium Series on Computational Intelligence. pp. 737–744. 10.1109/SSCI47803.2020.9308468.
  143. Zhou Y, Gao J and Asfour T (2020) Movement primitive learning and generalization: Using mixture density networks. IEEE Robotics & Automation Magazine 27(2): 22–32. 10.1109/MRA.2020.2980591.
  144. Proceedings of the IEEE 109(1): 43–76. 10.1109/JPROC.2020.3004555.
Citations (10)
View on Semantic Scholar

Summary

  • The paper demonstrates how transfer learning enables robots to quickly adapt to new tasks by applying pre-learned knowledge.
  • The paper addresses methodological challenges in aligning digital models with physical robotic actions, highlighting risks of negative transfer.
  • The paper explores promising techniques, including grounding semantic models in sensorimotor data and using Robotic Transformers for multimodal integration.

Transfer learning has long been a central concept in the development of intelligent systems, as evidenced by its success in domains like computer vision and natural language processing. The fundamental appeal of transfer learning is its promise to enable machines—particularly robots—to rapidly adapt to new tasks by exploiting previously acquired knowledge instead of starting from scratch. This approach not only saves time and computational resources but also poses the tantalizing possibility that robots could attain a level of adaptability similar to that of humans, who are exceptionally skilled at applying past experiences to new challenges.

The practical application of transfer learning within robotics, however, is fraught with unique complexities. A robot's interaction with the physical world introduces variables that are not present in purely digital environments. As such, questions arise concerning when to pursue transfer learning, the specifics of the information being transferred, and the method by which the transfer occurs. Addressing these questions calls for a nuanced understanding of the interplay between the robot's physical embodiment, tasks, and the environments in which those tasks take place.

One of the key hurdles is successfully identifying commonalities between the source knowledge and the new application context—a process crucial for enabling positive transfer and preventing "negative transfer," where improper application of knowledge can degrade a robot's performance. The stakes for robotics are significant: negative transfer can result in inefficient behavior at best and unsafe interactions at worst.

Several interesting directions are being explored to navigate these challenges. One approach involves grounding high-level semantic knowledge from large pre-trained models (like those used in AI) in the robot’s sensorimotor experience, thus drawing on a rich base of general knowledge that can be fine-tuned for specific robotic tasks. Another promising field is the pursuit of universal representations—data-derived frameworks that robots can leverage across different contexts without the need for model re-training.

Robotic Transformers, adaptations of models originally used in machine learning for handling large-scale, unsupervised data, also show potential. These models are fed multimodal inputs like images and text instructions and produce robot actions. While they achieve impressive feats of integration by fusing multiple levels of a robot's capabilities, they can be cumbersome and less performance-efficient than traditional methods catering to a particular action level.

Lastly, evaluation is a topic of heightened focus: quantifiable metrics are needed to measure the success of a transfer and to understand the transfer gaps in robot performance across varying tasks and environments. Progress on standardized benchmarks and simulations will help quantify these factors and drive future innovations in the field.

Transfer learning in robotics, despite its complexities, holds tremendous potential. It stands at the intersection of theoretical advancement and practical utility, promising to catalyze a leap forward in robotic capabilities. The roadblocks identified are stimulating extensive ongoing research that could eventually equip robots with the versatility and resourcefulness that are trademarks of human intelligence.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.

Sign Up for Free