Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Teach Me How to Learn: A Perspective Review towards User-centered Neuro-symbolic Learning for Robotic Surgical Systems (2307.03853v1)

Published 7 Jul 2023 in cs.RO, cs.AI, and cs.HC

Abstract: Recent advances in machine learning models allowed robots to identify objects on a perceptual nonsymbolic level (e.g., through sensor fusion and natural language understanding). However, these primarily black-box learning models still lack interpretation and transferability and require high data and computational demand. An alternative solution is to teach a robot on both perceptual nonsymbolic and conceptual symbolic levels through hybrid neurosymbolic learning approaches with expert feedback (i.e., human-in-the-loop learning). This work proposes a concept for this user-centered hybrid learning paradigm that focuses on robotic surgical situations. While most recent research focused on hybrid learning for non-robotic and some generic robotic domains, little work focuses on surgical robotics. We survey this related research while focusing on human-in-the-loop surgical robotic systems. This evaluation highlights the most prominent solutions for autonomous surgical robots and the challenges surgeons face when interacting with these systems. Finally, we envision possible ways to address these challenges using online apprenticeship learning based on implicit and explicit feedback from expert surgeons.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (121)
  1. Pieter Abbeel and Andrew Y. Ng. 2004. Apprenticeship Learning via Inverse Reinforcement Learning. In Proceedings of the Twenty-First International Conference on Machine Learning (Banff, Alberta, Canada) (ICML ’04). Association for Computing Machinery, New York, NY, USA, 1. https://doi.org/10.1145/1015330.1015430
  2. Toward a Versatile Robotic Platform for Fluoroscopy and MRI-Guided Endovascular Interventions: A Pre-Clinical Study. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 5411–5418. https://doi.org/10.1109/IROS40897.2019.8968237
  3. Heike Adel. 2018. Deep learning methods for knowledge base population. Ph. D. Dissertation. LMU.
  4. The potentials for hands-free interaction in micro-neurosurgery. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational. 401–410.
  5. Hanad Ahmed and Laurence Devoto. 2021. The potential of a digital twin in surgery. Surgical Innovation 28, 4 (2021), 509–510.
  6. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35, 4 (December 2014), 105–120. https://www.microsoft.com/en-us/research/publication/power-to-the-people-the-role-of-humans-in-interactive-machine-learning/
  7. Christian Arzate Cruz and Takeo Igarashi. 2020. A survey on interactive reinforcement learning: Design principles and open challenges. In Proceedings of the 2020 ACM designing interactive systems conference. 1195–1209.
  8. Masataro Asai and Alex Fukunaga. 2018. Classical planning in deep latent space: Bridging the subsymbolic-symbolic boundary. In Proceedings of the Conference on Artificial Intelligence (AAAI’18). AAAI Press, 6094–6101.
  9. Autonomy in surgical robotics. Annual Review of Control, Robotics, and Autonomous Systems 4 (2021), 651–679.
  10. Samy Badreddine and Michael Spranger. 2019. Injecting prior knowledge for transfer learning into reinforcement learning algorithms using logic tensor networks. arXiv preprint arXiv:1906.06576 (2019).
  11. Path planning for automation of surgery robot based on probabilistic roadmap and reinforcement learning. In 2018 15th International Conference on Ubiquitous Robots (UR). IEEE, 342–347.
  12. Path Planning for Automation of Surgery Robot based on Probabilistic Roadmap and Reinforcement Learning. In 2018 15th International Conference on Ubiquitous Robots (UR). 342–347. https://doi.org/10.1109/URAI.2018.8441801
  13. Robotic Surgery With Lean Reinforcement Learning. arXiv preprint arXiv:2105.01006 (2021).
  14. Robotic Surgery With Lean Reinforcement Learning. https://doi.org/10.48550/ARXIV.2105.01006
  15. Interaction Networks for Learning about Objects, Relations and Physics. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2016/file/3147da8ab4a0437c15ef51a5cc7f2dc4-Paper.pdf
  16. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261 (2018).
  17. Léon Bottou. 2014. From machine learning to machine reasoning. Machine learning 94, 2 (2014), 133–149.
  18. Integration of reinforcement learning in a virtual robotic surgical simulation. Surgical Innovation 30, 1 (2023), 94–102.
  19. Public perceptions on robotic surgery, hospitals with robots, and surgeons that use them. Surgical endoscopy 30 (2016), 1310–1316.
  20. Joanna J. Bryson and Andreas Theodorou. 2019. How Society Can Maintain Human-Centric Artificial Intelligence. Springer Singapore, Singapore, 305–323.
  21. Interpretability of deep learning models: A survey of results. In 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). 1–6. https://doi.org/10.1109/UIC-ATC.2017.8397411
  22. Collaborative Robot-Assisted Endovascular Catheterization with Generative Adversarial Imitation Learning. In 2020 IEEE International Conference on Robotics and Automation (ICRA). 2414–2420. https://doi.org/10.1109/ICRA40945.2020.9196912
  23. The EMPATHIC Framework for Task Learning from Implicit Human Feedback. In Proceedings of the 2020 Conference on Robot Learning (Proceedings of Machine Learning Research, Vol. 155), Jens Kober, Fabio Ramos, and Claire Tomlin (Eds.). PMLR, 604–626. https://proceedings.mlr.press/v155/cui21a.html
  24. CUREXO. 2022. ROBODOC Surgical System. http://www.curexo.com/english/medical/sub01p01.php
  25. Felipe Leno Da Silva and Anna Helena Reali Costa. 2019. A survey on transfer learning for multiagent reinforcement learning systems. Journal of Artificial Intelligence Research 64 (2019), 645–703.
  26. Reinforcement learning in surgery. Surgery 170, 1 (2021), 329–332. https://doi.org/10.1016/j.surg.2020.11.040
  27. Physics-guided neural networks (pgnn): An application in lake temperature modeling. arXiv preprint arXiv:1710.11431 (2017).
  28. The da Vinci surgical system. In Surgical robotics. Springer, 199–217.
  29. Logic Tensor Networks for Semantic Image Interpretation. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’17). IJCAI Organization, 1596–1602. https://doi.org/10.24963/ijcai.2017/221
  30. The digital twin revolution in healthcare. In 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT). IEEE, 1–7.
  31. Richard Evans and Edward Grefenstette. 2018. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research 61 (2018), 1–64.
  32. Survey of imitation learning for robotic manipulation. International Journal of Intelligent Robotics and Applications 3, 4 (2019), 362–369.
  33. Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search. In 2020 IEEE International Conference on Robotics and Automation (ICRA). 8440–8446. https://doi.org/10.1109/ICRA40945.2020.9196674
  34. Avila Garcez and Gerson Zaverucha. 1999. The connectionist inductive learning and logic programming system. Applied Intelligence Journal 11, 1 (1999), 59–77.
  35. Neural-symbolic learning and reasoning: A survey and interpretation. Neuro-Symbolic Artificial Intelligence: The State of the Art 342 (2022), 1.
  36. Artur d’Avila Garcez and Luis C Lamb. 2020. Neurosymbolic AI: the 3rd wave. arXiv preprint arXiv:2012.05876 (2020).
  37. Towards deep symbolic reinforcement learning. arXiv preprint arXiv:1609.05518 (2016).
  38. Alexander Gepperth and Barbara Hammer. 2016. Incremental learning algorithms and applications. In European symposium on artificial neural networks (ESANN).
  39. Adaptive multimodal interaction with industrial robot. In 2012 IEEE 10th Jubilee International Symposium on Intelligent Systems and Informatics. IEEE, 329–333.
  40. ML-PersRef: A Machine Learning-Based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle. In Proceedings of the 23rd International Conference on Multimodal Interaction. ACM, New York, NY, USA, 318–327.
  41. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
  42. Alain Grumbach. 1995. Learning at subsymbolic and symbolic levels. In Neural Networks: Artificial Intelligence and Industrial Applications – Proceedings of the Annual SNN Symposium on Neural Networks, Bert Kappen and Stan Gielen (Eds.). Springer London, 91–94.
  43. Online continual learning through mutual information maximization. In International Conference on Machine Learning. PMLR, 8109–8126.
  44. Cooperative Inverse Reinforcement Learning. In Proceedings of the International Conference on Neural Information Processing Systems (NIPS’16) (Barcelona, Spain). Curran Associates Inc., 3916–3924. http://dl.acm.org/citation.cfm?id=3157382.3157535
  45. Robot-assisted minimally invasive surgery—Surgical robotics in the data age. Proc. IEEE 110, 7 (2022), 835–846.
  46. Visual Concept Metaconcept Learning. In Advances in Neural Information Processing Systems (NIPS).
  47. Lisa Hassel and Eli Hagen. 2005. Adaptation of an automotive dialogue system to users’ expertise. In Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue. 222–226.
  48. Jonathan Ho and Stefano Ermon. 2016. Generative adversarial imitation learning. Advances in neural information processing systems 29 (2016), 4565–4573.
  49. Current capabilities and development potential in surgical robotics. International Journal of Advanced Robotic Systems 12, 5 (2015), 61.
  50. Eugene H Holly. 1976. Mouth guide for operating microscope. Journal of neurosurgery 44, 5 (1976), 642–643.
  51. Towards the augmented pathologist: Challenges of explainable-ai in digital pathology. arXiv preprint arXiv:1712.06657 (2017).
  52. Learning for a robot: Deep reinforcement learning, imitation learning, transfer learning. Sensors 21, 4 (2021), 1278.
  53. Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot. arXiv:2302.09772 [cs.RO]
  54. Imitation learning: A survey of learning methods. ACM Computing Surveys (CSUR) 50, 2 (2017), 1–35.
  55. Feasibility of a voice-enabled automated platform for medical data collection: CardioCube. International Journal of Medical Informatics 129 (2019), 388–393. https://doi.org/10.1016/j.ijmedinf.2019.07.001
  56. Srinivasan Janarthanam and Oliver Lemon. 2014. Adaptive generation in dialogue systems using dynamic user modeling. Computational Linguistics 40, 4 (2014), 883–920.
  57. Guidelines were developed for data collection from medical records for use in retrospective analyses. Journal of clinical epidemiology 58, 3 (2005), 269–274.
  58. Relational graph neural network for situation recognition. Pattern Recognition 108 (2020), 107544.
  59. Reinforcement learning: A survey. Journal of artificial intelligence research 4 (1996), 237–285.
  60. Optical Coherence Tomography-Guided Robotic Ophthalmic Microsurgery via Reinforcement Learning from Demonstration. IEEE Transactions on Robotics 36, 4 (2020), 1207–1218. https://doi.org/10.1109/TRO.2020.2980158
  61. Troy Dale Kelley. 2006. Developing a psychologically inspired cognitive architecture for robotic control: The Symbolic and Subsymbolic Robotic Intelligence Control System (SS-RICS). International Journal of Advanced Robotic Systems 3, 3 (2006), 32.
  62. Towards autonomous eye surgery by combining deep imitation learning with optimal control. In Conference on Robot Learning. PMLR, 2347–2358.
  63. Optimal and autonomous control using reinforcement learning: A survey. IEEE transactions on neural networks and learning systems 29, 6 (2017), 2042–2062.
  64. W Bradley Knox and Peter Stone. 2009. Interactively shaping agents via human reinforcement: The TAMER framework. In Proceedings of the fifth international conference on Knowledge capture. 9–16.
  65. Application of machine learning to medical diagnosis. Machine learning and data mining: Methods and applications 389 (1997), 408.
  66. Constantinos Koutsojannis. 2020. Medical Knowledge Extraction: Particular Difficulties And Obligations. 37–61.
  67. Adam Daniel Laud. 2004. Theory and Application of Reward Shaping in Reinforcement Learning. Ph. D. Dissertation. USA. Advisor(s) Dejong, Gerald. AAI3130966.
  68. Y LeCun. 2022. A path towards autonomous machine intelligence. preprint posted on openreview (2022).
  69. Jangwon Lee. 2017. A survey of robot learning from demonstrations for human-robot collaboration. arXiv preprint arXiv:1710.08789 (2017).
  70. 3D Perception based Imitation Learning under Limited Demonstration for Laparoscope Control in Robotic Surgery. In 2022 International Conference on Robotics and Automation (ICRA). 7664–7670. https://doi.org/10.1109/ICRA46639.2022.9812010
  71. Human-centered reinforcement learning: A survey. IEEE Transactions on Human-Machine Systems 49, 4 (2019), 337–349.
  72. Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization. In International Conference on Learning Representations. https://openreview.net/forum?id=0cgU-BZp2ky
  73. Infogail: Interpretable imitation learning from visual demonstrations. Advances in Neural Information Processing Systems 30 (2017).
  74. Qianli Liao and Tomaso Poggio. 2017. Object-oriented deep learning. Technical Report. Center for Brains, Minds and Machines (CBMM).
  75. Task-level planning of pick-and-place robot motions. Computer 22, 3 (1989), 21–29.
  76. A multimodal human-machine interface enabling situation-Adaptive control inputs for highly automated vehicles. In 2017 IEEE Intelligent Vehicles Symposium (IV). IEEE, 1195–1200.
  77. The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision. In International Conference on Learning Representations.
  78. The More You Know: Using Knowledge Graphs for Image Classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  79. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).
  80. Mark A Musen. 1992. Dimensions of knowledge sharing and reuse. Computers and biomedical research 25, 5 (1992), 435–467.
  81. Building measurement and data collection into medical practice. Annals of Internal Medicine 128, 6 (1998), 460–466.
  82. Moddrop: adaptive multi-modal gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 8 (2015), 1692–1706.
  83. Algorithms for inverse reinforcement learning.. In Icml, Vol. 1. 2.
  84. Manipulating Soft Tissues by Deep Reinforcement Learning for Autonomous Robotic Surgery. In 2019 IEEE International Systems Conference (SysCon). 1–7. https://doi.org/10.1109/SYSCON.2019.8836924
  85. Autonomic Robotic Ultrasound Imaging System Based on Reinforcement Learning. IEEE Transactions on Biomedical Engineering 68, 9 (2021), 2787–2797. https://doi.org/10.1109/TBME.2021.3054413
  86. Assessing artificial intelligence for humanity: Will AI be the our biggest ever advance? Or the biggest threat [Opinion]. IEEE Technology and Society Magazine 37, 4 (2018), 26–34.
  87. Impact of robotic surgery on decision making: perspectives of surgical teams. In AMIA Annual Symposium Proceedings, Vol. 2015. American Medical Informatics Association, 1057.
  88. Alex Ratner and Christopher Ré. 2018. Knowledge Base Construction in the Machine-learning Era. Queue 16, 3, Article 50 (June 2018), 12 pages. https://doi.org/10.1145/3236386.3243045
  89. Recent advances in robot learning from demonstration. Annual review of control, robotics, and autonomous systems 3 (2020), 297–330.
  90. Open-sourced reinforcement learning environments for surgical robotics. arXiv preprint arXiv:1903.02090 (2019).
  91. Open-Sourced Reinforcement Learning Environments for Surgical Robotics. https://doi.org/10.48550/ARXIV.1903.02090
  92. Adaptive user interfaces for automotive environments. In Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No. 00TH8511). IEEE, 662–667.
  93. A cloud computing solution for patient’s data collection in health care institutions. In 2010 Second International Conference on eHealth, Telemedicine, and Social Medicine. IEEE, 95–99.
  94. Inverse Reinforcement Learning Intra-Operative Path Planning for Steerable Needle. IEEE Transactions on Biomedical Engineering 69, 6 (2022), 1995–2005. https://doi.org/10.1109/TBME.2021.3133075
  95. Luciano Serafini and Artur d’Avila Garcez. 2016. Logic tensor networks: Deep learning and logical reasoning from data and knowledge. arXiv preprint arXiv:1606.04422 (2016).
  96. Burr Settles. 2009. Active Learning Literature Survey. Computer Sciences Technical Report 1648. University of Wisconsin–Madison. http://axon.cs.byu.edu/~martinez/classes/778/Papers/settles.activelearning.pdf
  97. Jude W. Shavlik. 1994. Combining symbolic and neural learning. Machine Learning 14, 3 (March 1994), 321–331. https://doi.org/10.1007/BF00993982
  98. Ben Shneiderman. 2020. Human-centered artificial intelligence: Reliable, safe & trustworthy. International Journal of Human–Computer Interaction 36, 6 (2020), 495–504.
  99. Machine Teaching: A New Paradigm for Building Machine Learning Systems. CoRR abs/1707.06742 (2017). arXiv:1707.06742 http://arxiv.org/abs/1707.06742
  100. Marc P Sindou. 2009. Practical handbook of neurosurgery. Springer.
  101. Public Perceptions of Artificial Intelligence and Robotics in Medicine. J Endourol 34, 10 (Sept. 2020), 1041–1048.
  102. Russell Stewart and Stefano Ermon. 2017. Label-Free Supervision of Neural Networks with Physics and Domain Knowledge. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (San Francisco, California, USA) (AAAI’17). AAAI Press, 2576–2582.
  103. Understanding and Interpreting Machine Learning in Medical Image Computing Applications. Springer.
  104. Multicamera 3D Viewpoint Adjustment for Robotic Surgery via Deep Reinforcement Learning. Journal of Medical Robotics Research 06, 01n02 (2021), 2140003. https://doi.org/10.1142/S2424905X21400031 arXiv:https://doi.org/10.1142/S2424905X21400031
  105. Intuitive Surgical. 2022a. The Da Vinci Surgical robot. https://www.intuitive.com/en-us/about-us/company
  106. Think Surgical. 2022b. Think Surgical TSolution One. https://thinksurgical.com/products-and-services/tsolution-one/
  107. Robot-Assisted Training in Laparoscopy Using Deep Reinforcement Learning. IEEE Robotics and Automation Letters 4, 2 (2019), 485–492. https://doi.org/10.1109/LRA.2019.2891311
  108. Behavioral cloning from observation. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 4950–4957.
  109. Frank van Harmelen and Annette Ten Teije. 2019. A boxology of design patterns for hybrid learning and reasoning systems. In 31st Benelux Conference on Artificial Intelligence and the 28th Belgian Dutch Conference on Machine Learning, BNAIC/BENELEARN 2019. CEUR-WS, 1–2.
  110. Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards. arXiv:1707.08817 [cs.AI]
  111. Programmatically interpretable reinforcement learning. In International Conference on Machine Learning. PMLR, 5045–5054.
  112. Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems. IEEE Transactions on Knowledge & Data Engineering 35, 01 (jan 2023), 614–633. https://doi.org/10.1109/TKDE.2021.3079836
  113. A learning robot for cognitive camera control in minimally invasive surgery. Surgical Endoscopy (2021), 1–10.
  114. WashingtonUniversity. 2022. Length of Residencies. https://residency.wustl.edu/residencies/length-of-residencies/
  115. Torcs, the open racing car simulator. Software available at http://torcs. sourceforge. net 4, 6 (2000), 2.
  116. SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 1821–1828. https://doi.org/10.1109/IROS51168.2021.9635867
  117. Unsupervised method for extracting machine understandable medical knowledge from a large free text collection. In AMIA annual symposium proceedings, Vol. 2009. American Medical Informatics Association, 709.
  118. Wei Xu. 2019. Toward human-centered AI: a perspective from human-computer interaction. interactions 26, 4 (2019), 42–46.
  119. Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. Advances in neural information processing systems 31 (2018).
  120. David Zakim and Matthias Schwab. 2015. Data collection as a barrier to personalized medicine. Trends in pharmacological sciences 36, 2 (2015), 68–71.
  121. Qunzhi Zhang and Didier Sornette. 2017. Learning like humans with Deep Symbolic Networks. arXiv preprint arXiv:1707.03377 (2017).
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now