2000 character limit reached
Human-Machine Co-Adaptation for Robot-Assisted Rehabilitation via Dual-Agent Multiple Model Reinforcement Learning (DAMMRL)
Published 31 Jul 2024 in cs.RO | (2407.21734v1)
Abstract: This study introduces a novel approach to robot-assisted ankle rehabilitation by proposing a Dual-Agent Multiple Model Reinforcement Learning (DAMMRL) framework, leveraging multiple model adaptive control (MMAC) and co-adaptive control strategies. In robot-assisted rehabilitation, one of the key challenges is modelling human behaviour due to the complexity of human cognition and physiological systems. Traditional single-model approaches often fail to capture the dynamics of human-machine interactions. Our research employs a multiple model strategy, using simple sub-models to approximate complex human responses during rehabilitation tasks, tailored to varying levels of patient incapacity. The proposed system's versatility is demonstrated in real experiments and simulated environments. Feasibility and potential were evaluated with 13 healthy young subjects, yielding promising results that affirm the anticipated benefits of the approach. This study not only introduces a new paradigm for robot-assisted ankle rehabilitation but also opens the way for future research in adaptive, patient-centred therapeutic interventions.
- S. Khalid, F. Alnajjar, M. Gochoo, A. Renawi, and S. Shimoda, “Robotic assistive and rehabilitation devices leading to motor recovery in upper limb: a systematic review,” Disability and Rehabilitation: Assistive Technology, vol. 18, no. 5, pp. 658–672, 2023.
- W. M. Naqvi, I. Naqvi, G. V. Mishra, and V. Vardhan, “The dual importance of virtual reality usability in rehabilitation: A focus on therapists and patients,” Cureus, vol. 16, no. 3, 2024.
- M. Stevanovic, T. Valkeapää, E. Weiste, and C. Lindholm, “Joint decision making in a mental health rehabilitation community: the impact of support workers’ proposal design on client responsiveness,” Counselling Psychology Quarterly, vol. 35, no. 1, pp. 129–154, 2022.
- R. Zhang, Q. Lv, J. Li, J. Bao, T. Liu, and S. Liu, “A reinforcement learning method for human-robot collaboration in assembly tasks,” Robotics and Computer-Integrated Manufacturing, vol. 73, p. 102227, 2022.
- D. Mukherjee, K. Gupta, L. H. Chang, and H. Najjaran, “A survey of robot learning strategies for human-robot collaboration in industrial settings,” Robotics and Computer-Integrated Manufacturing, vol. 73, p. 102231, 2022.
- B. B. Doll, D. A. Simon, and N. D. Daw, “The ubiquity of model-based reinforcement learning,” Current opinion in neurobiology, vol. 22, no. 6, pp. 1075–1081, 2012.
- F. Zini, F. Le Piane, and M. Gaspari, “Adaptive cognitive training with reinforcement learning,” ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 12, no. 1, pp. 1–29, 2022.
- A. Ghadirzadeh, J. Bütepage, A. Maki, D. Kragic, and M. Björkman, “A sensorimotor reinforcement learning framework for physical human-robot interaction,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2016, pp. 2682–2688.
- A. Plaat, W. Kosters, and M. Preuss, “Model-based deep reinforcement learning for high-dimensional problems, a survey,” arXiv preprint arXiv:2008.05598, 2020.
- J. B. Hamrick, “Analogues of mental simulation and imagination in deep learning,” Current Opinion in Behavioral Sciences, vol. 29, pp. 8–16, 2019, artificial Intelligence.
- D. Kahneman, D. Lovallo, and O. Sibony, “Before you make that big decision,” Harvard business review, vol. 89, no. 6, pp. 50–60, 2011.
- A. Kubota and L. D. Riek, “Methods for robot behavior adaptation for cognitive neurorehabilitation,” Annual review of control, robotics, and autonomous systems, vol. 5, pp. 109–135, 2022.
- K. Guo, A. Cheng, Y. Li, J. Li, R. Duffield, and S. W. Su, “Cooperative markov decision process model for human–machine co-adaptation in robot-assisted rehabilitation,” Knowledge-Based Systems, vol. 291, p. 111572, 2024.
- Y. Bai, T. Xie, N. Jiang, and Y.-X. Wang, “Provably efficient q-learning with low switching cost,” arXiv preprint arXiv:1905.12849, 2019.
- T. M. Moerland, J. Broekens, and C. M. Jonker, “Model-based reinforcement learning: A survey,” arXiv preprint arXiv:2006.16712, 2020.
- Y. Yang and J. Wang, “An overview of multi-agent reinforcement learning from game theoretical perspective,” arXiv preprint arXiv:2011.00583, 2020.
- G. Qingji, W. Kai, and L. Haijuan, “A robot emotion generation mechanism based on pad emotion space,” in International Conference on Intelligent Information Processing. Springer, 2008, pp. 138–147.
- G. Wang, Z. Wang, S. Teng, Y. Xie, and Y. Wang, “Emotion model of interactive virtual humans on the basis of mdp,” Frontiers of Electrical and Electronic Engineering in China, vol. 2, no. 2, pp. 156–160, 2007.
- Y. Huang, R. Song, A. Argha, A. V. Savkin, B. G. Celler, and S. W. Su, “Continuous description of human 3d motion intent through switching mechanism,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 28, no. 1, pp. 277–286, 2020.
- Y. Huang, R. Song, A. Argha, B. G. Celler, A. V. Savkin, and S. W. Su, “Human motion intent description based on bumpless switching mechanism for rehabilitation robot,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 29, pp. 673–682, 2021.
- A. Argha, S. W. Su, and B. G. Celler, “Heart rate regulation during cycle-ergometer exercise via event-driven biofeedback,” Medical & biological engineering & computing, vol. 55, pp. 483–492, 2017.
- W. Blewitt, A. Ayesh, R. I. John, and S. Coupland, “A millenson-based approach to emotion modelling,” in 2008 Conference on human system interactions. IEEE, 2008, pp. 491–496.
- H. Candra, M. Yuwono, R. Chai, A. Handojoseno, I. Elamvazuthi, H. T. Nguyen, and S. Su, “Investigation of window size in classification of eeg-emotion signal with wavelet entropy and support vector machine,” in 2015 37th Annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2015, pp. 7250–7253.
- M. Maadi, H. Akbarzadeh Khorshidi, and U. Aickelin, “A review on human–ai interaction in machine learning and insights for medical applications,” International journal of environmental research and public health, vol. 18, no. 4, p. 2121, 2021.
- L. Wang, W. Huang, M. Zhang, S. Pan, X. Chang, and S. W. Su, “Pruning graph neural networks by evaluating edge properties,” Knowledge-Based Systems, vol. 256, p. 109847, 2022.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.