On the Feasibility of EEG-based Motor Intention Detection for Real-Time Robot Assistive Control (2403.08149v1)
Abstract: This paper explores the feasibility of employing EEG-based intention detection for real-time robot assistive control. We focus on predicting and distinguishing motor intentions of left/right arm movements by presenting: i) an offline data collection and training pipeline, used to train a classifier for left/right motion intention prediction, and ii) an online real-time prediction pipeline leveraging the trained classifier and integrated with an assistive robot. Central to our approach is a rich feature representation composed of the tangent space projection of time-windowed sample covariance matrices from EEG filtered signals and derivatives; allowing for a simple SVM classifier to achieve unprecedented accuracy and real-time performance. In pre-recorded real-time settings (160 Hz), a peak accuracy of 86.88% is achieved, surpassing prior works. In robot-in-the-loop settings, our system successfully detects intended motion solely from EEG data with 70% accuracy, triggering a robot to execute an assistive task. We provide a comprehensive evaluation of the proposed classifier.
- A. Sciutti, M. Mara, V. Tagliasco, and G. Sandini, “Humanizing human-robot interaction: On the importance of mutual understanding,” IEEE Technology and Society Magazine, vol. 37, pp. 22–29, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:3812272
- D. Kulić and E. A. Croft, “Estimating intent for human-robot interaction,” Advanced Robotics, 2003. [Online]. Available: https://api.semanticscholar.org/CorpusID:1120056
- Y. Athavale and S. Krishnan, “Biosignal monitoring using wearables: Observations and opportunities,” Biomedical Signal Processing and Control, vol. 38, pp. 22–33, 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1746809417300617
- D. Esposito, J. Centracchio, E. Andreozzi, G. D. Gargiulo, G. R. Naik, and P. Bifulco, “Biosignal-Based Human-Machine interfaces for assistance and rehabilitation: A survey,” Sensors (Basel), vol. 21, no. 20, Oct. 2021.
- C. Tang, Z. Xu, E. Occhipinti, W. Yi, M. Xu, S. Kumar, G. S. Virk, S. Gao, and L. G. Occhipinti, “From brain to movement: Wearables-based motion intention prediction across the human nervous system,” Nano Energy, vol. 115, p. 108712, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2211285523005499
- A. Mohebbi, “Human-Robot Interaction in Rehabilitation and Assistance: a Review,” Current Robotics Reports, vol. 1, no. 3, pp. 131–144, Sep. 2020. [Online]. Available: https://doi.org/10.1007/s43154-020-00015-4
- H. M. Lakany and B. A. Conway, “Understanding intention of movement from electroencephalograms,” Expert Systems, vol. 24, 2007. [Online]. Available: https://api.semanticscholar.org/CorpusID:5936372
- T. Kirschstein and R. Köhling, “What is the source of the eeg?” Clinical EEG and neuroscience, vol. 40, no. 3, pp. 146–149, 2009.
- K. Värbu, M. Naveed, and Y. Muhammad, “Past, present, and future of eeg-based bci applications,” Sensors (Basel, Switzerland), vol. 22, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:248775078
- M. B. Khalid, N. I. Rao, I. Rizwan-i Haque, S. Munir, and F. Tahir, “Towards a brain computer interface using wavelet transform with averaged and time segmented adapted wavelets,” in 2009 2nd International Conference on Computer, Control and Communication, 2009, pp. 1–4.
- J. Meng, S. Zhang, A. Bekyo, J. Olsoe, B. Baxter, and B. He, “Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks,” Scientific Reports, vol. 6, no. 1, p. 38565, Dec. 2016. [Online]. Available: https://doi.org/10.1038/srep38565
- R. Zhang, S. Lee, M. Hwang, A. Hiranaka, C. Wang, W. Ai, J. J. R. Tan, S. Gupta, Y. Hao, G. Levine, R. Gao, A. Norcia, L. Fei-Fei, and J. Wu, “Noir: Neural signal operated intelligent robots for everyday activities,” in 7th Annual Conference on Robot Learning, 2023.
- A. Dillen, D. Steckelmacher, K. Efthymiadis, K. Langlois, A. De Beir, U. Marusic, B. Vanderborght, A. Nowé, R. Meeusen, F. Ghaffari, O. Romain, and K. De Pauw, “Deep learning for biosignal control: insights from basic to real-time methods with recommendations,” J Neural Eng, vol. 19, no. 1, Feb. 2022.
- F. S. Racz, R. Fakhreddine, S. Kumar, and J. Del R. Millan, “Riemannian geometry-based detection of slow cortical potentials during movement preparation,” in 2023 11th International IEEE/EMBS Conference on Neural Engineering (NER), 2023, pp. 1–5.
- F. Yger, M. Berar, and F. Lotte, “Riemannian approaches in brain-computer interfaces: A review,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 10, pp. 1753–1762, 2017.
- J. Ying, Q. Wei, and X. Zhou, “Riemannian geometry-based transfer learning for reducing training time in c-VEP BCIs,” Scientific Reports, vol. 12, no. 1, p. 9818, Jun. 2022. [Online]. Available: https://doi.org/10.1038/s41598-022-14026-y
- M. Congedo, A. Barachant, and R. Bhatia, “Riemannian geometry for eeg-based brain-computer interfaces; a primer and a review,” 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:13857467
- P. L. C. Rodrigues, C. Jutten, and M. Congedo, “Riemannian procrustes analysis: Transfer learning for brain–computer interfaces,” IEEE Transactions on Biomedical Engineering, vol. 66, pp. 2390–2401, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:58642888
- A. Barachant, S. Bonnet, M. Congedo, and C. Jutten, “Multiclass brain–computer interface classification by riemannian geometry,” IEEE Transactions on Biomedical Engineering, vol. 59, pp. 920–928, 2012. [Online]. Available: https://api.semanticscholar.org/CorpusID:423006
- ——, “Classification of covariance matrices using a riemannian-based kernel for bci applications,” Neurocomputing, vol. 112, pp. 172–178, 2013. [Online]. Available: https://api.semanticscholar.org/CorpusID:13873072
- C. M. Michel and D. Brunet, “Eeg source imaging: A practical review of the analysis steps,” Frontiers in Neurology, vol. 10, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:93003798
- Y. Wang, S. Gao, and X. Gao, “Common spatial pattern method for channel selelction in motor imagery based brain-computer interface,” in 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, 2005, pp. 5392–5395.
- D. A. Andreou and R. Poli, “Comparing eeg, its time-derivative and their joint use as features in a bci for 2-d pointer control,” 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5853–5856, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:172677
- M. Menceloglu, M. Grabowecky, and S. Suzuki, “Spectral-power associations reflect amplitude modulation and within-frequency interactions on the sub-second timescale and cross-frequency interactions on the seconds timescale,” PLoS ONE, vol. 15, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:214419358
- G. Chen, H. S. Helm, K. Lytvynets, W. Yang, and C. E. Priebe, “Mental state classification using multi-graph features,” Frontiers in Human Neuroscience, vol. 16, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:247187688
- L. Bi, A. G. Feleke, and C. Guan, “A review on emg-based motor intention prediction of continuous human upper limb motion for human-robot collaboration,” Biomedical Signal Processing and Control, vol. 51, pp. 113–127, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1746809419300473
- E. Trigili, L. Grazi, S. Crea, A. Accogli, J. Carpaneto, S. Micera, N. Vitiello, and A. Panarese, “Detection of movement onset using emg signals for upper-limb exoskeletons in reaching tasks,” Journal of neuroengineering and rehabilitation, vol. 16, pp. 1–16, 2019.
- A. Schurger, P. B. Hu, J. Pak, and A. L. Roskies, “What is the readiness potential?” Trends in Cognitive Sciences, vol. 25, no. 7, pp. 558–570, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1364661321000930
- E. Travers, N. Khalighinejad, A. Schurger, and P. Haggard, “Do readiness potentials happen all the time?” NeuroImage, vol. 206, p. 116286, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1053811919308778
- A. Schurger, P. Hu, J. Pak, and A. L. Roskies, “What is the readiness potential?” Trends in cognitive sciences, vol. 25, pp. 558 – 570, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:233471654
- X. Tang, W. Li, X. Li, W. Ma, and X. Dang, “Motor imagery eeg recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network,” Expert Systems with Applications, vol. 149, p. 113285, 2020.
- P. Batres-Mendoza, E. I. Guerra-Hernandez, A. Espinal, E. Pérez-Careta, and H. Rostro-Gonzalez, “Biologically-inspired legged robot locomotion controlled with a bci by means of cognitive monitoring,” IEEE Access, vol. 9, pp. 35 766–35 777, 2021.
- S. Bhattacharyya, A. Konar, and D. Tibarewala, “Motor imagery and error related potential induced position control of a robotic arm,” IEEE/CAA Journal of Automatica Sinica, vol. 4, no. 4, pp. 639–650, 2017.
- J. Choi, K. T. Kim, J. H. Jeong, L. Kim, S. J. Lee, and H. Kim, “Developing a motor imagery-based real-time asynchronous hybrid bci controller for a lower-limb exoskeleton,” Sensors, vol. 20, no. 24, p. 7309, 2020.
- H. Gao, L. Luo, M. Pi, Z. Li, Q. Li, K. Zhao, and J. Huang, “Eeg-based volitional control of prosthetic legs for walking in different terrains,” IEEE Transactions on Automation Science and Engineering, vol. 18, no. 2, pp. 530–540, 2019.
- J. Andreu-Perez, F. Cao, H. Hagras, and G.-Z. Yang, “A self-adaptive online brain–machine interface of a humanoid robot through a general type-2 fuzzy inference system,” IEEE Transactions on Fuzzy Systems, vol. 26, no. 1, pp. 101–116, 2016.
- L. Tonin, F. C. Bauer, and J. d. R. Millán, “The role of the control framework for continuous teleoperation of a brain–machine interface-driven mobile robot,” IEEE Transactions on Robotics, vol. 36, no. 1, pp. 78–91, 2019.
- D. Liu, W. Chen, Z. Pei, and J. Wang, “A brain-controlled lower-limb exoskeleton for human gait training,” Review of Scientific Instruments, vol. 88, no. 10, 2017.
- J. R. Millan and J. Mouriño, “Asynchronous bci and local neural classifiers: an overview of the adaptive brain interface project,” IEEE transactions on neural systems and rehabilitation engineering, vol. 11, no. 2, pp. 159–161, 2003.
- L. Junwei, S. Ramkumar, G. Emayavaramban, M. Thilagaraj, V. Muneeswaran, M. P. Rajasekaran, V. Venkataraman, A. F. Hussein et al., “Brain computer interface for neurodegenerative person using electroencephalogram,” IEEE Access, vol. 7, pp. 2439–2452, 2018.
- T. Li, J. Hong, J. Zhang, and F. Guo, “Brain–machine interface control of a manipulator using small-world neural network and shared control strategy,” Journal of neuroscience methods, vol. 224, pp. 26–38, 2014.
- Z. Tang, S. Sun, S. Zhang, Y. Chen, C. Li, and S. Chen, “A brain-machine interface based on erd/ers for an upper-limb exoskeleton control,” Sensors, vol. 16, no. 12, p. 2050, 2016.
- G. Kucukyildiz, H. Ocak, S. Karakaya, and O. Sayli, “Design and implementation of a multi sensor based brain computer interface for a robotic wheelchair,” Journal of Intelligent & Robotic Systems, vol. 87, pp. 247–263, 2017.
- C. H. Nguyen and P. K. Artemiadis, “Eeg feature descriptors and discriminant analysis under riemannian manifold perspective,” Neurocomputing, vol. 275, pp. 1871–1883, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:894293
- F. P. Kalaganis, N. A. Laskaris, E. Chatzilari, S. Nikolopoulos, and I. Kompatsiaris, “A riemannian geometry approach to reduced and discriminative covariance estimation in brain computer interfaces,” IEEE Transactions on Biomedical Engineering, vol. 67, pp. 245–255, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:122544415
- S. Calinon, “Gaussians on riemannian manifolds: Applications for robot learning and adaptive control,” IEEE Robotics & Automation Magazine, vol. 27, pp. 33–45, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:216322247
- EEG System for Clinical and Research Use — Bittium NeurOne. Bittium. Accessed 2023-03-06. [Online]. Available: https://www.bittium.com/medical/bittium-neuron
- J. A. Pineda, B. Z. Allison, and A. Vankov, “The effects of self-movement, observation, and imagination on/spl mu/rhythms and readiness potentials (rp’s): toward a brain-computer interface (bci),” IEEE Transactions on Rehabilitation Engineering, vol. 8, no. 2, pp. 219–222, 2000.
- Y. Wang, S. Gao, and X. Gao, “Common spatial pattern method for channel selelction in motor imagery based brain-computer interface,” 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, pp. 5392–5395, 2005. [Online]. Available: https://api.semanticscholar.org/CorpusID:20814659
- W. Wen, R. Minohara, S. Hamasaki, T. Maeda, Q. An, Y. Tamura, H. Yamakawa, A. Yamashita, and H. Asama, “The readiness potential reflects the reliability of action consequence,” Scientific Reports, vol. 8, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:51940807