Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 85 tok/s
Gemini 2.5 Pro 39 tok/s Pro
GPT-5 Medium 35 tok/s Pro
GPT-5 High 19 tok/s Pro
GPT-4o 98 tok/s Pro
GPT OSS 120B 468 tok/s Pro
Kimi K2 202 tok/s Pro
2000 character limit reached

Multiagent Copilot Approach for Shared Autonomy between Human EEG and TD3 Deep Reinforcement Learning (2312.14458v1)

Published 22 Dec 2023 in cs.HC, cs.LG, eess.SP, and q-bio.NC

Abstract: Deep reinforcement learning (RL) algorithms enable the development of fully autonomous agents that can interact with the environment. Brain-computer interface (BCI) systems decipher human implicit brain signals regardless of the explicit environment. In this study, we integrated deep RL and BCI to improve beneficial human interventions in autonomous systems and the performance in decoding brain activities by considering environmental factors. Shared autonomy was allowed between the action command decoded from the electroencephalography (EEG) of the human agent and the action generated from the twin delayed DDPG (TD3) agent for a given environment. Our proposed copilot control scheme with a full blocker (Co-FB) significantly outperformed the individual EEG (EEG-NB) or TD3 control. The Co-FB model achieved a higher target approaching score, lower failure rate, and lower human workload than the EEG-NB model. The Co-FB control scheme had a higher invisible target score and level of allowed human intervention than the TD3 model. We also proposed a disparity d-index to evaluate the effect of contradicting agent decisions on the control accuracy and authority of the copilot model. We found a significant correlation between the control authority of the TD3 agent and the performance improvement of human EEG classification with respect to the d-index. We also observed that shifting control authority to the TD3 agent improved performance when BCI decoding was not optimal. These findings indicate that the copilot system can effectively handle complex environments and that BCI performance can be improved by considering environmental factors. Future work should employ continuous action space and different multi-agent approaches to evaluate copilot performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Y. Li, et al., “An EEG-based BCI system for 2-D cursor control by combining Mu/Beta rhythm and P300 potential,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 10, pp. 2495–2505, 2010.
  2. J. Long, et al., “Target selection with hybrid feature for BCI-based 2-D cursor control,” IEEE Transactions on Biomedical Engineering, vol. 59, no. 1, pp. 132–140, 2011.
  3. B. Z. Allison, et al., “A hybrid ERD/SSVEP BCI for continuous simultaneous two dimensional cursor control,” Journal of Neuroscience Methods, vol. 209, no. 2, pp. 299–307, 2012.
  4. Y. Li, et al., “A hybrid BCI system combining P300 and SSVEP and its application to wheelchair control,” IEEE Transactions on Biomedical Engineering, vol. 60, no. 11, pp. 3156–3166, 2013.
  5. T. Carlson and J. d. R. Millan, “Brain-controlled wheelchairs: a robotic architecture,” IEEE Robotics & Automation Magazine, vol. 20, no. 1, pp. 65–73, 2013.
  6. A. Frisoli, et al., “A new gaze-BCI-driven control of an upper limb exoskeleton for rehabilitation in real-world tasks,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 1169–1179, 2012.
  7. M. Barsotti, et al., “A full upper limb robotic exoskeleton for reaching and grasping rehabilitation triggered by MI-BCI,” 2015 IEEE International Conference on Rehabilitation Robotics (ICORR), pp. 49–54, 2015.
  8. A. Frolov, et al., “Preliminary results of a controlled study of BCI-exoskeleton technology efficacy in patients with poststroke arm paresis,” Bulletin of Russian State Medical University, , no. 2, 2016.
  9. J. Decety and D. H. Ingvar, “Brain structures participating in mental simulation of motor behavior: A neuropsychological interpretation,” Acta Psychologica, vol. 73, no. 1, pp. 13–34, 1990.
  10. K. D. Nielsen, et al., “Eeg based bci-towards a better control. brain-computer interface research at aalborg university,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 14, no. 2, pp. 202–204, 2006.
  11. Y.-H. Liu, et al., “Analysis of electroencephalography event-related desynchronisation and synchronisation induced by lower-limb stepping motor imagery,” Journal of Medical and Biological Engineering, vol. 39, no. 1, pp. 54–69, 2019.
  12. M. Sun, et al., “Asynchronous brain-computer interface with foot motor imagery,” 2013 ICME International Conference on Complex Medical Engineering, CME 2013, pp. 191–196, 2013.
  13. Y. Hashimoto and J. Ushiba, “EEG-based classification of imaginary left and right foot movements using beta rebound,” Clinical Neurophysiology, vol. 124, no. 11, pp. 2153–2160, 2013.
  14. E. A. Mohamed, et al., “Comparison of EEG signal decomposition methods in classification of motor-imagery BCI,” Multimedia Tools and Applications, vol. 77, no. 16, pp. 21305–21327, 2018.
  15. K. K. Ang, et al., “Filter bank common spatial pattern (FBCSP) in brain-computer interface,” 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), pp. 2390–2397, 2008.
  16. C.-R. Phang, et al., “Frontoparietal dysconnection in covert bipedal activity for enhancing the performance of the motor preparation-based brain–computer interface,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 31, pp. 139–149, 2022.
  17. B. Blankertz, et al., “Neurophysiological predictor of SMR-based BCI performance,” Neuroimage, vol. 51, no. 4, pp. 1303–1309, 2010.
  18. C. Vidaurre and B. Blankertz, “Towards a cure for BCI illiteracy,” Brain topography, vol. 23, no. 2, pp. 194–198, 2010.
  19. H. Bharadhwaj, et al., “Conservative safety critics for exploration,” arXiv preprint arXiv:2010.14497, 2020.
  20. Q. Li, et al., “Efficient learning of safe driving policy via human-ai copilot optimization,” arXiv preprint arXiv:2202.10341, 2022.
  21. D. Xu, et al., “Accelerating reinforcement learning using eeg-based implicit human feedback,” Neurocomputing, vol. 460, pp. 139–153, 2021.
  22. “Still not solved: A call for renewed focus on user-centered teleoperation interfaces,” Frontiers in Robotics and AI, vol. 9, pp. 704225, 2022.
  23. C. Yang, et al., “A review of human–machine cooperation in the robotics domain,” IEEE Transactions on Human-Machine Systems, vol. 52, no. 1, pp. 12–25, 2021.
  24. F. Flemisch, et al., “Joining the blunt and the pointy end of the spear: towards a common framework of joint action, human–machine cooperation, cooperative guidance and control, shared, traded and supervisory control,” Cognition, Technology & Work, vol. 21, pp. 555–568, 2019.
  25. W. Schwarting, et al., “Parallel autonomy in automated vehicles: Safe motion generation with minimal intervention,” 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1928–1935, 2017.
  26. F. Mars, et al., “Analysis of human-machine cooperation when driving with different degrees of haptic shared control,” IEEE transactions on haptics, vol. 7, no. 3, pp. 324–333, 2014.
  27. S. Reddy, et al., “Shared autonomy via deep reinforcement learning,” arXiv preprint arXiv:1802.01744, 2018.
  28. X. Wang, et al., “Error-related potential-based shared autonomy via deep recurrent reinforcement learning,” Journal of Neural Engineering, vol. 19, no. 6, pp. 066023, 2022.
  29. M. Kaya, et al., “A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces,” Scientific data, vol. 5, no. 1, pp. 1–16, 2018.
  30. M. Tangermann, et al., “Review of the bci competition iv,” Frontiers in neuroscience, p. 55, 2012.
  31. S. Fujimoto, et al., “Addressing function approximation error in actor-critic methods,” International conference on machine learning, pp. 1587–1596, 2018.
  32. W. Saunders, et al., “Trial without error: Towards safe reinforcement learning via human intervention,” arXiv preprint arXiv:1707.05173, 2017.
  33. B. Prakash, et al., “Improving safety in reinforcement learning using model-based architectures and human intervention,” arXiv preprint arXiv:1903.09328, 2019.
  34. H. Van Hasselt, et al., “Deep reinforcement learning with double q-learning,” Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, 2016.
  35. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  36. W. J. Conover, Practical nonparametric statistics, vol. 350, john wiley & sons, 1999.
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Ai Generate Text Spark Streamline Icon: https://streamlinehq.com

Paper Prompts

Sign up for free to create and run prompts on this paper using GPT-5.

List Streamline Icon: https://streamlinehq.com Explore 10 Community Prompts
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now