Sparse Wearable Sonomyography Sensor-based Proprioceptive Proportional Control Across Multiple Gestures (2403.05308v1)
Abstract: Sonomyography (SMG) is a non-invasive technique that uses ultrasound imaging to detect the dynamic activity of muscles. Wearable SMG systems have recently gained popularity due to their potential as human-computer interfaces for their superior performance compared to conventional methods. This paper demonstrates real-time positional proportional control of multiple gestures using a multiplexed 8-channel wearable SMG system. The amplitude-mode ultrasound signals from the SMG system were utilized to detect muscle activity from the forearm of 8 healthy individuals. The derived signals were used to control the on-screen movement of the cursor. A target achievement task was performed to analyze the performance of our SMG-based human-machine interface. Our wearable SMG system provided accurate, stable, and intuitive control in real-time by achieving an average success rate greater than 80% with all gestures. Furthermore, the wearable SMG system's abilities to detect volitional movement and decode movement kinematic information from SMG trajectories using standard performance metrics were evaluated. Our results provide insights to validate SMG as an intuitive human-machine interface.
- J. M. Hahne, M. A. Schweisfurth, M. Koppe, and D. Farina, “Simultaneous control of multiple functions of bionic hand prostheses: Performance and robustness in end users,” Science Robotics, vol. 3, no. 19, 2018.
- N. Jiang, J. L. Vest-Nielsen, S. Muceli, and D. Farina, “EMG-based simultaneous and proportional estimation of wrist/hand kinematics in uni-lateral trans-radial amputees,” Journal of NeuroEngineering and Rehabilitation, vol. 9, no. 1, 2012.
- L. Resnik, H. H. Huang, A. Winslow, D. L. Crouch, F. Zhang, and N. Wolk, “Evaluation of EMG pattern recognition for upper limb prosthesis control: A case study in Comparison with direct myoelectric control,” Journal of NeuroEngineering and Rehabilitation, vol. 15, no. 1, pp. 1–13, 2018.
- N. Jiang, K. B. Englehart, and P. A. Parker, “Extracting simultaneous and proportional neural control information for multiple-dof prostheses from the surface electromyographic signal,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 4, pp. 1070–1080, 2009.
- E. Scheme and K. Englehart, “Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use.” Journal of Rehabilitation Research and Development, vol. 48, no. 6, pp. 643–659, 2011.
- R. Vinjamuri, Z.-H. Mao, R. Sclabassi, and M. Sun, “Limitations of surface emg signals of extrinsic muscles in predicting postures of human hand,” in 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, 2006, pp. 5491–5494.
- Q. Zhang, A. Iyer, K. Kim, and N. Sharma, “Evaluation of non-invasive ankle joint effort prediction methods for use in neurorehabilitation using electromyography and ultrasound imaging,” IEEE Transactions on Biomedical Engineering, vol. 68, no. 3, pp. 1044–1055, 2021.
- L. A. Hallock, A. Velu, A. Schwartz, and R. Bajcsy, “Muscle deformation correlates with output force during isometric contraction,” in 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), 2020, pp. 1188–1195.
- J. Shi, Y. Zheng, and Z. Yan, “The relationship between SEMG and change in pennation angle of brachialis,” Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings, pp. 4802–4805, 2007.
- P. W. Hodges, L. H. M. Pengel, R. D. Herbert, and S. C. Gandevia, “Measurement of muscle contraction with ultrasound imaging.” Muscle & Nerve, 2003.
- J. McIntosh, A. Marzo, M. Fraser, and C. Phillips, “Echoflex: Hand gesture recognition using ultrasound imaging,” in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ser. CHI ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 1923–1934.
- N. Akhlaghi, C. A. Baker, M. Lahlou, H. Zafar, K. G. Murthy, H. S. Rangwala, J. Kosecka, W. M. Joiner, J. J. Pancrazio, and S. Sikdar, “Real-time classification of hand motions using ultrasound imaging of forearm muscles,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 8, pp. 1687–1698, 2016.
- J. Shi, Q. Chang, and Y.-P. Zheng, “Feasibility of controlling prosthetic hand using sonomyography signal in real-time: Preliminary study,” Journal of Rehabilitation Research and Development, 2010.
- A. S. Dhawan, B. Mukherjee, S. Patwardhan, N. Akhlaghi, G. Diao, G. Levay, R. Holley, W. M. Joiner, M. Harris-Love, and S. Sikdar, “Proprioceptive Sonomyographic Control: A novel method for intuitive and proportional control of multiple degrees-of-freedom for individuals with upper extremity limb loss,” Scientific Reports, vol. 9, no. 1, p. 9499, 2019.
- S. M. Engdahl, S. A. Acuña, E. L. King, A. Bashatah, and S. Sikdar, “First Demonstration of Functional Task Performance Using a Sonomyographic Prosthesis: A Case Study,” Frontiers in Bioengineering and Biotechnology, vol. 10, no. May, pp. 1–20, 2022.
- N. Akhlaghi, A. Dhawan, A. A. Khan, B. Mukherjee, G. Diao, C. Truong, and S. Sikdar, “Sparsity analysis of a sonomyographic muscle–computer interface,” IEEE Transactions on Biomedical Engineering, vol. 67, no. 3, pp. 688–696, 2020.
- A. T. Kamatham, M. Alzamani, A. Dockum, S. Sikdar, and B. Mukherjee, “Sparse sonomyography-based estimation of isometric force: A comparison of methods and features,” IEEE Transactions on Medical Robotics and Bionics, vol. 4, no. 3, pp. 821–829, 2022.
- X. Yang, Y. Zhou, and H. Liu, “Wearable ultrasound-based decoding of simultaneous wrist/hand kinematics,” IEEE Transactions on Industrial Electronics, vol. 68, no. 9, pp. 8667–8675, 2021.
- A. T. Kamatham and B. Mukherjee, “Design and optimization of a wearable sonomyography sensor for dynamic muscle activity monitoring,” in 2023 IEEE Applied Sensing Conference (APSCON), 2023, pp. 1–3.
- I. AlMohimeed and Y. Ono, “Ultrasound measurement of skeletal muscle contractile parameters using flexible and wearable single-element ultrasonic sensor.” Sensors, 2020.
- J. Yan, X. Yang, X. Sun, Z. Chen, and H. Liu, “A lightweight ultrasound probe for wearable human–machine interfaces,” IEEE Sensors Journal, vol. 19, no. 14, pp. 5895–5903, 2019.
- J.-Y. Guo, Y.-P. Zheng, Q.-H. Huang, X. Chen, and J.-F. He, “Comparison of sonomyography and electromyography of forearm muscles in the guided wrist extension,” in 2008 5th International Summer School and Symposium on Medical Devices and Biosensors, 2008, pp. 235–238.
- A. T. Kamatham, A. Shariati, H. A. Wurdemann, and B. Mukherjee, “A multiplexed sonomyography system for proprioceptive proportional control of biomechatronic interfaces,” in 2023 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), 2023, pp. 1–6.
- X. Yang, Y. Li, Y. Fang, and H. Liu, “A preliminary study on the relationship between grip force and muscle thickness,” in 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), 2017, pp. 118–121.
- S. Cai, Z. Lu, L. Guo, Z. Qing, and L. Yao, “The let procedure for gesture recognition with multiple forearm angles,” IEEE Sensors Journal, vol. 22, no. 13, pp. 13 226–13 233, 2022.
- J. He, H. Luo, J. Jia, J. T. W. Yeow, and N. Jiang, “Wrist and finger gesture recognition with single-element ultrasound signals: A comparison with single-channel surface electromyogram,” IEEE Transactions on Biomedical Engineering, vol. 66, no. 5, pp. 1277–1284, 2019.
- W. Xia, Y. Zhou, X. Yang, K. He, and H. Liu, “Toward portable hybrid surface electromyography/a-mode ultrasound sensing for human–machine interface,” IEEE Sensors Journal, vol. 19, no. 13, pp. 5219–5228, 2019.
- X. Yang, X. Sun, D. Zhou, Y. Li, and H. Liu, “Towards wearable a-mode ultrasound sensing for real-time finger motion recognition,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 26, no. 6, pp. 1199–1208, 2018.
- L. Guo, Z. Lu, L. Yao, and S. Cai, “A gesture recognition strategy based on a-mode ultrasound for identifying known and unknown gestures,” IEEE Sensors Journal, vol. 22, no. 11, pp. 10 730–10 739, 2022.
- Z. Yin, H. Chen, X. Yang, Y. Liu, N. Zhang, J. Meng, and H. Liu, “A wearable ultrasound interface for prosthetic hand control,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 11, pp. 5384–5393, 2022.
- A. Fougner, y. Stavdahl, P. J. Kyberd, Y. G. Losier, and P. A. Parker, “Control of upper limb prostheses: Terminology and proportional myoelectric control—a review,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 20, no. 5, pp. 663–677, 2012.
- X. Chen, Y.-P. Zheng, J.-Y. Guo, and J. Shi, “Sonomyography (smg) control for powered prosthetic hand: a study with normal subjects.” Ultrasound in Medicine and Biology, 2010.
- X. Yang, J. Yan, Z. Chen, H. Ding, and H. Liu, “A proportional pattern recognition control scheme for wearable a-mode ultrasound sensing,” IEEE Transactions on Industrial Electronics, vol. 67, no. 1, pp. 800–808, 2020.
- P. M. Fitts, “The information capacity of the human motor system in controlling the amplitude of movement. 1954.” Journal of experimental psychology. General, vol. 121, no. 3, pp. 262–269, sep 1992.
- B. Hoff and M. A. Arbib, “Models of trajectory formation and temporal interaction of reach and grasp,” Journal of Motor Behavior, vol. 25, no. 3, pp. 175–192, 1993.
- M. Mon-Williams and J. R. Tresilian, “A simple rule of thumb for elegant prehension,” Current Biology, vol. 11, no. 13, pp. 1058–1061, 2001.
- E. Todorov, “Optimality principles in sensorimotor control,” Nature Neuroscience, vol. 7, no. 9, pp. 907–915, 2004.