Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Millimeter Wave Radar-based Human Activity Recognition for Healthcare Monitoring Robot (2405.01882v1)

Published 3 May 2024 in cs.RO, cs.AI, and eess.SP

Abstract: Healthcare monitoring is crucial, especially for the daily care of elderly individuals living alone. It can detect dangerous occurrences, such as falls, and provide timely alerts to save lives. Non-invasive millimeter wave (mmWave) radar-based healthcare monitoring systems using advanced human activity recognition (HAR) models have recently gained significant attention. However, they encounter challenges in handling sparse point clouds, achieving real-time continuous classification, and coping with limited monitoring ranges when statically mounted. To overcome these limitations, we propose RobHAR, a movable robot-mounted mmWave radar system with lightweight deep neural networks for real-time monitoring of human activities. Specifically, we first propose a sparse point cloud-based global embedding to learn the features of point clouds using the light-PointNet (LPN) backbone. Then, we learn the temporal pattern with a bidirectional lightweight LSTM model (BiLiLSTM). In addition, we implement a transition optimization strategy, integrating the Hidden Markov Model (HMM) with Connectionist Temporal Classification (CTC) to improve the accuracy and robustness of the continuous HAR. Our experiments on three datasets indicate that our method significantly outperforms the previous studies in both discrete and continuous HAR tasks. Finally, we deploy our system on a movable robot-mounted edge computing platform, achieving flexible healthcare monitoring in real-world scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Z. Wang, P. Sokliep, C. Xu, J. Huang, L. Lu, and Z. Shi, “Indoor position algorithm based on the fusion of wifi and image,” in 2019 Eleventh International Conference on Advanced Computational Intelligence (ICACI).   IEEE, 2019, pp. 212–216.
  2. H. H. Ali, J. R. Naif, and W. R. Humood, “A new smart home intruder detection system based on deep learning,” Al-Mustansiriyah Journal of Science, vol. 34, no. 2, pp. 60–69, 2023.
  3. Z. Gu, X. Yang, W. Jia, C. Xu, P. Yu, X. He, H. Chen, and Y. Lin, “Strokepeo: Construction of a clinical ontology for physical examination of stroke,” in 2022 9th International Conference on Digital Home (ICDH).   IEEE, 2022, pp. 218–223.
  4. Y.-f. Zhang, J. Zheng, L. Li, N. Liu, W. Jia, X. Fan, C. Xu, and X. He, “Rethinking feature aggregation for deep rgb-d salient object detection,” Neurocomputing, vol. 423, pp. 463–473, 2021.
  5. Z. Gu, X. He, P. Yu, W. Jia, X. Yang, G. Peng, P. Hu, S. Chen, H. Chen, and Y. Lin, “Automatic quantitative stroke severity assessment based on chinese clinical named entity recognition with domain-adaptive pre-trained large language model,” Artificial Intelligence in Medicine, p. 102822, 2024.
  6. H. Arab, I. Ghaffari, L. Chioukh, S. O. Tatu, and S. Dufour, “A convolutional neural network for human motion recognition and classification using a millimeter-wave doppler radar,” IEEE Sensors Journal, vol. 22, no. 5, pp. 4494–4502, 2022.
  7. F. Jin, A. Sengupta, and S. Cao, “mmfall: Fall detection using 4-d mmwave radar and a hybrid variational rnn autoencoder,” IEEE Transactions on Automation Science and Engineering, vol. 19, no. 2, pp. 1245–1257, 2022.
  8. K. Rambabu, V. Haritha, S. N. Srinivas, and P. S. Reddy, “Iot based human intrusion detection system using labview,” International Journal of Advanced Science and Technology, vol. 127, no. 1, pp. 162–166, 2019.
  9. U. M. Khan, Z. Kabir, S. A. Hassan, and S. H. Ahmed, “A deep learning framework using passive wifi sensing for respiration monitoring,” in GLOBECOM 2017-2017 IEEE Global Communications Conference.   IEEE, 2017, pp. 1–6.
  10. P. Zhao, C. X. Lu, B. Wang, C. Chen, L. Xie, M. Wang, N. Trigoni, and A. Markham, “Heart rate sensing with a robot mounted mmwave radar,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 2812–2818.
  11. J. Hannink, T. Kautz, C. F. Pasluosta, K.-G. Gaßmann, J. Klucken, and B. M. Eskofier, “Sensor-based gait parameter extraction with deep convolutional neural networks,” IEEE journal of biomedical and health informatics, vol. 21, no. 1, pp. 85–93, 2016.
  12. Y.-L. Hsu, S.-C. Yang, H.-C. Chang, and H.-C. Lai, “Human daily and sport activity recognition using a wearable inertial sensor network,” IEEE Access, vol. 6, pp. 31 715–31 728, 2018.
  13. B. Reily, F. Han, L. E. Parker, and H. Zhang, “Skeleton-based bio-inspired human activity prediction for real-time human–robot interaction,” Autonomous Robots, vol. 42, no. 6, pp. 1281–1298, 2018.
  14. T. Singh and D. K. Vishwakarma, “Human activity recognition in video benchmarks: A survey,” in Advances in Signal Processing and Communication.   Springer, 2019, pp. 247–259.
  15. S. Z. Gurbuz and M. G. Amin, “Radar-based human-motion recognition with deep learning: Promising applications for indoor monitoring,” IEEE Signal Processing Magazine, vol. 36, no. 4, pp. 16–28, 2019.
  16. O. D. Lara and M. A. Labrador, “A survey on human activity recognition using wearable sensors,” IEEE communications surveys & tutorials, vol. 15, no. 3, pp. 1192–1209, 2012.
  17. X. Li, Y. He, and X. Jing, “A survey of deep learning-based human activity recognition in radar,” Remote Sensing, vol. 11, no. 9, p. 1068, 2019.
  18. J. Zhang, R. Xi, Y. He, Y. Sun, X. Guo, W. Wang, X. Na, Y. Liu, Z. Shi, and T. Gu, “A survey of mmwave-based human sensing: Technology, platforms and applications,” IEEE Communications Surveys & Tutorials, vol. 25, no. 4, pp. 2052–2087, 2023.
  19. P. Zhao, C. X. Lu, J. Wang, C. Chen, W. Wang, N. Trigoni, and A. Markham, “mid: Tracking and identifying people with millimeter wave radar,” in 2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS).   IEEE, 2019, pp. 33–40.
  20. M. G. Amin, Y. D. Zhang, F. Ahmad, and K. D. Ho, “Radar signal processing for elderly fall detection: The future for in-home monitoring,” IEEE Signal Processing Magazine, vol. 33, no. 2, pp. 71–80, 2016.
  21. A. D. Singh, S. S. Sandha, L. Garcia, and M. Srivastava, “Radhar: Human activity recognition from point clouds generated through a millimeter-wave radar,” in Proceedings of the 3rd ACM Workshop on Millimeter-wave Networks and Sensing Systems, 2019, pp. 51–56.
  22. R. Zhang and S. Cao, “Real-time human motion behavior detection via cnn using mmwave radar,” IEEE Sensors Letters, vol. 3, no. 2, pp. 1–4, 2018.
  23. X. Han, Y. Tang, Z. Wang, and X. Li, “Mamba3d: Enhancing local features for 3d point cloud analysis via state space model,” arXiv preprint arXiv:2404.14966, 2024.
  24. N. Elsayed, Z. ElSayed, and A. S. Maida, “Litelstm architecture for deep recurrent neural networks,” in 2022 IEEE International Symposium on Circuits and Systems (ISCAS).   IEEE, 2022, pp. 1304–1308.
  25. S. R. Eddy, “Hidden markov models,” Current opinion in structural biology, vol. 6, no. 3, pp. 361–365, 1996.
  26. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 369–376.
  27. Y. Zhou and O. Tuzel, “Voxelnet: End-to-end learning for point cloud based 3d object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4490–4499.
  28. C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 652–660.
  29. C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” in Advances in neural information processing systems, 2017, pp. 5099–5108.
  30. Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen, “Pointcnn: Convolution on x-transformed points,” in Advances in Neural Information Processing Systems, 2018, pp. 820–830.
  31. Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM Transactions on Graphics (TOG), vol. 38, no. 5, pp. 1–12, 2019.
  32. J. Liu, R. Yu, Y. Wang, Y. Zheng, T. Deng, W. Ye, and H. Wang, “Point mamba: A novel point cloud backbone based on state space model with octree-based ordering strategy,” arXiv preprint arXiv:2403.06467, 2024.
  33. S. Saydam, C. Xu, B. Li, B. Topal, and S. Saydam, “Feature sampling and balancing for detecting rock bolts from the lidar point clouds,” in ISRM Congress.   ISRM, 2023, pp. ISRM–15CONGRESS.
  34. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  35. H. Zhao, L. Jiang, J. Jia, P. H. Torr, and V. Koltun, “Point transformer,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 16 259–16 268.
  36. X. Wu, Y. Lao, L. Jiang, X. Liu, and H. Zhao, “Point transformer v2: Grouped vector attention and partition-based pooling,” Advances in Neural Information Processing Systems, vol. 35, pp. 33 330–33 342, 2022.
  37. X. Wu, L. Jiang, P.-S. Wang, Z. Liu, X. Liu, Y. Qiao, W. Ouyang, T. He, and H. Zhao, “Point transformer v3: Simpler, faster, stronger,” arXiv preprint arXiv:2312.10035, 2023.
  38. A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752, 2023.
  39. C. Xu, R. Wang, S. Lin, X. Luo, B. Zhao, L. Shao, and M. Hu, “Lecture2note: Automatic generation of lecture notes from slide-based educational videos,” in 2019 IEEE International Conference on Multimedia and Expo (ICME).   IEEE, 2019, pp. 898–903.
  40. C. Xu, W. Jia, R. Wang, X. Luo, and X. He, “Morphtext: Deep morphology regularized accurate arbitrary-shape scene text detection,” IEEE Trans. Multimedia, 2022.
  41. C. Xu, H. Fu, L. Ma, W. Jia, C. Zhang, F. Xia, X. Ai, B. Li, and W. Zhang, “Seeing text in the dark: Algorithm and benchmark,” arXiv preprint arXiv:2404.08965, 2024.
  42. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, vol. 35, no. 12.   AAAI Press, 2021, pp. 11 106–11 115.
  43. A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” in Proceedings of the AAAI conference on artificial intelligence, vol. 37, no. 9, 2023, pp. 11 121–11 128.
  44. M. Liu, A. Zeng, M. Chen, Z. Xu, Q. Lai, L. Ma, and Q. Xu, “Scinet: Time series modeling and forecasting with sample convolution and interaction,” Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), 2022, 2022.
  45. T.-y. Xie, C.-d. Zhang, Q.-l. Zhou, Z.-q. Tian, S. Liu, and H.-j. Guo, “Tsc prediction and dynamic control of bof steelmaking with state-of-the-art machine learning and deep learning methods,” Journal of Iron and Steel Research International, pp. 1–21, 2024.
  46. C. Ding, H. Hong, Y. Zou, H. Chu, X. Zhu, F. Fioranelli, J. Le Kernec, and C. Li, “Continuous human motion recognition with a dynamic range-doppler trajectory method based on fmcw radar,” IEEE Transactions on Geoscience and Remote Sensing, 2019.
  47. C. Coppola, S. Cosar, D. R. Faria, and N. Bellotto, “Social activity recognition on continuous rgb-d video sequences,” International Journal of Social Robotics, pp. 1–15, 2019.
  48. C. Xu, W. Jia, T. Cui, R. Wang, Y.-f. Zhang, and X. He, “Arbitrary-shape scene text detection via visual-relational rectification and contour approximation,” IEEE Trans. Multimedia, 2022.
  49. C. Xu, W. Jia, R. Wang, X. He, B. Zhao, and Y. Zhang, “Semantic navigation of powerpoint-based lecture video for autonote generation,” IEEE Transactions on Learning Technologies, vol. 16, no. 1, pp. 1–17, 2022.
  50. Z. Zhang, Z. Tian, and M. Zhou, “Latern: Dynamic continuous hand gesture recognition using fmcw radar sensor,” IEEE Sensors Journal, vol. 18, no. 8, pp. 3278–3289, 2018.
  51. L. Morrison, G. Forbes, S. Massie, S. Craw, L. Fraser, and G. Hamilton, “Representing temporal dependencies in human activity recognition.”   CEUR Workshop Proceedings, 2020.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now