Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Machine Learning Techniques for Sensor-based Human Activity Recognition with Data Heterogeneity -- A Review (2403.15422v1)

Published 12 Mar 2024 in eess.SP, cs.AI, cs.HC, and cs.LG

Abstract: Sensor-based Human Activity Recognition (HAR) is crucial in ubiquitous computing, analysing behaviours through multi-dimensional observations. Despite research progress, HAR confronts challenges, particularly in data distribution assumptions. Most studies often assume uniform data distributions across datasets, contrasting with the varied nature of practical sensor data in human activities. Addressing data heterogeneity issues can improve performance, reduce computational costs, and aid in developing personalized, adaptive models with less annotated data. This review investigates how machine learning addresses data heterogeneity in HAR, by categorizing data heterogeneity types, applying corresponding suitable machine learning methods, summarizing available datasets, and discussing future challenges.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (113)
  1. Adaptive mobile activity recognition system with evolving data streams. Neurocomputing 150 (2015), 304–317.
  2. Activity recognition with evolving data streams: A review. ACM Computing Surveys (CSUR) 51, 4 (2018), 1–36.
  3. Zero-shot human activity recognition using non-visual sensors. Sensors 20, 3 (2020), 825.
  4. A robust deep learning approach for position-independent smartphone-based human activity recognition. Sensors 18, 11 (2018), 3726.
  5. Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognition 43, 10 (2010), 3605–3620.
  6. A public domain dataset for human activity recognition using smartphones. In Proceedings of the 21th international European symposium on artificial neural networks, computational intelligence and machine learning. 437–442.
  7. mHealthDroid: a novel framework for agile development of mobile health applications. In International workshop on ambient assisted living. Springer, 91–98.
  8. Swimming style recognition and lap counting using a smartwatch and deep learning. In Proceedings of the 2019 ACM International Symposium on Wearable Computers. 23–31.
  9. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 46, 3 (2014), 1–33.
  10. Heterogeneous non-local fusion for multimodal activity recognition. In Proceedings of the 2020 International Conference on Multimedia Retrieval. 63–72.
  11. Personalization and user verification in wearable systems using biometric walking patterns. Personal and Ubiquitous Computing 16, 5 (2012), 563–580.
  12. A systematic study of unsupervised domain adaptation for robust human-activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1–30.
  13. The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters 34, 15 (2013), 2033–2042.
  14. UTD-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. In 2015 IEEE International conference on image processing (ICIP). IEEE, 168–172.
  15. Distributionally robust semi-supervised learning for people-centric sensing. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 3321–3328.
  16. Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. ACM Computing Surveys (CSUR) 54, 4 (2021), 1–40.
  17. Activity recognition using transfer learning. Sens. Mater 29, 7 (2017), 897–904.
  18. Fedhealth: A federated transfer learning framework for wearable healthcare. IEEE Intelligent Systems 35, 4 (2020), 83–93.
  19. Cross-position activity recognition with stratified transfer learning. Pervasive and Mobile Computing 57 (2019), 1–13.
  20. Nuactiv: Recognizing unseen new activities using semantic attribute-based learning. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. 361–374.
  21. Sensor data acquisition and multimodal sensor fusion for human activity recognition using deep learning. Sensors 19, 7 (2019), 1716.
  22. Utilizing consumer-grade wearable sensors for unobtrusive rehabilitation outcome prediction. In 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI). IEEE, 1–4.
  23. Transfer learning for activity recognition: A survey. Knowledge and information systems 36, 3 (2013), 537–556.
  24. CASAS: A smart home in a box. Computer 46, 7 (2012), 62–69.
  25. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44, 7 (2021), 3366–3385.
  26. Empirical study and improvement on deep transfer learning for human activity recognition. Sensors 19, 1 (2018), 57.
  27. Robust human activity recognition using multimodal feature-level fusion. IEEE Access 7 (2019), 60736–60751.
  28. Evaluation of federated learning aggregation algorithms: application to human activity recognition. In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers. 638–643.
  29. Robust domain generalisation by enforcing distribution invariance. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16). AAAI Press, 1455–1461.
  30. Ramin Fallahzadeh and Hassan Ghasemzadeh. 2017. Personalization without user interruption: Boosting activity recognition in new subjects using unlabeled data. In Proceedings of the 8th International Conference on Cyber-Physical Systems. 293–302.
  31. Siwei Feng and Marco F Duarte. 2019. Few-shot learning-based human activity recognition. Expert Systems with Applications 138 (2019), 112782.
  32. On the personalization of classification models for human activity recognition. IEEE Access 8 (2020), 32066–32079.
  33. Kyle Dillon Feuz and Diane J Cook. 2014. Heterogeneous transfer learning for activity recognition using heuristic search techniques. International Journal of Pervasive Computing and Communications (2014).
  34. Kyle D Feuz and Diane J Cook. 2015. Transfer learning across feature-rich heterogeneous feature spaces via feature-space remapping (FSR). ACM transactions on intelligent systems and technology (TIST) 6, 1 (2015), 1–27.
  35. Kyle D Feuz and Diane J Cook. 2017. Collegial activity learning between heterogeneous sensors. Knowledge and information systems 53, 2 (2017), 337–364.
  36. Personalized human activity recognition based on integrated wearable sensor and transfer learning. Sensors 21, 3 (2021), 885.
  37. A survey on concept drift adaptation. ACM computing surveys (CSUR) 46, 4 (2014), 1–37.
  38. Multi-view stacking for activity recognition with sound and accelerometer data. Information Fusion 40 (2018), 45–56.
  39. Recent advances in open set recognition: A survey. IEEE transactions on pattern analysis and machine intelligence 43, 10 (2020), 3614–3631.
  40. Deep neural network based human activity recognition for the order picking process. In Proceedings of the 4th international Workshop on Sensor-based Activity Recognition and Interaction. 1–6.
  41. Interactive and Interpretable Online Human Activity Recognition. In 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops). IEEE, 109–111.
  42. Toward personalized activity recognition systems with a semipopulation approach. IEEE Transactions on Human-Machine Systems 46, 1 (2015), 101–112.
  43. A novel random forests based class incremental learning method for activity recognition. Pattern Recognition 78 (2018), 277–290.
  44. Discovery of activity patterns using topic models. In Proceedings of the 10th international conference on Ubiquitous computing. 10–19.
  45. Towards environment independent device free human activity recognition. In Proceedings of the 24th annual international conference on mobile computing and networking. 289–304.
  46. Mmact: A large-scale dataset for cross modal human action understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8658–8667.
  47. Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newsletter 12, 2 (2011), 74–82.
  48. Learning and managing context enriched behavior patterns in smart homes. Future Generation Computer Systems 91 (2019), 191–205.
  49. Athanasios Lentzas and Dimitris Vrakas. 2020. Non-intrusive human activity recognition and abnormal behavior detection on elderly people: A review. Artificial Intelligence Review 53, 3 (2020), 1975–2021.
  50. Deep transfer learning for time series data based on sensor modality classification. Sensors 20, 15 (2020), 4271.
  51. Fusing wearable and remote sensing data streams by fast incremental learning with swarm decision table for human activity recognition. Information Fusion 60 (2020), 41–64.
  52. NOHAR-NOvelty discrete data stream for Human Activity Recognition based on smartphones with inertial sensors. Expert Systems with Applications 166 (2021), 114093.
  53. Experiencing SAX: a novel symbolic representation of time series. Data Mining and knowledge discovery 15, 2 (2007), 107–144.
  54. Sensor-Based Human Activity and Behavior Research: Where Advanced Sensing and Recognition Technologies Meet. , 125 pages.
  55. Motion Units: Generalized sequence modeling of human activities for sensor-based activity recognition. In 2021 29th European signal processing conference (EUSIPCO). IEEE, 1506–1510.
  56. Hui Liu and Tanja Schultz. 2022. How Long Are Various Types of Daily Activities? Statistical Analysis of a Multimodal Wearable Sensor-based Human Activity Dataset.. In HEALTHINF. 680–688.
  57. Local and global alignments for generalizable sensor-based human activity recognition. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3833–3837.
  58. Protecting sensory data against sensitive inferences. In Proceedings of the 1st Workshop on Privacy by Design in Distributed Systems. 1–6.
  59. Addressing concept-evolution in concept-drifting data streams. In 2010 IEEE International Conference on Data Mining. IEEE, 929–934.
  60. Incremental real-time personalization in human activity recognition using domain adaptive batch normalization. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 4 (2020), 1–20.
  61. Towards online and personalized daily activity recognition, habit modeling, and anomaly detection for the solitary elderly through unobtrusive sensing. Multimedia Tools and Applications 76, 8 (2017), 10779–10799.
  62. Unimib shar: A dataset for human activity recognition using acceleration data from smartphones. Applied Sciences 7, 10 (2017), 1101.
  63. Attributes’ importance for zero-shot pose-classification based on wearable sensors. Sensors 18, 8 (2018), 2485.
  64. FedCLAR: Federated Clustering for Personalized Sensor-Based Human Activity Recognition. In 2022 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 227–236.
  65. Activity Recognition Using Biomechanical Model Based Pose Estimation.. In EuroSSC. Springer, 42–55.
  66. Attila Reiss and Didier Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In 2012 16th international symposium on wearable computers. IEEE, 108–109.
  67. A survey on ontologies for human behavior recognition. ACM Computing Surveys (CSUR) 46, 4 (2014), 1–33.
  68. The adarc pattern analysis architecture for adaptive human activity recognition systems. Journal of Ambient Intelligence and Humanized Computing 4, 2 (2013), 169–186.
  69. Seyed Ali Rokni and Hassan Ghasemzadeh. 2018. Autonomous training of activity recognition algorithms in mobile sensors: A transfer learning approach in context-invariant views. IEEE Transactions on Mobile Computing 17, 8 (2018), 1764–1777.
  70. Personalized human activity recognition using convolutional neural networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
  71. Personalized human activity recognition using wearables: A manifold learning-based knowledge transfer. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 1193–1196.
  72. Omer Sagi and Lior Rokach. 2018. Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8, 4 (2018), e1249.
  73. Unsupervised domain adaptation in activity recognition: A GAN-based approach. IEEE Access 9 (2021), 19421–19438.
  74. Environment-robust device-free human activity recognition with channel-state-information enhancement and one-shot learning. IEEE Transactions on Mobile Computing 21, 2 (2020), 540–554.
  75. Fusion of smartphone motion sensors for physical activity recognition. Sensors 14, 6 (2014), 10146–10176.
  76. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM conference on embedded networked sensor systems. 127–140.
  77. Timo Sztyler and Heiner Stuckenschmidt. 2016. On-body localization of wearable devices: An investigation of position-aware activity recognition. In 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1–9.
  78. Position-aware activity recognition with wearable devices. Pervasive and mobile computing 38 (2017), 281–295.
  79. Transitive transfer learning. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1155–1164.
  80. An activity monitoring system for elderly care using generative and discriminative models. Personal and ubiquitous computing 14 (2010), 489–498.
  81. Human activity recognition from wireless sensor network data: Benchmark and software. In Activity recognition in pervasive intelligent environments. Springer, 165–186.
  82. The mobiact dataset: Recognition of activities of daily living using smartphones. In International Conference on Information and Communication Technologies for Ageing Well and e-Health, Vol. 2. SciTePress, 143–151.
  83. Personalization in mobile activity recognition system using K-medoids clustering algorithm. International Journal of Distributed Sensor Networks (2013).
  84. Deep learning for sensor-based activity recognition: A survey. Pattern recognition letters 119 (2019), 3–11.
  85. Stratified transfer learning for cross-domain activity recognition. In 2018 IEEE international conference on pervasive computing and communications (PerCom). IEEE, 1–10.
  86. Device-free wireless sensing: Challenges, opportunities, and applications. IEEE network 32, 2 (2018), 132–137.
  87. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering (2022).
  88. Cross-scenario device-free activity recognition based on deep adversarial networks. IEEE Transactions on Vehicular Technology 69, 5 (2020), 5416–5425.
  89. Deep transfer learning for cross-domain activity recognition. In proceedings of the 3rd International Conference on Crowd Science and Engineering. 1–8.
  90. Lidong Wang. 2017. Heterogeneous data and big data analytics. Automatic Control and Information Sciences 3, 1 (2017), 8–15.
  91. Zero-shot human activity recognition via nonlinear compatibility based method. In Proceedings of the International Conference on Web Intelligence. 322–330.
  92. A survey of zero-shot learning: Settings, methods, and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 10, 2 (2019), 1–37.
  93. Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur) 53, 3 (2020), 1–34.
  94. Kernel fusion based extreme learning machine for cross-location activity recognition. Information Fusion 37 (2017), 1–9.
  95. A knowledge-light approach to personalised and open-ended human activity recognition. Knowledge-based systems 192 (2020), 105651.
  96. Enabling edge devices that learn from each other: Cross modal training for activity recognition. In Proceedings of the 1st International Workshop on Edge Systems, Analytics and Networking. 37–42.
  97. A survey on multi-view learning. arXiv preprint arXiv:1304.5634 (2013).
  98. Deepfusion: A deep learning framework for the fusion of heterogeneous sensory data. In Proceedings of the Twentieth ACM International Symposium on Mobile Ad Hoc Networking and Computing. 151–160.
  99. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 10, 2 (2019), 1–19.
  100. Rong Yang and Baowei Wang. 2016. PACP: a position-independent activity recognition method using smartphone sensors. Information 7, 4 (2016), 72.
  101. Open-set human activity recognition based on micro-Doppler signatures. Pattern Recognition 85 (2019), 60–69.
  102. Juan Ye. 2018. SLearn: Shared learning human activity labels across multiple datasets. In 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1–10.
  103. Juan Ye and Elise Callus. 2020. Evolving models for incrementally learning emerging activities. Journal of Ambient Intelligence and Smart Environments 12, 4 (2020), 313–325.
  104. Continual activity recognition with generative adversarial networks. ACM Transactions on Internet of Things 2, 2 (2021), 1–25.
  105. USMART: An unsupervised semantic mining activity recognition technique. ACM Transactions on Interactive Intelligent Systems (TiiS) 4, 4 (2014), 1–27.
  106. Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection. In European Conference on Wireless Sensor Networks. Springer, 17–33.
  107. Mi Zhang and Alexander A Sawchuk. 2012. USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the 2012 ACM conference on ubiquitous computing. 1036–1043.
  108. Shibo Zhang and Nabil Alshurafa. 2020. Deep generative cross-modal on-body accelerometer data synthesis from videos. In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers. 223–227.
  109. Yu Zhang and Qiang Yang. 2021. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering (2021).
  110. A user-adaptive algorithm for activity recognition based on k-means clustering, local outlier factor, and multivariate gaussian distribution. Sensors 18, 6 (2018), 1850.
  111. Zero-effort cross-domain gesture recognition with Wi-Fi. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services. 313–325.
  112. Phone Position Independent Recognition of Fine Walking Modes with Phone Sensors. Adhoc & Sensor Wireless Networks 46 (2020).
  113. A comprehensive survey on transfer learning. Proc. IEEE 109, 1 (2020), 43–76.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now