Papers
Topics
Authors
Recent
Search
2000 character limit reached

rTsfNet: a DNN model with Multi-head 3D Rotation and Time Series Feature Extraction for IMU-based Human Activity Recognition

Published 30 Oct 2023 in cs.HC and cs.LG | (2310.19283v3)

Abstract: Although many deep learning (DL) algorithms have been proposed for the IMU-based HAR domain, traditional machine learning that utilizes handcrafted time series features (TSFs) still often performs well. It is not rare that combinations among DL and TSFs show better accuracy than DL-only approaches. However, there is a problem with time series features in IMU-based HAR. The amount of derived features can vary greatly depending on the method used to select the 3D basis. Fortunately, DL's strengths include capturing the features of input data and adaptively deriving parameters. Thus, as a new DNN model for IMU-based human activity recognition (HAR), this paper proposes rTsfNet, a DNN model with Multi-head 3D Rotation and Time Series Feature Extraction. rTsfNet automatically selects 3D bases from which features should be derived by extracting 3D rotation parameters within the DNN. Then, time series features (TSFs), based on many researchers' wisdom, are derived to achieve HAR using MLP. Although rTsfNet is a model that does not use CNN, it achieved higher accuracy than existing models under well-managed benchmark conditions and multiple datasets: UCI HAR, PAMAP2, Daphnet, and OPPORTUNITY, all of which target different activities.

Authors (1)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine. In Ambient Assisted Living and Home Care, José Bravo, Ramón Hervás, and Marcela Rodríguez (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 216–223.
  2. A public domain dataset for human activity recognition using smartphones.. In Esann, Vol. 3. 3.
  3. Wearable Assistant for Parkinson’s Disease Patients With the Freezing of Gait Symptom. IEEE Transactions on Information Technology in Biomedicine 14, 2 (2010), 436–446. https://doi.org/10.1109/TITB.2009.2036165
  4. CID: an efficient complexity-invariant distance for time series. Data Mining and Knowledge Discovery 28 (2014), 634–669.
  5. IoT Wearable Sensor and Deep Learning: An Integrated Approach for Personalized Human Activity Recognition in a Smart Home Environment. IEEE Internet of Things Journal 6, 5 (2019), 8553–8562. https://doi.org/10.1109/JIOT.2019.2920283
  6. Human activity recognition in IoHT applications using Arithmetic Optimization Algorithm and deep learning. Measurement 199 (2022), 111445. https://doi.org/10.1016/j.measurement.2022.111445
  7. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (2002), 182–197. https://doi.org/10.1109/4235.996017
  8. DSmT-Based Fusion Strategy for Human Activity Recognition in Body Sensor Networks. IEEE Transactions on Industrial Informatics (Feb. 2020). https://doi.org/10.1109/TII.2020.2976812
  9. Assessing the State of Self-Supervised Human Activity Recognition Using Wearables. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 3, Article 116 (sep 2022), 47 pages. https://doi.org/10.1145/3550299
  10. Weakly Supervised Human Activity Recognition From Wearable Sensors by Recurrent Attention Learning. IEEE Sensors Journal 19, 6 (2019), 2287–2297. https://doi.org/10.1109/JSEN.2018.2885796
  11. Train longer, generalize better: closing the generalization gap in large batch training of neural networks. In Advances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2017/file/a5e0ff62be0b08456fc7f1e88812af3d-Paper.pdf
  12. Andrey Ignatov. 2018. Real-time human activity recognition from accelerometer data using Convolutional Neural Networks. Applied Soft Computing 62 (2018), 915–922. https://doi.org/10.1016/j.asoc.2017.09.027
  13. Wenchao Jiang and Zhaozheng Yin. 2015. Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks. In Proceedings of the 23rd ACM International Conference on Multimedia (Brisbane, Australia) (MM ’15). Association for Computing Machinery, New York, NY, USA, 1307–1310. https://doi.org/10.1145/2733373.2806333
  14. Augmented Adversarial Learning for Human Activity Recognition with Partial Sensor Sets. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 3, Article 122 (sep 2022), 30 pages. https://doi.org/10.1145/3550285
  15. Zanobya N. Khan and Jamil Ahmad. 2021. Attention induced multi-head convolutional neural network for human activity recognition. Applied Soft Computing 110 (2021), 107671. https://doi.org/10.1016/j.asoc.2021.107671
  16. Hidden Markov Model Ensemble for Activity Recognition Using Tri-Axis Accelerometer. In 2015 IEEE International Conference on Systems, Man, and Cybernetics. 3036–3041. https://doi.org/10.1109/SMC.2015.528
  17. Activity Recognition Using Cell Phone Accelerometers. SIGKDD Explor. Newsl. 12, 2 (mar 2011), 74–82. https://doi.org/10.1145/1964897.1964918
  18. Bi-LSTM Network for Multimodal Continuous Human Activity Recognition and Fall Detection. IEEE Sensors Journal 20, 3 (2020), 1191–1201. https://doi.org/10.1109/JSEN.2019.2946095
  19. Unsupervised Feature Learning for Human Activity Recognition Using Smartphone Sensors. In Mining Intelligence and Knowledge Exploration, Rajendra Prasath, Philip O’Reilly, and T. Kathirvalavakumar (Eds.). Springer International Publishing, Cham, 99–107.
  20. A Multichannel CNN-GRU Model for Human Activity Recognition. IEEE Access 10 (2022), 66797–66810. https://doi.org/10.1109/ACCESS.2022.3185112
  21. Ronald Mutegeki and Dong Seog Han. 2020. A CNN-LSTM Approach to Human Activity Recognition. In 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC). 362–366. https://doi.org/10.1109/ICAIIC48513.2020.9065078
  22. Attila Reiss and Didier Stricker. 2012. Introducing a New Benchmarked Dataset for Activity Monitoring. In 2012 16th International Symposium on Wearable Computers. 108–109. https://doi.org/10.1109/ISWC.2012.13
  23. Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh International Conference on Networked Sensing Systems (INSS). 233–240. https://doi.org/10.1109/INSS.2010.5573462
  24. iSPLInception: An Inception-ResNet Deep Learning Architecture for Human Activity Recognition. IEEE Access 9 (2021), 68985–69001. https://doi.org/10.1109/ACCESS.2021.3078184
  25. Charissa Ann Ronao and Sung-Bae Cho. 2014. Human activity recognition using smartphone sensors with two-stage continuous hidden Markov models. In 2014 10th International Conference on Natural Computation (ICNC). 681–686. https://doi.org/10.1109/ICNC.2014.6975918
  26. Charissa Ann Ronao and Sung-Bae Cho. 2015. Deep Convolutional Neural Networks for Human Activity Recognition with Smartphone Sensors. In Neural Information Processing, Sabri Arik, Tingwen Huang, Weng Kin Lai, and Qingshan Liu (Eds.). Springer International Publishing, Cham, 46–53.
  27. Charissa Ann Ronao and Sung-Bae Cho. 2016. Human activity recognition with smartphone sensors using deep learning neural networks. Expert Systems with Applications 59 (2016), 235–244. https://doi.org/10.1016/j.eswa.2016.04.032
  28. Charissa Ann Ronao and Sung-Bae Cho. 2017. Recognizing human activities from smartphone sensors using hierarchical continuous hidden Markov models. International Journal of Distributed Sensor Networks 13, 1 (2017), 1550147716683687. https://doi.org/10.1177/1550147716683687 arXiv:https://doi.org/10.1177/1550147716683687
  29. Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition. In 2015 IEEE Symposium Series on Computational Intelligence. 1399–1406. https://doi.org/10.1109/SSCI.2015.199
  30. Going Deeper With Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  31. Timo Sztyler and Heiner Stuckenschmidt. 2016. On-body localization of wearable devices: An investigation of position-aware activity recognition. In 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom). 1–9. https://doi.org/10.1109/PERCOM.2016.7456521
  32. Predicting Performance Improvement of Human Activity Recognition Model by Additional Data Collection. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 3, Article 142 (sep 2022), 33 pages. https://doi.org/10.1145/3550319
  33. Nguyen Thi Hoai Thu and Dong Seog Han. 2020. Utilization of Postural Transitions in Sensor-based Human Activity Recognition. In 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC). 177–181. https://doi.org/10.1109/ICAIIC48513.2020.9065250
  34. Human Action Recognition Using Deep Learning Methods on Limited Sensory Data. IEEE Sensors Journal 20, 6 (2020), 3101–3112. https://doi.org/10.1109/JSEN.2019.2956901
  35. Mcfly: Automated deep learning on time series. SoftwareX 12 (2020), 100548. https://doi.org/10.1016/j.softx.2020.100548
  36. Attention-Based Convolutional Neural Network for Weakly Labeled Human Activities’ Recognition With Wearable Sensors. IEEE Sensors Journal 19, 17 (2019), 7598–7604. https://doi.org/10.1109/JSEN.2019.2917225
  37. LSTM-CNN Architecture for Human Activity Recognition. IEEE Access 8 (2020), 56855–56866. https://doi.org/10.1109/ACCESS.2020.2982225
  38. InnoHAR: A Deep Neural Network for Complex Human Activity Recognition. IEEE Access 7 (2019), 9893–9902. https://doi.org/10.1109/ACCESS.2018.2890675
  39. Deep residual bidir-LSTM for human activity recognition using wearable sensors. Mathematical Problems in Engineering 2018 (2018), 1–13.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free