Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unleashing the Power of Shared Label Structures for Human Activity Recognition (2301.03462v2)

Published 1 Jan 2023 in cs.LG, cs.AI, and eess.SP

Abstract: Current human activity recognition (HAR) techniques regard activity labels as integer class IDs without explicitly modeling the semantics of class labels. We observe that different activity names often have shared structures. For example, "open door" and "open fridge" both have "open" as the action; "kicking soccer ball" and "playing tennis ball" both have "ball" as the object. Such shared structures in label names can be translated to the similarity in sensory data and modeling common structures would help uncover knowledge across different activities, especially for activities with limited samples. In this paper, we propose SHARE, a HAR framework that takes into account shared structures of label names for different activities. To exploit the shared structures, SHARE comprises an encoder for extracting features from input sensory time series and a decoder for generating label names as a token sequence. We also propose three label augmentation techniques to help the model more effectively capture semantic structures across activities, including a basic token-level augmentation, and two enhanced embedding-level and sequence-level augmentations utilizing the capabilities of pre-trained models. SHARE outperforms state-of-the-art HAR models in extensive experiments on seven HAR benchmark datasets. We also evaluate in few-shot learning and label imbalance settings and observe even more significant performance gap.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. A public domain dataset for human activity recognition using smartphones.. In Esann, Vol. 3. 3.
  2. Wearable assistant for Parkinson’s disease patients with the freezing of gait symptom. IEEE Transactions on Information Technology in Biomedicine 14, 2 (2009), 436–446.
  3. The UEA multivariate time series classification archive, 2018. arXiv preprint arXiv:1811.00075 (2018).
  4. mHealthDroid: a novel framework for agile development of mobile health applications. In International workshop on ambient assisted living. Springer, 91–98.
  5. Distributionally robust semi-supervised learning for people-centric sensing. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33.
  6. Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. ACM Computing Surveys (CSUR) 54, 4 (2021).
  7. Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 785–794.
  8. Belkacem Chikhaoui and Frank Gouineau. 2017. Towards automatic feature extraction for activity recognition from wearable sensors: a deep learning approach. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE.
  9. PrimeNet: Pre-Training for Irregular Multivariate Time Series. In Proceedings of the AAAI Conference on Artificial Intelligence.
  10. TARNet: Task-Aware Reconstruction for Time-Series Transformer. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA. 14–18.
  11. Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv:1603.06995 (2016).
  12. ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery 34, 5 (2020), 1454–1495.
  13. Minirocket: A very fast (almost) deterministic transform for time series classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 248–257.
  14. HHAR-net: hierarchical human activity recognition using neural networks. In International Conference on Intelligent Human Computer Interaction. Springer, 48–58.
  15. Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing 14, 7 (2010), 645–662.
  16. Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880 (2016).
  17. Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics (1979), 65–70.
  18. HM Sajjad Hossain and Nirmalya Roy. 2019. Active deep learning for activity recognition with context aware annotator selection. In KDD. 1862–1870.
  19. Inceptiontime: Finding alexnet for time series classification. Data Mining and Knowledge Discovery 34, 6 (2020), 1936–1962.
  20. SenseHAR: a robust virtual activity sensor for smartphones and wearables. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems.
  21. LSTM fully convolutional networks for time series classification. IEEE access 6 (2017), 1662–1669.
  22. Multivariate LSTM-FCNs for time series classification. Neural Networks 116 (2019), 237–245.
  23. Predicting deep zero-shot convolutional neural networks using textual descriptions. In Proceedings of the IEEE international conference on computer vision. 4247–4255.
  24. Two-stream convolution augmented transformer for human activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 286–293.
  25. Meta-har: Federated representation learning for human activity recognition. In Proceedings of the Web Conference 2021. 912–922.
  26. Shapenet: A shapelet-neural network approach for multivariate time series classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 8375–8383.
  27. Time series classification with HIVE-COTE: The hierarchical vote collective of transformation-based ensembles. ACM Transactions on Knowledge Discovery from Data 12, 5 (2018).
  28. HARTH: A Human Activity Recognition Dataset for Machine Learning. Sensors 21, 23 (2021), 7853.
  29. AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition.. In IJCAI. 3109–3115.
  30. Yuchao Ma and Hassan Ghasemzadeh. 2019. LabelForest: Non-parametric semi-supervised learning for activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 4520–4527.
  31. Characterizing word embeddings for zero-shot sensor-based human activity recognition. Sensors 19, 22 (2019), 5043.
  32. OpenAI. 2023. GPT-4 Technical Report. ArXiv abs/2303.08774 (2023).
  33. Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.
  34. Sensor-based activity recognition via learning from distributions. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
  35. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. PMLR, 8748–8763.
  36. Multimodal deep learning for activity and context recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 1–27.
  37. Attila Reiss and Didier Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In 2012 16th international symposium on wearable computers. IEEE, 108–109.
  38. Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh international conference on networked sensing systems (INSS). IEEE, 233–240.
  39. Patrick Schäfer and Ulf Leser. 2017. Multivariate time series classification with WEASEL+ MUSE. arXiv preprint arXiv:1711.11343 (2017).
  40. TS-CHIEF: a scalable and accurate forest algorithm for time series classification. Data Mining and Knowledge Discovery 34, 3 (2020), 742–775.
  41. On the non-trivial generalization of dynamic time warping to the multi-dimensional case. In Proceedings of the 2015 SIAM international conference on data mining. SIAM, 289–297.
  42. Zero-Shot Learning for IMU-Based Activity Recognition Using Video Embeddings. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 4 (2021), 1–23.
  43. Informed Machine Learning–A Taxonomy and Survey of Integrating Knowledge into Learning Systems. arXiv preprint arXiv:1903.12394 (2019).
  44. Zero-shot human activity recognition via nonlinear compatibility based method. In Proceedings of the International Conference on Web Intelligence. 322–330.
  45. Time series classification from scratch with deep neural networks: A strong baseline. In 2017 International joint conference on neural networks (IJCNN). IEEE, 1578–1585.
  46. Smartphone and smartwatch-based biometrics using activities of daily living. IEEE Access 7 (2019), 133190–133202.
  47. Falls detection and notification system using tri-axial accelerometer and gyroscope sensors of a smartphone. In 2013 Conference on Technologies and Applications of Artificial Intelligence. IEEE, 382–385.
  48. Ronald J Williams and David Zipser. 1989. A learning algorithm for continually running fully recurrent neural networks. Neural computation 1, 2 (1989), 270–280.
  49. Multi-layer cross loss model for zero-shot human activity recognition. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 210–221.
  50. Deepsense: A unified deep learning framework for time-series mobile sensing data processing. In Proceedings of the 26th International Conference on World Wide Web. 351–360.
  51. Zhi Zeng and Qiang Ji. 2010. Knowledge based activity recognition with dynamic bayesian network. In European conference on computer vision. Springer, 532–546.
  52. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2114–2124.
  53. Towards Similarity-Aware Time-Series Classification. arXiv preprint arXiv:2201.01413 (2022).
  54. Navigating Alignment for Non-identical Client Class Sets: A Label Name-Anchored Federated Learning Framework. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM.
  55. Mi Zhang and Alexander A Sawchuk. 2012. USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the 2012 ACM conference on ubiquitous computing. 1036–1043.
  56. Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances. arXiv preprint arXiv:2111.00418 (2021).
  57. ESC-GAN: Extending spatial coverage of physical sensors. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 1347–1356.
  58. Tapnet: Multivariate time series classification with attentional prototypical network. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 6845–6852.
  59. Deep semantic dictionary learning for multi-label image classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 3572–3580.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now