Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unsupervised Statistical Feature-Guided Diffusion Model for Sensor-based Human Activity Recognition (2306.05285v2)

Published 30 May 2023 in eess.SP and cs.LG

Abstract: Human activity recognition (HAR) from on-body sensors is a core functionality in many AI applications: from personal health, through sports and wellness to Industry 4.0. A key problem holding up progress in wearable sensor-based HAR, compared to other ML areas, such as computer vision, is the unavailability of diverse and labeled training data. Particularly, while there are innumerable annotated images available in online repositories, freely available sensor data is sparse and mostly unlabeled. We propose an unsupervised statistical feature-guided diffusion model specifically optimized for wearable sensor-based human activity recognition with devices such as inertial measurement unit (IMU) sensors. The method generates synthetic labeled time-series sensor data without relying on annotated training data. Thereby, it addresses the scarcity and annotation difficulties associated with real-world sensor data. By conditioning the diffusion model on statistical information such as mean, standard deviation, Z-score, and skewness, we generate diverse and representative synthetic sensor data. We conducted experiments on public human activity recognition datasets and compared the method to conventional oversampling and state-of-the-art generative adversarial network methods. Experimental results demonstrate that this can improve the performance of human activity recognition and outperform existing techniques.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (81)
  1. Comparing sampling strategies for tackling imbalanced data in human activity recognition. Sensors 22, 4 (2022), 1373.
  2. Wearable assistant for Parkinson’s disease patients with the freezing of gait symptom. IEEE Transactions on Information Technology in Biomedicine 14, 2 (2009), 436–446.
  3. Dmitrijs Balabka. 2019. Semi-supervised learning for human activity recognition using adversarial autoencoders. In Adjunct proceedings of the 2019 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2019 ACM international symposium on wearable computers. 685–688.
  4. Hossam Magdy Balaha and Asmaa El-Sayed Hassan. 2023. Comprehensive machine and deep learning analysis of sensor-based human activity recognition. Neural Computing and Applications 35, 17 (2023), 12793–12831.
  5. Opportunistic activity recognition in IoT sensor ecosystems via multimodal transfer learning. Neural Processing Letters (2021), 1–29.
  6. Kinect= imu? learning mimo signal mappings to automatically translate activity recognition systems across sensor modalities. In 2012 16th International Symposium on Wearable Computers. IEEE, 92–99.
  7. MWMOTE–majority weighted minority oversampling technique for imbalanced data set learning. IEEE Transactions on knowledge and data engineering 26, 2 (2012), 405–425.
  8. The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters 34, 15 (2013), 2033–2042.
  9. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research 16 (2002), 321–357.
  10. Learning hierarchical time series data augmentation invariances via contrastive supervision for human activity recognition. Knowledge-Based Systems (2023), 110789.
  11. MLCNNwav: Multi-level Convolutional Neural Network with Wavelet Transformations for Sensor-based Human Activity Recognition. IEEE Internet of Things Journal (2023).
  12. Cocoa: Cross modality contrastive learning for sensor data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1–28.
  13. Unobtrusive user-authentication on mobile phones using biometric gait recognition. In 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. IEEE, 306–311.
  14. Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion Models Beat GANs on Image Synthesis. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.), Vol. 34. Curran Associates, Inc., 8780–8794. https://proceedings.neurips.cc/paper_files/paper/2021/file/49ad23d1ec9fa4bd8d77d02681df5cfa-Paper.pdf
  15. Wearable sensor-based human activity recognition with transformer model. Sensors 22, 5 (2022), 1911.
  16. GENIE: Higher-order denoising diffusion solvers. arXiv preprint arXiv:2210.05475 (2022).
  17. ATFA: Adversarial Time–Frequency Attention network for sensor-based multimodal human activity recognition. Expert Systems with Applications 236 (2024), 121296.
  18. Anna Ferrari et al. 2023. Deep learning and model personalization in sensor-based human activity recognition. Journal of Reliable Intelligent Environments 9, 1 (2023), 27–39.
  19. Masquare: A Functional Smart Mask Design for Health Monitoring. In 2021 International Symposium on Wearable Computers. 175–178.
  20. Translating videos into synthetic training data for wearable sensor-based activity recognition systems using residual deep convolutional networks. Applied Sciences 11, 7 (2021), 3094.
  21. Learning from the Best: Contrastive Representations Learning Across Sensor Locations for Wearable Activity Recognition. In Proceedings of the 2022 ACM International Symposium on Wearable Computers. 28–32.
  22. Unsupervised Human Activity Recognition Via Large Language Models and Iterative Evolution. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 91–95.
  23. Zakaria Gheid and Yacine Challal. 2016. Novel Efficient and Privacy-Preserving Protocols For Sensor-Based Human Activity Recognition. In 13th International Conference on Ubiquitous Intelligence and Computing (UIC 2016). IEEE, 301–308.
  24. Diffuseq: Sequence to sequence text generation with diffusion models. arXiv preprint arXiv:2210.08933 (2022).
  25. Generative Adversarial Nets. In NIPS.
  26. Vector Quantized Diffusion Model for Text-to-Image Synthesis. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 10686–10696.
  27. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 6840–6851. https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf
  28. Video Diffusion Models. ArXiv abs/2204.03458 (2022).
  29. Yifan Hu. 2023. BSDGAN: Balancing Sensor Data Generative Adversarial Networks for Human Activity Recognition. In 2023 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.
  30. DiffAR: adaptive conditional diffusion model for temporal-augmented human activity recognition. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence. 3812–3820.
  31. Sensor-data augmentation for human activity recognition with time-warping and data masking. Multimedia Tools and Applications 80 (2021), 20991–21009.
  32. Model-based data augmentation for user-independent fatigue estimation. Computers in Biology and Medicine 137 (2021), 104839.
  33. Adversarial score matching and improved sampling for image generation. ArXiv abs/2009.05475 (2020).
  34. Flame: Free-form language-based motion synthesis & editing. arXiv preprint arXiv:2209.00349 (2022).
  35. Mooseop Kim and Chi Yoon Jeong. 2021. Label-preserving data augmentation for mobile sensor data. Multidimensional Systems and Signal Processing 32, 1 (2021), 115–129.
  36. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  37. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761 (2020).
  38. Imutube: Automatic extraction of virtual on-body accelerometry from video for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1–29.
  39. ActivityGAN: Generative adversarial networks for data augmentation in sensor-based human activity recognition. In Adjunct proceedings of the 2020 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2020 ACM international symposium on wearable computers. 249–254.
  40. Block annotation: Better image annotation with sub-image decomposition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5290–5300.
  41. FOCAL: Contrastive learning for multimodal time-series sensing signals in factorized orthogonal latent space. Advances in Neural Information Processing Systems 36 (2024).
  42. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11461–11471.
  43. A review: Data pre-processing and data augmentation techniques. Global Transitions Proceedings 3, 1 (2022), 91–99.
  44. Using deep data augmentation training to address software and hardware heterogeneities in wearable and smartphone sensing devices. In 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 200–211.
  45. Borderline over-sampling for imbalanced data classification. International Journal of Knowledge Engineering and Soft Data Paradigms 3, 1 (2011), 4–21.
  46. Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning. PMLR, 8162–8171.
  47. Study on human activity recognition using semi-supervised active transfer learning. Sensors 21, 8 (2021), 2760.
  48. Augmenting wearable sensor data with physical constraint for DNN-based human-action recognition. In ICML 2017 times series workshop. 6–11.
  49. MultiCNN-FilterLSTM: Resource-efficient sensor-based human activity recognition in IoT applications. Future Generation Computer Systems 139 (2023), 196–209.
  50. Transformer-based deep reverse attention network for multi-sensory human activity recognition. Engineering Applications of Artificial Intelligence 122 (2023), 106150.
  51. A hybrid hierarchical framework for gym physical activity recognition and measurement using wearable sensors. IEEE Internet of Things Journal 6, 2 (2018), 1384–1393.
  52. Attila Reiss and Didier Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In 2012 16th international symposium on wearable computers. IEEE, 108–109.
  53. Let there be IMU data: generating training data for wearable, motion sensor based activity recognition from monocular RGB videos. In Adjunct proceedings of the 2019 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2019 ACM international symposium on wearable computers. 699–708.
  54. High-Resolution Image Synthesis with Latent Diffusion Models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 10674–10685.
  55. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, 234–241.
  56. LabelMe: a database and web-based tool for image annotation. International journal of computer vision 77 (2008), 157–173.
  57. Synthetic smartwatch imu data generation from in-the-wild asl videos. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 2 (2023), 1–34.
  58. Shuai Shao and Victor Sanchez. 2023. A study on diffusion modelling for sensor-based human activity recognition. In 2023 11th International Workshop on Biometrics and Forensics (IWBF). IEEE, 1–7.
  59. Taoran Sheng and Manfred Huber. 2020. Unsupervised embedding learning for human activity recognition using wearable sensor data. In The Thirty-Third International Flairs Conference.
  60. Connor Shorten and Taghi M Khoshgoftaar. 2019. A survey on image data augmentation for deep learning. Journal of big data 6, 1 (2019), 1–48.
  61. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. ArXiv abs/1503.03585 (2015).
  62. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020).
  63. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations. https://openreview.net/forum?id=PxTIG12RRHS
  64. Mm-fit: Multimodal deep learning for automatic exercise logging across sensing devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 4 (2020), 1–22.
  65. Smart-mat: Recognizing and counting gym exercises with low-cost resistive pressure sensing matrix. In Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing. 373–382.
  66. Synthetic IMU datasets and protocols can simplify fall detection experiments and optimize sensor configuration. IEEE transactions on neural systems and rehabilitation engineering (2024).
  67. Co-simulation of human digital twins and wearable inertial sensors to analyse gait event estimation. Frontiers in Bioengineering and Biotechnology 11 (2023), 1104000.
  68. Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM international conference on multimodal interaction. 216–220.
  69. Attention is all you need. Advances in neural information processing systems 30 (2017).
  70. Sensorygans: An effective generative adversarial framework for sensor-based human activity recognition. In 2018 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.
  71. A deep learning method for complex human activity recognition using virtual wearable sensors. In Spatial Data and Intelligence: First International Conference, SpatialDI 2020, Virtual Event, May 8–9, 2020, Proceedings 1. Springer, 261–270.
  72. Two-stream transformer network for sensor-based human activity recognition. Neurocomputing 512 (2022), 253–268.
  73. Enabling edge devices that learn from each other: Cross modal training for activity recognition. In Proceedings of the 1st International Workshop on Edge Systems, Analytics and Networking. 37–42.
  74. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.02923 (2022).
  75. Ts-gan: Time-series gan for sensor-based health data augmentation. ACM Transactions on Computing for Healthcare 4, 2 (2023), 1–21.
  76. Sensegan: Enabling deep learning for internet of things with a semi-supervised framework. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 2, 3 (2018), 1–21.
  77. Time-series generative adversarial networks. Advances in neural information processing systems 32 (2019).
  78. Deconvolutional networks. In 2010 IEEE Computer Society Conference on computer vision and pattern recognition. IEEE, 2528–2535.
  79. Collective protection: Preventing sensitive inferences via integrative transformation. In 2019 IEEE international conference on data mining (ICDM). IEEE, 1498–1503.
  80. Quali-Mat: Evaluating the Quality of Execution in Body-Weight Exercises with a Pressure Sensitive Sports Mat. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 2 (2022), 1–45.
  81. 3d shape generation and completion through point-voxel diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5826–5835.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Si Zuo (4 papers)
  2. Vitor Fortes Rey (24 papers)
  3. Sungho Suh (52 papers)
  4. Stephan Sigg (20 papers)
  5. Paul Lukowicz (90 papers)
Citations (2)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets