Papers
Topics
Authors
Recent
Search
2000 character limit reached

Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition

Published 28 Oct 2023 in cs.CV, cs.AI, and cs.LG | (2310.18562v2)

Abstract: Human Activity Recognition (HAR) models often suffer from performance degradation in real-world applications due to distribution shifts in activity patterns across individuals. Test-Time Adaptation (TTA) is an emerging learning paradigm that aims to utilize the test stream to adjust predictions in real-time inference, which has not been explored in HAR before. However, the high computational cost of optimization-based TTA algorithms makes it intractable to run on resource-constrained edge devices. In this paper, we propose an Optimization-Free Test-Time Adaptation (OFTTA) framework for sensor-based HAR. OFTTA adjusts the feature extractor and linear classifier simultaneously in an optimization-free manner. For the feature extractor, we propose Exponential DecayTest-time Normalization (EDTN) to replace the conventional batch normalization (CBN) layers. EDTN combines CBN and Test-time batch Normalization (TBN) to extract reliable features against domain shifts with TBN's influence decreasing exponentially in deeper layers. For the classifier, we adjust the prediction by computing the distance between the feature and the prototype, which is calculated by a maintained support set. In addition, the update of the support set is based on the pseudo label, which can benefit from reliable features extracted by EDTN. Extensive experiments on three public cross-person HAR datasets and two different TTA settings demonstrate that OFTTA outperforms the state-of-the-art TTA approaches in both classification performance and computational efficiency. Finally, we verify the superiority of our proposed OFTTA on edge devices, indicating possible deployment in real applications. Our code is available at https://github.com/Claydon-Wang/OFTTA.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In Ambient Assisted Living and Home Care: 4th International Workshop, IWAAL 2012, Vitoria-Gasteiz, Spain, December 3-5, 2012. Proceedings 4. Springer, 216–223.
  2. Ling Bao and Stephen S Intille. 2004. Activity recognition from user-annotated acceleration data. In Pervasive Computing: Second International Conference, PERVASIVE 2004, Linz/Vienna, Austria, April 21-23, 2004. Proceedings 2. Springer, 1–17.
  3. IoT wearable sensor and deep learning: An integrated approach for personalized human activity recognition in a smart home environment. IEEE Internet of Things Journal 6, 5 (2019), 8553–8562.
  4. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP). IEEE, 141–159.
  5. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 46, 3 (2014), 1–33.
  6. A systematic study of unsupervised domain adaptation for robust human-activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1–30.
  7. Adaptive methods for real-world domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14340–14349.
  8. Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17, 1 (2016), 2096–2030.
  9. Robust human action recognition via long short-term memory. In The 2013 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.
  10. Deep residual learning for image recognition. In Proceedings of the IEEE onference on Computer Vision and Pattern Recognition. 770–778.
  11. MECTA: Memory-Economic Continual Test-Time Model Adaptation. In The Eleventh International Conference on Learning Representations.
  12. Mixnorm: Test-time adaptation through online normalization estimation. arXiv preprint arXiv:2110.11478 (2021).
  13. Normalization techniques in training dnns: Methodology, analysis and application. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).
  14. Deep ensemble learning for human activity recognition using wearable sensors via filter activation. ACM Transactions on Embedded Computing Systems 22, 1 (2022), 1–23.
  15. Channel-Equalization-HAR: a light-weight convolutional neural network for wearable sensor based human activity recognition. IEEE Transactions on Mobile Computing 22, 9 (2023), 5064–5077.
  16. Yusuke Iwasawa and Yutaka Matsuo. 2021. Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems 34 (2021), 2427–2440.
  17. Minguk Jang and Sae-Young Chung. 2023. Test-time adaptation via self-training with nearest neighbor information. In The Eleventh International Conference on Learning Representations.
  18. Scaling human activity recognition via deep learning-based domain adaptation. In 2018 IEEE international conference on Pervasive Computing and Communications (PerCom). IEEE, 1–9.
  19. Reshma Khemchandani and Sweta Sharma. 2016. Robust least squares twin support vector machine for human activity recognition. Applied Soft Computing 47 (2016), 33–46.
  20. Similarity of neural network representations revisited. In International conference on machine learning. PMLR, 3519–3529.
  21. Dong-Hyun Lee et al. 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, Vol. 3. 896.
  22. A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts. arXiv preprint arXiv:2303.15361 (2023).
  23. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning. PMLR, 6028–6039.
  24. TTN: A domain-shift aware batch normalization in test-time adaptation. In 11th International Conference on Learning Representations.
  25. Jianhua Lin. 1991. Divergence measures based on the Shannon entropy. IEEE Transactions on Information theory 37, 1 (1991), 145–151.
  26. Local and global alignments for generalizable sensor-based human activity recognition. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3833–3837.
  27. Semantic-discriminative mixup for generalizable sensor-based cross-domain activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 2 (2022), 1–19.
  28. Out-of-distribution Representation Learning for Time Series Classification. In The Eleventh International Conference on Learning Representations.
  29. Unimib shar: A dataset for human activity recognition using acceleration data from smartphones. Applied Sciences 7, 10 (2017), 1101.
  30. Efficient test-time model adaptation without forgetting. In International conference on machine learning. PMLR, 16888–16905.
  31. Towards stable test-time adaptation in dynamic wild world. In 11th International Conference on Learning Representations.
  32. Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.
  33. Preksha Pareek and Ankit Thakkar. 2021. A survey on video-based human action recognition: recent updates, datasets, challenges, and applications. Artificial Intelligence Review 54 (2021), 2259–2322.
  34. Sayak Paul and Pin-Yu Chen. 2022. Vision transformers are robust learners. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 2071–2081.
  35. Latent independent excitation for generalizable sensor-based cross-person activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11921–11929.
  36. Domain Generalization for Activity Recognition via Adaptive Feature Fusion. ACM Transactions on Intelligent Systems and Technology 14, 1 (2022), 1–21.
  37. Do vision transformers see like convolutional neural networks? Advances in Neural Information Processing Systems 34 (2021), 12116–12128.
  38. Activity recognition from accelerometer data. In Aaai, Vol. 5. Pittsburgh, PA, 1541–1546.
  39. Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh international conference on networked sensing systems (INSS). IEEE, 233–240.
  40. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems 33 (2020), 11539–11551.
  41. Learning disentangled behaviour patterns for wearable-based human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1–19.
  42. Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering. Advances in Neural Information Processing Systems 35 (2022), 17543–17555.
  43. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).
  44. Vladimir Vapnik. 1991. Principles of risk minimization for learning theory. Advances in neural information processing systems 4 (1991).
  45. Tent: Fully Test-Time Adaptation by Entropy Minimization. In 9th International Conference on Learning Representations.
  46. Deep learning for sensor-based activity recognition: A survey. Pattern recognition letters 119 (2019), 3–11.
  47. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering (2022).
  48. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7201–7211.
  49. A survey on wearable sensor modality centred human activity recognition in health care. Expert Systems with Applications 137 (2019), 167–190.
  50. Revisiting locally supervised learning: an alternative to end-to-end training. In 9th International Conference on Learning Representations.
  51. Uncovering Adversarial Risks of Test-Time Adaptation. arXiv preprint arXiv:2301.12576 (2023).
  52. Deep convolutional neural networks on multichannel time series for human activity recognition.. In Ijcai, Vol. 15. Buenos Aires, Argentina, 3995–4001.
  53. Test-time batch statistics calibration for covariate shift. arXiv preprint arXiv:2110.04065 (2021).
  54. Convolutional neural networks for human activity recognition using mobile sensors. In 6th international conference on mobile computing, applications and services. IEEE, 197–205.
  55. Memo: Test time robustness via adaptation and augmentation. Advances in Neural Information Processing Systems 35 (2022), 38629–38642.
  56. AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation. (2023).
  57. Domain-specific risk minimization. ECCV 2022 Workshop on Out-of-Distribution Generalization in Computer Vision (2022).
  58. Learning domain invariant representations for generalizable person re-identification. IEEE Transactions on Image Processing (2022).
  59. DELTA: degradation-free fully test-time adaptation. arXiv preprint arXiv:2301.13018 (2023).
  60. Domain Generalization with MixStyle. In 9th International Conference on Learning Representations.
Citations (7)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up