Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ADAPT^2: Adapting Pre-Trained Sensing Models to End-Users via Self-Supervision Replay (2404.15305v1)

Published 29 Mar 2024 in eess.SP and cs.LG

Abstract: Self-supervised learning has emerged as a method for utilizing massive unlabeled data for pre-training models, providing an effective feature extractor for various mobile sensing applications. However, when deployed to end-users, these models encounter significant domain shifts attributed to user diversity. We investigate the performance degradation that occurs when self-supervised models are fine-tuned in heterogeneous domains. To address the issue, we propose ADAPT2, a few-shot domain adaptation framework for personalizing self-supervised models. ADAPT2 proposes self-supervised meta-learning for initial model pre-training, followed by a user-side model adaptation by replaying the self-supervision with user-specific data. This allows models to adjust their pre-trained representations to the user with only a few samples. Evaluation with four benchmarks demonstrates that ADAPT2 outperforms existing baselines by an average F1-score of 8.8%p. Our on-device computational overhead analysis on a commodity off-the-shelf (COTS) smartphone shows that ADAPT2 completes adaptation within an unobtrusive latency (in three minutes) with only a 9.54% memory consumption, demonstrating the computational efficiency of the proposed method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Z. Wang, S. Tan, L. Zhang, Y. Ren, Z. Wang, and J. Yang, “An ear canal deformation based continuous user authentication using earables,” in Proceedings of the 27th Annual International Conference on Mobile Computing and Networking, 2021, pp. 819–821.
  2. S. Liu, W. Shao, T. Li, W. Xu, and L. Song, “Recent advances in biometrics-based user authentication for wearable devices: A contemporary survey,” Digital Signal Processing, vol. 125, p. 103120, 2022.
  3. H. Park, Y. Lee, and J. Ko, “Enabling real-time sign language translation on mobile platforms with on-board depth cameras,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 5, no. 2, pp. 1–30, 2021.
  4. H. Haresamudram, I. Essa, and T. Plötz, “Assessing the state of self-supervised human activity recognition using wearables,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 6, no. 3, pp. 1–47, 2022.
  5. H. Haresamudram, I. Essa, and T. Plötz, “Contrastive predictive coding for human activity recognition,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 5, no. 2, pp. 1–26, 2021.
  6. H. Haresamudram, I. Essa, and T. Plötz, “Investigating enhancements to contrastive predictive coding for human activity recognition,” in 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom), 2023, pp. 232–241.
  7. C. I. Tang, I. Perez-Pozuelo, D. Spathis, and C. Mascolo, “Exploring contrastive learning in human activity recognition for healthcare,” arXiv preprint arXiv:2011.11542, 2020.
  8. A. Saeed, T. Ozcelebi, and J. Lukkien, “Multi-task self-supervised learning for human activity detection,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 3, no. 2, pp. 1–30, 2019.
  9. Y. E. Ustev, O. Durmaz Incel, and C. Ersoy, “User, device and orientation independent human activity recognition on mobile phones: Challenges and a proposal,” in Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication, 2013, pp. 1427–1436.
  10. H. Qian, S. J. Pan, and C. Miao, “Latent independent excitation for generalizable sensor-based cross-person activity recognition,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2021, vol. 35, no. 13, pp. 11921–11929.
  11. X. Qin, J. Wang, Y. Chen, W. Lu, and X. Jiang, “Domain generalization for activity recognition via adaptive feature fusion,” ACM Transactions on Intelligent Systems and Technology, vol. 14, no. 1, pp. 1–21, 2022.
  12. W. Lu, J. Wang, Y. Chen, S. J. Pan, C. Hu, and X. Qin, “Semantic-discriminative mixup for generalizable sensor-based cross-domain activity recognition,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 6, no. 2, pp. 1–19, 2022.
  13. Y. Chang, A. Mathur, A. Isopoussu, J. Song, and F. Kawsar, “A systematic study of unsupervised domain adaptation for robust human-activity recognition,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 4, no. 1, pp. 1–30, 2020.
  14. T. Gong, Y. Kim, J. Shin, and S.-J. Lee, “Metasense: few-shot adaptation to untrained conditions in deep mobile sensing,” in Proceedings of the 17th Conference on Embedded Networked Sensor Systems, 2019, pp. 110–123.
  15. J. Wang, Y. Chen, L. Hu, X. Peng, and S. Y. Philip, “Stratified transfer learning for cross-domain activity recognition,” in 2018 IEEE international conference on pervasive computing and communications (PerCom), 2018, pp. 1–10.
  16. A. Reiss and D. Stricker, “Introducing a new benchmarked dataset for activity monitoring,” in 2012 16th international symposium on wearable computers, 2012, pp. 108–109.
  17. K. Altun, B. Barshan, and O. Tunçel, “Comparative study on classifying human activities with miniature inertial and magnetic sensors,” Pattern Recognition, vol. 43, no. 10, pp. 3605–3620, 2010.
  18. X. Zhang, L. Zhou, R. Xu, P. Cui, Z. Shen, and H. Liu, “Towards unsupervised domain generalization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4910–4920.
  19. D. B. Lee, S. Lee, J. Ko, K. Kawaguchi, J. Lee, and S. J. Hwang, “Self-Supervised Set Representation Learning for Unsupervised Meta-Learning,” International Conference On Learning Representations, 2023.
  20. A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon, “A survey on contrastive self-supervised learning,” Technologies, vol. 9, no. 1, p. 2, 2020.
  21. K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9729–9738.
  22. T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning, 2020, pp. 1597–1607.
  23. J. Wang, T. Zhu, J. Gan, L. L. Chen, H. Ning, and Y. Wan, “Sensor data augmentation by resampling in contrastive learning for human activity recognition,” IEEE Sensors Journal, vol. 22, no. 23, pp. 22994–23008, 2022.
  24. S. Deldari, H. Xue, A. Saeed, D. V. Smith, and F. D. Salim, “Cocoa: Cross modality contrastive learning for sensor data,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 6, no. 3, pp. 1–28, 2022.
  25. Y. Jain, C. I. Tang, C. Min, F. Kawsar, and A. Mathur, “Collossl: Collaborative self-supervised learning for human activity recognition,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 6, no. 1, pp. 1–28, 2022.
  26. H. Xu, P. Zhou, R. Tan, M. Li, and G. Shen, “Limu-bert: Unleashing the potential of unlabeled data for imu sensing applications,” in Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, 2021, pp. 220–233.
  27. K. Muandet, D. Balduzzi, and B. Schölkopf, “Domain generalization via invariant feature representation,” in International conference on machine learning, 2013, pp. 10–18.
  28. D. Li, J. Zhang, Y. Yang, C. Liu, Y.-Z. Song, and T. M. Hospedales, “Episodic training for domain generalization,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 1446–1455.
  29. H. Li, S. J. Pan, S. Wang, and A. C. Kot, “Domain generalization with adversarial feature learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5400–5409.
  30. Y. Wang, H. Li, and A. C. Kot, “Heterogeneous domain generalization via domain mixup,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 3622–3626.
  31. K. Zhou, Y. Yang, Y. Qiao, and T. Xiang, “Domain generalization with mixstyle,” International Conference On Learning Representations, 2021.
  32. D. Li, Y. Yang, Y.-Z. Song, and T. Hospedales, “Learning to generalize: Meta-learning for domain generalization,” in Proceedings of the AAAI conference on artificial intelligence, 2018, vol. 32, no. 1.
  33. F. Qiao, L. Zhao, and X. Peng, “Learning to learn single domain generalization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12556–12565.
  34. D. Kim, Y. Yoo, S. Park, J. Kim, and J. Lee, “Selfreg: Self-supervised contrastive regularization for domain generalization,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 9619–9628.
  35. G. Wilson and D. J. Cook, “A survey of unsupervised deep domain adaptation,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 11, no. 5, pp. 1–46, 2020.
  36. Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in International conference on machine learning, 2015, pp. 1180–1189.
  37. M. M. Rahman, C. Fookes, M. Baktashmotlagh, and S. Sridharan, “Correlation-aware adversarial domain adaptation and generalization,” Pattern Recognition, vol. 100, p. 107124, 2020.
  38. W. Lu, Y. Chen, J. Wang, and X. Qin, “Cross-domain activity recognition via substructural optimal transport,” Neurocomputing, vol. 454, pp. 65–75, 2021.
  39. M. A. A. H. Khan, N. Roy, and A. Misra, “Scaling human activity recognition via deep learning-based domain adaptation,” in 2018 IEEE international conference on pervasive computing and communications (PerCom), 2018, pp. 1–9.
  40. S. Khodadadeh, L. Boloni, and M. Shah, “Unsupervised meta-learning for few-shot image classification,” Advances in neural information processing systems, vol. 32, 2019.
  41. S. Khodadadeh, S. Zehtabian, S. Vahidian, W. Wang, B. Lin, and L. Bölöni, “Unsupervised meta-learning through latent-space interpolation in generative models,” arXiv preprint arXiv:2006.10236, 2020.
  42. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International conference on machine learning, 2017, pp. 1126–1135.
  43. T. Gong, Y. Kim, R. Choi, J. Shin, and S.-J. Lee, “Adapting to unknown conditions in learning-based mobile sensing,” IEEE Transactions on Mobile Computing, vol. 21, no. 10, pp. 3470–3485, 2021.
  44. Z. Wu, Y. Xiong, S. X. Yu, and D. Lin, “Unsupervised feature learning via non-parametric instance discrimination,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3733–3742.
  45. A. Raghu, M. Raghu, S. Bengio, and O. Vinyals, “Rapid learning or feature reuse? towards understanding the effectiveness of maml,” International Conference On Learning Representations, 2020.
  46. L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” arXiv preprint arXiv:1802.03426, 2018.
  47. “Microsoft Termux,” https://termux.dev/en/

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets