Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unsupervised Domain Adaptation Using Compact Internal Representations (2401.07207v1)

Published 14 Jan 2024 in cs.CV

Abstract: A major technique for tackling unsupervised domain adaptation involves mapping data points from both the source and target domains into a shared embedding space. The mapping encoder to the embedding space is trained such that the embedding space becomes domain agnostic, allowing a classifier trained on the source domain to generalize well on the target domain. To further enhance the performance of unsupervised domain adaptation (UDA), we develop an additional technique which makes the internal distribution of the source domain more compact, thereby improving the model's ability to generalize in the target domain.We demonstrate that by increasing the margins between data representations for different classes in the embedding space, we can improve the model performance for UDA. To make the internal representation more compact, we estimate the internally learned multi-modal distribution of the source domain as Gaussian mixture model (GMM). Utilizing the estimated GMM, we enhance the separation between different classes in the source domain, thereby mitigating the effects of domain shift. We offer theoretical analysis to support outperofrmance of our method. To evaluate the effectiveness of our approach, we conduct experiments on widely used UDA benchmark UDA datasets. The results indicate that our method enhances model generalizability and outperforms existing techniques.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In ECCV, pages 447–463, 2018.
  2. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of ECCV, pages 447–463, 2018.
  3. N. Bonnotte. Unidimensional and evolution methods for optimal transportation. PhD thesis, Paris 11, 2013.
  4. A theoretical analysis of the number of shots in few-shot learning. In International Conference on Learning Representations, 2019.
  5. Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation. In Proceedings of the AAAI, pages 3296–3303, 2019.
  6. Homm: Higher-order moment matching for unsupervised domain adaptation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3422–3429, 2020.
  7. Activate and reject: Towards safe domain generalization under category shift. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11552–11563, 2023.
  8. Progressive feature alignment for unsupervised domain adaptation. In Proceedings of CVPR, pages 627–636, 2019.
  9. Optimal transport for domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9):1853–1865, 2016.
  10. Margin-aware adversarial domain adaptation with optimal transport. In International Conference on Machine Learning, pages 2514–2524. PMLR, 2020.
  11. Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3937–3946, 2021.
  12. Hierarchical optimal transport for unsupervised domain adaptation. Machine Learning, 111(11):4159–4182, 2022.
  13. Learning a domain-invariant embedding for unsupervised domain adaptation using class-conditioned distribution alignment. In 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 352–359. IEEE, 2019.
  14. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 17(1):2096–2030, 2016.
  15. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180–1189. PMLR, 2015.
  16. Unsupervised domain adaptation by backpropagation. In Proceedings of Proceedings of ICML, pages 1180–1189, 2015.
  17. Deep reconstruction-classification networks for unsupervised domain adaptation. In Proceedings of ECCV, pages 597–613. Springer, 2016.
  18. Deep residual learning for image recognition. In Proceedings of CVPR, pages 770–778, 2016.
  19. Cycada: Cycle-consistent adversarial domain adaptation. In Proceedings of ICML, pages 1989–1998. PMLR, 2018.
  20. Model adaptation: Historical contrastive learning for unsupervised domain adaptation without source data. Advances in Neural Information Processing Systems, 34, 2021.
  21. Balancing transferability and discriminability for unsupervised domain adaptation. IEEE Transactions on Neural Networks and Learning Systems, 2022.
  22. Unsupervised domain adaptation for training event-based networks using contrastive learning and uncorrelated conditioning. arXiv preprint arXiv:2303.12424, 2023.
  23. Unsupervised visual domain adaptation: A deep max-margin gaussian process approach. In Proceedings of CVPR, pages 4380–4390, 2019.
  24. Did you get what you paid for? rethinking annotation cost of deep learning based computer aided detection in chest radiographs. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part III, pages 261–270. Springer, 2022.
  25. Universal source-free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4544–4553, 2020.
  26. Attending to discriminative certainty for domain adaptation. In CVPR, pages 491–500, 2019.
  27. Sliced wasserstein discrepancy for unsupervised domain adaptation. In Proceedings of CVPR, pages 10285–10295, 2019.
  28. Sequential learning for domain generalization. In European Conference on Computer Vision, pages 603–619. Springer, 2020.
  29. Pseudo-labeling integrating centers and samples with consistent selection mechanism for unsupervised domain adaptation. Information Sciences, 628:50–69, 2023.
  30. Enhanced transport distance for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13936–13944, 2020.
  31. Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9641–9650, 2020.
  32. Adaptive batch normalization for practical domain adaptation. Pattern Recognition, 80:109–117, 2018.
  33. Intelligent fault diagnosis by fusing domain adversarial training and maximum mean discrepancy via ensemble learning. IEEE Transactions on Industrial Informatics, 17(4):2833–2841, 2020.
  34. Cot: Unsupervised domain adaptation with clustering and optimal transport. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19998–20007, 2023.
  35. Learning transferable features with deep adaptation networks. In Proceedings of Proceedings of ICML, pages 97–105, 2015.
  36. Conditional adversarial domain adaptation. In Proceedings of NeurIPS, pages 1640–1650, 2018.
  37. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th Proceedings of ICML-Volume 70, pages 2208–2217. JMLR. org, 2017.
  38. Automatic loss function search for adversarial unsupervised domain adaptation. IEEE Transactions on Circuits and Systems for Video Technology, 2023.
  39. Todd K Moon. The expectation-maximization algorithm. IEEE Signal processing magazine, 13(6):47–60, 1996.
  40. Unified deep supervised domain adaptation and generalization. In Proceedings of CVPR, pages 5715–5725, 2017.
  41. Fixbi: Bridging domain spaces for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1094–1103, 2021.
  42. R. Neal. Slice sampling. Annals of statistics, pages 705–741, 2003.
  43. Unsupervised domain adaptation of object detectors: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  44. Transferrable prototypical networks for unsupervised domain adaptation. In Proceedings of CVPR, pages 2239–2247, 2019.
  45. Multi-adversarial domain adaptation. In Proceedings Thirty-Second AAAI, pages 3934–3941, 2018.
  46. A. Redko, I.and Habrard and M. Sebban. Theoretical analysis of domain adaptation with optimal transport. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 737–753. Springer, 2017.
  47. Mohammad Rostami. Lifelong domain adaptation via consolidated internal distribution. Advances in Neural Information Processing Systems, 34:11172–11183, 2021.
  48. Mohammad Rostami. Increasing model generalizability for unsupervised visual domain adaptation. In Conference on Lifelong Learning Agents, pages 281–293. PMLR, 2022.
  49. Domain adaptation for sentiment analysis using increased intraclass separation. In EMNLP, 2021.
  50. Cognitively inspired learning of incremental drifting concepts. In International Joint Conference on Artificial Intelligence, 2023.
  51. Overcoming concept shift in domain-aware settings through consolidated internal distributions. In Thirty-Seventh AAAI Conference on Artificial Intelligence, 2023.
  52. Transfer learning via representation learning. In Federated and Transfer Learning, pages 233–257. Springer, 2022.
  53. A crowdsourcing triage algorithm for geopolitical event forecasting. In Proceedings of the 12th ACM Conference on Recommender Systems, pages 377–381. ACM, 2018.
  54. Deep transfer learning for few-shot sar image classification. Remote Sensing, 11(11):1374, 2019.
  55. Sar image classification using few-shot cross-domain transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0–0, 2019.
  56. Detection and continual learning of novel face presentation attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14851–14860, 2021.
  57. Sam T Roweis. Em algorithms for pca and spca. In Proceedings of NeurIPS, pages 626–632, 1998.
  58. Generate to adapt: Aligning domains using generative adversarial networks. In Proceedings of CVPR, pages 8503–8512, 2018.
  59. Multi-head distillation for continual unsupervised domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3751–3760, 2022.
  60. Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.
  61. Unsupervised model adaptation for continual semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2593–2601, 2021.
  62. Domain adaptation for the segmentation of confidential medical images. In British Machine Vision Conference, 2022.
  63. Secure domain adaptation with multiple sources. Transactions on Machine Learning Research, 2022.
  64. Unsupervised model adaptation for source-free segmentation of medical images. arXiv preprint arXiv:2211.00807, 2022.
  65. Preserving fairness in ai under domain shift. arXiv preprint arXiv:2301.12369, 2023.
  66. Deep coral: Correlation alignment for deep domain adaptation. In Proceedings of ECCV, pages 443–450. Springer, 2016.
  67. Adversarial discriminative domain adaptation. In Proceedings of CVPR, pages 7167–7176, 2017.
  68. Metaalign: Coordinating domain alignment and classification for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16643–16653, 2021.
  69. Gradual source domain expansion for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1946–1955, 2024.
  70. Unsupervised domain adaptation for graph-structured data using class-conditional distribution alignment. arXiv preprint arXiv:2301.12361, 2023.
  71. Dual mixup regularized learning for adversarial domain adaptation. In Proceedings of ECCV, pages 540–555. Springer, 2020.
  72. Ni Xiao and Lei Zhang. Dynamic weighted learning for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15242–15251, 2021.
  73. A collaborative alignment framework of transferable knowledge extraction for unsupervised domain adaptation. IEEE Transactions on Knowledge and Data Engineering, 2022.
  74. Reliable weighted optimal transport for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4394–4403, 2020.
  75. Central moment discrepancy (cmd) for domain-invariant representation learning. In ICLR, 2017.
  76. Unsupervised domain adaptation for cross-lingual text labeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 3527–3536, 2020.
  77. Adaptive risk minimization: Learning to adapt to domain shift. Advances in Neural Information Processing Systems, 34:23664–23678, 2021.
  78. Category-level selective dual-adversarial network using significance-augmented unsupervised domain adaptation for surface defect detection. Expert Systems with Applications, 238:121879, 2024.
  79. Discriminative joint probability maximum mean discrepancy (djp-mmd) for domain adaptation. In 2020 international joint conference on neural networks (IJCNN), pages 1–8. IEEE, 2020.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now