Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
123 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
51 tokens/sec
2000 character limit reached

AdaEmbed: Semi-supervised Domain Adaptation in the Embedding Space (2401.12421v1)

Published 23 Jan 2024 in cs.CV and cs.AI

Abstract: Semi-supervised domain adaptation (SSDA) presents a critical hurdle in computer vision, especially given the frequent scarcity of labeled data in real-world settings. This scarcity often causes foundation models, trained on extensive datasets, to underperform when applied to new domains. AdaEmbed, our newly proposed methodology for SSDA, offers a promising solution to these challenges. Leveraging the potential of unlabeled data, AdaEmbed facilitates the transfer of knowledge from a labeled source domain to an unlabeled target domain by learning a shared embedding space. By generating accurate and uniform pseudo-labels based on the established embedding space, the model overcomes the limitations of conventional SSDA, thus enhancing performance significantly. Our method's effectiveness is validated through extensive experiments on benchmark datasets such as DomainNet, Office-Home, and VisDA-C, where AdaEmbed consistently outperforms all the baselines, setting a new state of the art for SSDA. With its straightforward implementation and high data efficiency, AdaEmbed stands out as a robust and pragmatic solution for real-world scenarios, where labeled data is scarce. To foster further research and application in this area, we are sharing the codebase of our unified framework for semi-supervised domain adaptation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Fast generalized distillation for semi-supervised domain adaptation. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, pp.  1719–1725. AAAI Press, 2017.
  2. Improving few-shot visual classification with unlabelled examples. CoRR, 2020.
  3. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. CoRR, 2019a.
  4. Mixmatch: A holistic approach to semi-supervised learning. CoRR, 2019b.
  5. Adamatch: A unified approach to semi-supervised learning and domain adaptation. arXiv preprint arXiv:2106.04732, 2021.
  6. Domain adaptation problems: A dasvm classification technique and a circular validation strategy. IEEE Trans. Pattern Anal. Mach. Intell., 2010.
  7. Memory matching networks for one-shot image recognition. CoRR, 2018.
  8. Unsupervised learning of visual features by contrasting cluster assignments. CoRR, 2020.
  9. Progressive feature alignment for unsupervised domain adaptation. CoRR, 2018.
  10. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  295–305, 2022.
  11. Co-training for domain adaptation. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
  12. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp.  1597–1607. PMLR, 2020a.
  13. Big self-supervised models are strong semi-supervised learners. CoRR, 2020b.
  14. A closer look at few-shot classification. In International Conference on Learning Representations, 2019.
  15. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp.  702–703, 2020.
  16. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.  248–255. Ieee, 2009.
  17. A baseline for few-shot image classification. In International Conference on Learning Representations, 2020.
  18. Decaf: A deep convolutional activation feature for generic visual recognition. In Proceedings of the 31st International Conference on Machine Learning, 2014.
  19. Towards a neural statistician. In International Conference on Learning Representations, 2017.
  20. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, 2017.
  21. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp.  1180–1189. PMLR, 2015.
  22. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096–2030, 2016.
  23. A kernel method for the two-sample-problem. Advances in neural information processing systems, 19, 2006.
  24. Bootstrap your own latent: A new approach to self-supervised learning. CoRR, 2020.
  25. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  9729–9738, 2020.
  26. Task agnostic meta-learning for few-shot learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  27. Deep adversarial attention alignment for unsupervised domain adaptation: the benefit of target expectation maximization. In Proceedings of the European conference on computer vision (ECCV), pp.  401–416, 2018.
  28. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2. Lille, 2015.
  29. Temporal ensembling for semi-supervised learning. CoRR, 2016.
  30. Ecacl: A holistic framework for semi-supervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  8578–8587, 2021.
  31. Meta-sgd: Learning to learn quickly for few-shot learning, 2017.
  32. Aggregating randomized clustering-promoting invariant projections for domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
  33. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. CoRR, 2020.
  34. Learning to propagate labels: Transductive propagation network for few-shot learning. In International Conference on Learning Representations, 2019.
  35. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  12009–12019, 2022.
  36. Transfer feature learning with joint distribution adaptation. In 2013 IEEE International Conference on Computer Vision, 2013.
  37. Learning transferable features with deep adaptation networks. In International conference on machine learning, pp.  97–105. PMLR, 2015.
  38. Unsupervised domain adaptation with residual transfer networks. Advances in neural information processing systems, 29, 2016.
  39. A simple neural attentive meta-learner. In International Conference on Learning Representations, 2018.
  40. Meta networks. CoRR, abs/1703.00837, 2017.
  41. Learning rapid-temporal adaptations. CoRR, 2017.
  42. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  43. Tadam: Task dependent adaptive metric for improved few-shot learning, 2018.
  44. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924, 2017.
  45. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  1406–1415, 2019.
  46. Optimization as a model for few-shot learning. In International conference on learning representations, 2016.
  47. Meta-learning with latent embedding optimization. In International Conference on Learning Representations, 2019.
  48. Contrast and mix: Temporal contrastive video domain adaptation with background mixing. Advances in Neural Information Processing Systems, 34:23386–23400, 2021.
  49. Asymmetric tri-training for unsupervised domain adaptation. CoRR, 2017.
  50. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  3723–3732, 2018.
  51. Semi-supervised domain adaptation via minimax entropy. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  8050–8058, 2019.
  52. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. Advances in neural information processing systems, 29, 2016.
  53. Meta-learning with memory-augmented neural networks. In International conference on machine learning, pp.  1842–1850, 2016.
  54. Few-shot learning with graph neural networks. In International Conference on Learning Representations, 2018.
  55. Re-ranking for image retrieval and transductive few-shot classification. Advances in Neural Information Processing Systems, 34:25932–25943, 2021.
  56. Ankit Singh. Clda: Contrastive learning for semi-supervised domain adaptation. Advances in Neural Information Processing Systems, 34:5089–5101, 2021.
  57. Prototypical networks for few-shot learning, 2017.
  58. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in Neural Information Processing Systems, 33:596–608, 2020.
  59. Deep coral: Correlation alignment for deep domain adaptation. In European conference on computer vision, pp.  443–450. Springer, 2016.
  60. Meta-transfer learning for few-shot learning. In CVPR, pp.  403–412, 2019.
  61. Learning to compare: Relation network for few-shot learning, 2017.
  62. Weight-averaged consistency targets improve semi-supervised deep learning results. CoRR, 2017.
  63. Adversarial discriminative domain adaptation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  64. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474, 2014.
  65. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  5018–5027, 2017.
  66. Matching networks for one shot learning. In Advances in neural information processing systems, pp.  3630–3638, 2016.
  67. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021.
  68. Simpleshot: Revisiting nearest-neighbor classification for few-shot learning. arXiv preprint arXiv:1911.04623, 2019.
  69. Ni Xiao and Lei Zhang. Dynamic weighted learning for unsupervised domain adaptation. CoRR, 2021.
  70. Learning semantic representations for unsupervised domain adaptation. In Proceedings of the 35th International Conference on Machine Learning. PMLR, 2018.
  71. Semi-supervised domain adaptation with subspace learning for visual recognition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.  2142–2150, 2015.
  72. Importance weighted adversarial nets for partial domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  8156–8164, 2018a.
  73. Collaborative and adversarial network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018b.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets