Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Enhancing Information Maximization with Distance-Aware Contrastive Learning for Source-Free Cross-Domain Few-Shot Learning (2403.01966v1)

Published 4 Mar 2024 in cs.CV

Abstract: Existing Cross-Domain Few-Shot Learning (CDFSL) methods require access to source domain data to train a model in the pre-training phase. However, due to increasing concerns about data privacy and the desire to reduce data transmission and training costs, it is necessary to develop a CDFSL solution without accessing source data. For this reason, this paper explores a Source-Free CDFSL (SF-CDFSL) problem, in which CDFSL is addressed through the use of existing pretrained models instead of training a model with source data, avoiding accessing source data. This paper proposes an Enhanced Information Maximization with Distance-Aware Contrastive Learning (IM-DCL) method to address these challenges. Firstly, we introduce the transductive mechanism for learning the query set. Secondly, information maximization (IM) is explored to map target samples into both individual certainty and global diversity predictions, helping the source model better fit the target data distribution. However, IM fails to learn the decision boundary of the target task. This motivates us to introduce a novel approach called Distance-Aware Contrastive Learning (DCL), in which we consider the entire feature set as both positive and negative sets, akin to Schrodinger's concept of a dual state. Instead of a rigid separation between positive and negative sets, we employ a weighted distance calculation among features to establish a soft classification of the positive and negative sets for the entire feature set. Furthermore, we address issues related to IM by incorporating contrastive constraints between object features and their corresponding positive and negative sets. Evaluations of the 4 datasets in the BSCD-FSL benchmark indicate that the proposed IM-DCL, without accessing the source domain, demonstrates superiority over existing methods, especially in the distant domain task.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, “Human-level concept learning through probabilistic program induction,” Science, vol. 350, no. 6266, pp. 1332–1338, 2015.
  2. Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a few examples: A survey on few-shot learning,” ACM CSUR, vol. 53, no. 3, pp. 1–34, 2020.
  3. S. Sun, H. Xu, Y. Li, P. Li, B. Sheng, and X. Lin, “Fastal: Fast evaluation module for efficient dynamic deep active learning using broad learning system,” TCSVT, 2023.
  4. Y. Song, T. Wang, P. Cai, S. K. Mondal, and J. P. Sahoo, “A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities,” ACM CSUR, 2023.
  5. Z. Yang, C. Zhang, R. Li, Y. Xu, and G. Lin, “Efficient few-shot object detection via knowledge inheritance,” TIP, vol. 32, pp. 321–334, 2022.
  6. A. Vu, T. Do, N. Nguyen, V. Nguyen, T. Ngo, and T. Nguyen, “Instance-level few-shot learning with class hierarchy mining,” TIP, 2023.
  7. Y. Guo, R. Du, X. Li, J. Xie, Z. Ma, and Y. Dong, “Learning calibrated class centers for few-shot classification by pair-wise similarity,” TIP, vol. 31, pp. 4543–4555, 2022.
  8. H. Xu, S. Zhi, S. Sun, V. M. Patel, and L. Liu, “Deep learning for cross-domain few-shot visual recognition: A survey,” arXiv preprint arXiv:2303.08557, 2023.
  9. H. Tseng, H. Lee, J. Huang, and M. Yang, “Cross-domain few-shot classification via learned feature-wise transformation,” in ICLR, 2020.
  10. C. P. Phoo and B. Hariharan, “Self-training for few-shot transfer across extreme task differences,” in ICLR, 2020.
  11. A. Islam, C. R. Chen, R. Panda, L. Karlinsky, R. Feris, and R. J. Radke, “Dynamic distillation network for cross-domain few-shot recognition with unlabeled data,” NIPS, vol. 34, pp. 3584–3595, 2021.
  12. H. Xu, S. Zhi, and L. Liu, “Cross-domain few-shot classification via inter-source stylization,” ICIP, 2023.
  13. Y. Fu, Y. Xie, Y. Fu, and Y. Jiang, “Styleadv: Meta style adversarial training for cross-domain few-shot learning,” in CVPR, 2023, pp. 24 575–24 584.
  14. Y. Zhao and N. Cheung, “Fs-ban: Born-again networks for domain generalization few-shot classification,” TIP, 2023.
  15. P. Li, S. Gong, C. Wang, and Y. Fu, “Ranking distance calibration for cross-domain few-shot learning,” in CVPR, 2022, pp. 9099–9108.
  16. Y. Zhao, T. Zhang, J. Li, and Y. Tian, “Dual adaptive representation alignment for cross-domain few-shot learning,” TPAMI, 2023.
  17. J. Liang, D. Hu, and J. Feng, “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,” in ICML.   PMLR, 2020, pp. 6028–6039.
  18. N. Ding, Y. Xu, Y. Tang, C. Xu, Y. Wang, and D. Tao, “Source-free domain adaptation via distribution estimation,” in CVPR, 2022, pp. 7212–7222.
  19. A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, and J. Clark, “Learning transferable visual models from natural language supervision,” in ICML.   PMLR, 2021, pp. 8748–8763.
  20. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, and S. Gelly, “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR, 2020.
  21. B. Liu, Z. Zhao, Z. Li, J. Jiang, Y. Guo, and J. Ye, “Feature transformation ensemble model with batch spectral regularization for cross-domain few-shot classification,” CVPR Challenge, 2020.
  22. A. Gammerman, V. Vovk, and V. Vapnik, “Learning by transduction,” arXiv preprint arXiv:1301.7375, 2013.
  23. Y. Guo, N. C. Codella, L. Karlinsky, J. V. Codella, J. R. Smith, K. Saenko, T. Rosing, and R. Feris, “A broader study of cross-domain few-shot learning,” in ECCV.   Springer, 2020, pp. 124–141.
  24. J. Sun, S. Lapuschkin, W. Samek, Y. Zhao, N. Cheung, and A. Binder, “Explanation-guided training for cross-domain few-shot classification,” in ICPR.   IEEE, 2021, pp. 7609–7616.
  25. Y. Fu, Y. Fu, and Y. Jiang, “Meta-fdmixup: Cross-domain few-shot learning guided by labeled target data,” in ACM MM, 2021, pp. 5326–5334.
  26. Y. Fu, Y. Fu, J. Chen, and Y. Jiang, “Generalized meta-fdmixup: Cross-domain few-shot learning guided by labeled target data,” TIP, vol. 31, pp. 7078–7090, 2022.
  27. Y. Fu, Y. Xie, Y. Fu, J. Chen, and Y. Jiang, “Wave-san: Wavelet based style augmentation network for cross-domain few-shot learning,” arXiv preprint arXiv:2203.07656, 2022.
  28. J. Zhang, J. Song, L. Gao, and H. Shen, “Free-lunch for cross-domain few-shot learning: Style-aware episodic training with robust contrastive learning,” in ACM MM, 2022, pp. 2586–2594.
  29. Y. Zhang, W. Li, M. Zhang, S. Wang, R. Tao, and Q. Du, “Graph information aggregation cross-domain few-shot learning for hyperspectral image classification,” TNNLS, pp. 1–14, 2022.
  30. Y. Zhang, W. Li, W. Sun, R. Tao, and Q. Du, “Single-source domain expansion network for cross-scene hyperspectral image classification,” TIP, vol. 32, pp. 1498–1512, 2023.
  31. Y. Zhang, W. Li, M. Zhang, Y. Qu, R. Tao, and H. Qi, “Topological structure and semantic information transfer network for cross-scene hyperspectral image classification,” TNNLS, 2021.
  32. J. Pei, Z. Jiang, A. Men, L. Chen, Y. Liu, and Q. Chen, “Uncertainty-induced transferability representation for source-free unsupervised domain adaptation,” TIP, vol. 32, pp. 2033–2048, 2023.
  33. S. Roy, M. Trapp, A. Pilzer, J. Kannala, N. Sebe, E. Ricci, and A. Solin, “Uncertainty-guided source-free domain adaptation,” in ECCV.   Springer, 2022, pp. 537–555.
  34. H. Xia, H. Zhao, and Z. Ding, “Adaptive adversarial network for source-free domain adaptation,” in ICCV, 2021, pp. 9010–9019.
  35. S. Yang, J. van de Weijer, L. Herranz, and S. Jui, “Exploiting the intrinsic neighborhood structure for source-free domain adaptation,” NIPS, vol. 34, pp. 29 393–29 405, 2021.
  36. Y. Liu, J. Lee, M. Park, S. Kim, E. Yang, S. Hwang, and Y. Yang, “Learning to propagate labels: Transductive propagation network for few-shot learning,” in ICLR, 2019.
  37. L. Qiao, Y. Shi, J. Li, Y. Wang, T. Huang, and Y. Tian, “Transductive episodic-wise adaptive metric for few-shot learning,” in ICCV, 2019, pp. 3603–3612.
  38. Y. Ma, S. Bai, S. An, W. Liu, A. Liu, X. Zhen, and X. Liu, “Transductive relation-propagation network for few-shot learning.” in IJCAI, vol. 20, 2020, pp. 804–810.
  39. M. Boudiaf, I. Ziko, J. Rony, J. Dolz, P. Piantanida, and I. Ben Ayed, “Information maximization for few-shot learning,” NIPS, vol. 33, pp. 2445–2457, 2020.
  40. A. Krause, P. Perona, and R. Gomes, “Discriminative clustering by regularized information maximization,” NIPS, vol. 23, 2010.
  41. W. Hu, T. Miyato, S. Tokui, E. Matsumoto, and M. Sugiyama, “Learning discrete representations via information maximizing self-augmented training,” in ICML.   PMLR, 2017, pp. 1558–1567.
  42. S. Yang, Y. Wang, K. Wang, and S. Jui, “Attracting and dispersing: A simple approach for source-free domain adaptation,” in NIPS, 2022.
  43. J. Thoma, D. P. Paudel, and L. V. Gool, “Soft contrastive learning for visual localization,” NIPS, vol. 33, pp. 11 119–11 130, 2020.
  44. J. Liang, D. Hu, and J. Feng, “Domain adaptation with auxiliary target domain-oriented classifier,” in CVPR, 2021, pp. 16 632–16 642.
  45. S. Yang, Y. Wang, J. Van De Weijer, L. Herranz, and S. Jui, “Generalized source-free domain adaptation,” in ICCV, 2021, pp. 8978–8987.
  46. S. P. Mohanty, D. P. Hughes, and M. Salathé, “Using deep learning for image-based plant disease detection,” Front. Plant Sci., vol. 7, p. 1419, 2016.
  47. P. Helber, B. Bischke, A. Dengel, and D. Borth, “Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification,” IEEE J Sel Top Appl Earth Obs Remote Sens, vol. 12, no. 7, pp. 2217–2226, 2019.
  48. P. Tschandl, C. Rosendahl, and H. Kittler, “The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Sci. Data, vol. 5, no. 1, pp. 1–9, 2018.
  49. N. Codella, V. Rotemberg, P. Tschandl, M. E. Celebi, S. Dusza, D. Gutman, B. Helba, A. Kalloo, K. Liopyris, and M. Marchetti, “Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic),” arXiv preprint arXiv:1902.03368, 2019.
  50. X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases,” in CVPR, 2017, pp. 2097–2106.
  51. O. Vinyals, C. Blundell, T. Lillicrap, and D. Wierstra, “Matching networks for one shot learning,” NIPS, vol. 29, 2016.
  52. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in ICML.   PMLR, 2017, pp. 1126–1135.
  53. F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. Torr, and T. M. Hospedales, “Learning to compare: Relation network for few-shot learning,” in CVPR, 2018, pp. 1199–1208.
  54. J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” NIPS, vol. 30, 2017.
  55. V. G. Satorras and J. B. Estrach, “Few-shot learning with graph neural networks,” in ICLR, 2018.
  56. H. Wang and Z. Deng, “Cross-domain few-shot classification via adversarial task augmentation,” IJCAI, 2021.
  57. H. Zheng, R. Wang, J. Liu, and A. Kanezaki, “Cross-level distillation and feature denoising for cross-domain few-shot classification,” in ICLR, 2023.
  58. M. Yazdanpanah and P. Moradi, “Visual domain bridge: A source-free domain adaptation for cross-domain few-shot learning,” in CVPR, 2022, pp. 2868–2877.
  59. J. Oh, S. Kim, N. Ho, J. Kim, H. Song, and S. Yun, “Refine: Re-randomization before fine-tuning for cross-domain few-shot learning,” in ACM CIKM, 2022, pp. 4359–4363.
  60. P. Li, F. Liu, L. Jiao, S. Li, L. Li, X. Liu, and X. Huang, “Knowledge transduction for cross-domain few-shot learning,” PR, vol. 141, p. 109652, 2023.
  61. K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, and Y. Xu, “A survey on vision transformer,” TPAMI, vol. 45, no. 1, pp. 87–110, 2022.
  62. Y. Hu and A. J. Ma, “Adversarial feature augmentation for cross-domain few-shot classification,” in ECCV.   Springer, 2022, pp. 20–37.
  63. D. Das, S. Yun, and F. Porikli, “Confess: A framework for single source cross-domain few-shot learning,” in ICLR, 2022.
  64. H. Liang, Q. Zhang, P. Dai, and J. Lu, “Boosting the generalization capability in cross-domain few-shot learning via noise-enhanced supervised autoencoder,” in ICCV, 2021, pp. 9424–9434.
  65. F. Zhou, P. Wang, L. Zhang, W. Wei, and Y. Zhang, “Revisiting prototypical network for cross domain few-shot learning,” in CVPR, 2023, pp. 20 061–20 070.
  66. A. v. d. Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748, 2018.
  67. W. Zhou, S. Newsam, C. Li, and Z. Shao, “Patternnet: A benchmark dataset for performance evaluation of remote sensing image retrieval,” ISPRS journal of photogrammetry and remote sensing, vol. 145, pp. 197–209, 2018.
  68. M. Jia, L. Tang, B. Chen, C. Cardie, S. Belongie, B. Hariharan, and S. Lim, “Visual prompt tuning,” in ECCV.   Springer, 2022, pp. 709–727.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Huali Xu (6 papers)
  2. Li Liu (311 papers)
  3. Shuaifeng Zhi (15 papers)
  4. Shaojing Fu (6 papers)
  5. Zhuo Su (32 papers)
  6. Ming-Ming Cheng (185 papers)
  7. Yongxiang Liu (27 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now