Papers
Topics
Authors
Recent
Search
2000 character limit reached

Contrastive Learning and Cycle Consistency-based Transductive Transfer Learning for Target Annotation

Published 22 Jan 2024 in cs.CV, cs.AI, cs.LG, eess.IV, and stat.ML | (2401.12340v1)

Abstract: Annotating automatic target recognition (ATR) is a highly challenging task, primarily due to the unavailability of labeled data in the target domain. Hence, it is essential to construct an optimal target domain classifier by utilizing the labeled information of the source domain images. The transductive transfer learning (TTL) method that incorporates a CycleGAN-based unpaired domain translation network has been previously proposed in the literature for effective ATR annotation. Although this method demonstrates great potential for ATR, it severely suffers from lower annotation performance, higher Fr\'echet Inception Distance (FID) score, and the presence of visual artifacts in the synthetic images. To address these issues, we propose a hybrid contrastive learning base unpaired domain translation (H-CUT) network that achieves a significantly lower FID score. It incorporates both attention and entropy to emphasize the domain-specific region, a noisy feature mixup module to generate high variational synthetic negative patches, and a modulated noise contrastive estimation (MoNCE) loss to reweight all negative patches using optimal transport for better performance. Our proposed contrastive learning and cycle-consistency-based TTL (C3TTL) framework consists of two H-CUT networks and two classifiers. It simultaneously optimizes cycle-consistency, MoNCE, and identity losses. In C3TTL, two H-CUT networks have been employed through a bijection mapping to feed the reconstructed source domain images into a pretrained classifier to guide the optimal target domain classifier. Extensive experimental analysis conducted on three ATR datasets demonstrates that the proposed C3TTL method is effective in annotating civilian and military vehicles, as well as ship targets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. N. M. Nasrabadi, “DeepTarget: An automatic target recognition using deep convolutional neural networks,” IEEE Transactions on Aerospace and Electronic Systems, vol. 55, no. 6, pp. 2687–2697, 2019.
  2. J. Chen, L. Du, H. He, and Y. Guo, “Convolutional factor analysis model with application to radar automatic target recognition,” Pattern Recognition, vol. 87, pp. 140–156, 2019.
  3. K. El-Darymli, E. W. Gill, P. Mcguire, D. Power, and C. Moloney, “Automatic target recognition in synthetic aperture radar imagery: A state-of-the-art review,” IEEE access, vol. 4, pp. 6014–6058, 2016.
  4. J. M. Topple and J. A. Fawcett, “MiNet: Efficient deep learning automatic target recognition for small autonomous vehicles,” IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 6, pp. 1014–1018, 2020.
  5. S. Dang, Z. Cao, Z. Cui, Y. Pi, and N. Liu, “Open set incremental learning for automatic target recognition,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 7, pp. 4445–4456, 2019.
  6. P. Y. Simard, D. Steinkraus, J. C. Platt et al., “Best practices for convolutional neural networks applied to visual document analysis.” in Icdar, vol. 3, no. 2003.   Edinburgh, 2003.
  7. T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning.   PMLR, 2020, pp. 1597–1607.
  8. T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. E. Hinton, “Big self-supervised models are strong semi-supervised learners,” Advances in neural information processing systems, vol. 33, pp. 22 243–22 255, 2020.
  9. S. M. Sami, N. M. Nasrabadi, and R. Rao, “Deep transductive transfer learning for automatic target recognition,” in Automatic Target Recognition XXXIII, vol. 12521.   SPIE, 2023, pp. 31–40.
  10. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223–2232.
  11. J. Han, M. Shoeiby, L. Petersson, and M. A. Armin, “Dual contrastive learning for unsupervised image-to-image translation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 746–755.
  12. T. Park, A. A. Efros, R. Zhang, and J.-Y. Zhu, “Contrastive learning for unpaired image-to-image translation,” in European conference on computer vision.   Springer, 2020, pp. 319–345.
  13. X. Hu, X. Zhou, Q. Huang, Z. Shi, L. Sun, and Q. Li, “QS-Attn: Query-selected attention for contrastive learning in I2I translation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 291–18 300.
  14. F. Zhan, J. Zhang, Y. Yu, R. Wu, and S. Lu, “Modulated contrast for versatile image synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 280–18 290.
  15. Y. Kalantidis, M. B. Sariyildiz, N. Pion, P. Weinzaepfel, and D. Larlus, “Hard negative mixing for contrastive learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 21 798–21 809, 2020.
  16. S. H. Lim, N. B. Erichson, F. Utrera, W. Xu, and M. W. Mahoney, “Noisy feature mixup,” arXiv preprint arXiv:2110.02180, 2021.
  17. Us army night vision and electronic sensors directorate (NVESD). [Online]. Available: https://dsiac.org/databases/atr-algorithm-development-image-database/
  18. J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar et al., “Bootstrap your own latent-a new approach to self-supervised learning,” Advances in neural information processing systems, vol. 33, pp. 21 271–21 284, 2020.
  19. M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin, “Unsupervised learning of visual features by contrasting cluster assignments,” Advances in neural information processing systems, vol. 33, pp. 9912–9924, 2020.
  20. J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny, “Barlow twins: Self-supervised learning via redundancy reduction,” in International Conference on Machine Learning.   PMLR, 2021, pp. 12 310–12 320.
  21. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 580–587.
  22. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single shot multibox detector,” in European conference on computer vision.   Springer, 2016, pp. 21–37.
  23. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
  24. V. Vs, D. Poster, S. You, S. Hu, and V. M. Patel, “Meta-UDA: Unsupervised domain adaptive thermal object detection using meta-learning,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 1412–1423.
  25. C. Zheng, X. Jiang, and X. Liu, “Semi-supervised SAR ATR via multi-discriminator generative adversarial network,” IEEE Sensors Journal, vol. 19, no. 17, pp. 7525–7533, 2019.
  26. F. Zhang, C. Hu, Q. Yin, W. Li, H. Li, and W. Hong, “SAR target recognition using the multi-aspect-aware bidirectional LSTM recurrent neural networks,” arXiv preprint arXiv:1707.09875, 2017.
  27. S. Deng, L. Du, C. Li, J. Ding, and H. Liu, “SAR automatic target recognition based on Euclidean distance restricted autoencoder,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 10, no. 7, pp. 3323–3333, 2017.
  28. V. M. Patel, N. M. Nasrabadi, and R. Chellappa, “Sparsity-motivated automatic target recognition,” Applied optics, vol. 50, no. 10, pp. 1425–1433, 2011.
  29. J. Ding, B. Chen, H. Liu, and M. Huang, “Convolutional neural network with data augmentation for SAR target recognition,” IEEE Geoscience and remote sensing letters, vol. 13, no. 3, pp. 364–368, 2016.
  30. J. Wang, T. Zheng, P. Lei, and X. Bai, “Ground target classification in noisy SAR images using convolutional neural networks,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 11, pp. 4180–4192, 2018.
  31. L. Wang, X. Bai, R. Xue, and F. Zhou, “Few-shot SAR automatic target recognition based on Conv-BiLSTM prototypical network,” Neurocomputing, vol. 443, pp. 235–246, 2021.
  32. L. Wang, X. Bai, C. Gong, and F. Zhou, “Hybrid inference network for few-shot SAR automatic target recognition,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 11, pp. 9257–9269, 2021.
  33. K. Fu, T. Zhang, Y. Zhang, Z. Wang, and X. Sun, “Few-shot SAR target classification via metalearning,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–14, 2021.
  34. R. M. Marcacini, R. G. Rossi, I. P. Matsuno, and S. O. Rezende, “Cross-domain aspect extraction for sentiment analysis: A transductive learning approach,” Decis. Support Syst., vol. 114, pp. 70–80, Oct. 2018.
  35. Y. He, J. Yuan, and L. Li, “Enhancing RNN based OCR by transductive transfer learning from text to images,” AAAI, vol. 32, no. 1, Apr. 2018.
  36. Y. Zong, W. Zheng, X. Huang, K. Yan, J. Yan, and T. Zhang, “Emotion recognition in the wild via sparse transductive transfer linear discriminant analysis,” Journal on Multimodal User Interfaces, vol. 10, no. 2, pp. 163–172, Jun. 2016.
  37. K. Yan, W. Zheng, T. Zhang, Y. Zong, C. Tang, C. Lu, and Z. Cui, “Cross-domain facial expression recognition based on transductive deep transfer learning,” IEEE Access, vol. 7, pp. 108 906–108 915, 2019.
  38. J. Kobylarz, J. J. Bird, D. R. Faria, E. P. Ribeiro, and A. Ekárt, “Thumbs up, thumbs down: non-verbal human-robot interaction through real-time EMG classification via inductive and supervised transductive transfer learning,” J. Ambient Intell. Humaniz. Comput., vol. 11, no. 12, pp. 6021–6031, Dec. 2020.
  39. W. Fu, B. Xue, M. Zhang, and X. Gao, “Transductive transfer learning in genetic programming for document classification,” in Simulated Evolution and Learning.   Springer International Publishing, 2017, pp. 556–568.
  40. A. Moreo, A. Esuli, and F. Sebastiani, “Lost in transduction: Transductive transfer learning in text classification,” ACM Trans. Knowl. Discov. Data, vol. 16, no. 1, pp. 1–21, Jul. 2021.
  41. Y. Luo, Z. Zhang, L. Zhang, J. Han, J. Cao, and J. Zhang, “Developing high-resolution crop maps for major crops in the european union based on transductive transfer learning and limited ground data,” Remote Sensing, vol. 14, p. 1809, Apr. 2022.
  42. L. V. Utkin and M. A. Ryabinin, “A deep forest for transductive transfer learning by using a consensus measure,” ser. Communications in computer and information science.   Springer International Publishing, 2018, pp. 194–208.
  43. X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz, “Multimodal unsupervised image-to-image translation,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 172–189.
  44. H.-Y. Lee, H.-Y. Tseng, Q. Mao, J.-B. Huang, Y.-D. Lu, M. Singh, and M.-H. Yang, “DRIT++: Diverse image-to-image translation via disentangled representations,” International Journal of Computer Vision, vol. 128, no. 10, pp. 2402–2417, 2020.
  45. M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised image-to-image translation networks,” Advances in neural information processing systems, vol. 30, 2017.
  46. S. Benaim and L. Wolf, “One-sided unsupervised domain mapping,” Advances in neural information processing systems, vol. 30, 2017.
  47. M. Amodio and S. Krishnaswamy, “TraVeLGAN: Image-to-image translation by transformation vector learning,” in Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 2019, pp. 8983–8992.
  48. H. Fu, M. Gong, C. Wang, K. Batmanghelich, K. Zhang, and D. Tao, “Geometry-consistent generative adversarial networks for one-sided unsupervised domain mapping,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2427–2436.
  49. W. Wang, W. Zhou, J. Bao, D. Chen, and H. Li, “Instance-wise hard negative example generation for contrastive learning in unpaired image-to-image translation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14 020–14 029.
  50. C. Zheng, T.-J. Cham, and J. Cai, “The spatially-correlative loss for various image translation tasks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16 407–16 417.
  51. M. Zhao, F. Bao, C. Li, and J. Zhu, “EGSDE: Unpaired image-to-image translation via energy-guided stochastic differential equations,” Advances in Neural Information Processing Systems, vol. 35, pp. 3609–3623, 2022.
  52. S. Sun, L. Wei, J. Xing, J. Jia, and Q. Tian, “SDDM: Score-decomposed diffusion models on manifolds for unpaired image-to-image translation,” in Proceedings of the 40th International Conference on Machine Learning, vol. 202.   PMLR, 23–29 Jul 2023, pp. 33 115–33 134.
  53. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
  54. J. Robinson, C.-Y. Chuang, S. Sra, and S. Jegelka, “Contrastive learning with hard negative samples,” arXiv preprint arXiv:2010.04592, 2020.
  55. G. Peyré, M. Cuturi et al., “Computational optimal transport: With applications to data science,” Foundations and Trends® in Machine Learning, vol. 11, no. 5-6, pp. 355–607, 2019.
  56. M. Cuturi, “Sinkhorn distances: Lightspeed computation of optimal transport,” Advances in neural information processing systems, vol. 26, 2013.
  57. X. Chen, C. Xu, X. Yang, and D. Tao, “Attention-GAN for object transfiguration in wild images,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 164–180.
  58. V. Verma, A. Lamb, C. Beckham, A. Najafi, I. Mitliagkas, D. Lopez-Paz, and Y. Bengio, “Manifold mixup: Better representations by interpolating hidden states,” in International conference on machine learning.   PMLR, 2019, pp. 6438–6447.
  59. V. Verma, K. Kawaguchi, A. Lamb, J. Kannala, A. Solin, Y. Bengio, and D. Lopez-Paz, “Interpolation consistency training for semi-supervised learning,” Neural Networks, vol. 145, pp. 90–106, 2022.
  60. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  61. M. M. Zhang, J. Choi, K. Daniilidis, M. T. Wolf, and C. Kanan, “VAIS: A dataset for recognizing maritime imagery in the visible and infrared spectrums,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015, pp. 10–16.
  62. A. Mehmood and N. M. Nasrabadi, “Anomaly detection for longwave FLIR imagery using kernel Wavelet-RX,” in 2010 20th International Conference on Pattern Recognition.   IEEE, 2010, pp. 1385–1388.
  63. Cyclegan summer-winter image translation[pytorch]. [Online]. Available: http://www.kaggle.com/code/balraj98/cyclegan-summer-winter-image-translation-nvesd/data/
  64. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  65. A. e. a. Paszke, “PyTorch: An imperative style, high-performance deep learning library,” in Advances in Neural Information Processing Systems 32.   Curran Associates, Inc., 2019, pp. 8024–8035.
  66. M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “GANs trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems, vol. 30, 2017.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free