Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cross-Domain Few-Shot Semantic Segmentation via Doubly Matching Transformation (2405.15265v1)

Published 24 May 2024 in cs.CV

Abstract: Cross-Domain Few-shot Semantic Segmentation (CD-FSS) aims to train generalized models that can segment classes from different domains with a few labeled images. Previous works have proven the effectiveness of feature transformation in addressing CD-FSS. However, they completely rely on support images for feature transformation, and repeatedly utilizing a few support images for each class may easily lead to overfitting and overlooking intra-class appearance differences. In this paper, we propose a Doubly Matching Transformation-based Network (DMTNet) to solve the above issue. Instead of completely relying on support images, we propose Self-Matching Transformation (SMT) to construct query-specific transformation matrices based on query images themselves to transform domain-specific query features into domain-agnostic ones. Calculating query-specific transformation matrices can prevent overfitting, especially for the meta-testing stage where only one or several images are used as support images to segment hundreds or thousands of images. After obtaining domain-agnostic features, we exploit a Dual Hypercorrelation Construction (DHC) module to explore the hypercorrelations between the query image with the foreground and background of the support image, based on which foreground and background prediction maps are generated and supervised, respectively, to enhance the segmentation result. In addition, we propose a Test-time Self-Finetuning (TSF) strategy to more accurately self-tune the query prediction in unseen domains. Extensive experiments on four popular datasets show that DMTNet achieves superior performance over state-of-the-art approaches. Code is available at https://github.com/ChenJiayi68/DMTNet.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Few-shot segmentation without meta-learning: A good transductive inference is all you need? 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13974–13983, 2020.
  2. Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Transactions on Medical Imaging, 33:577–590, 2014.
  3. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, 2017.
  4. Dense affinity matching for few-shot segmentation, 2023.
  5. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). ArXiv, abs/1902.03368, 2019.
  6. Deepglobe 2018: A challenge to parse the earth through satellite images. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 172–17209, 2018.
  7. Self-support few-shot semantic segmentation. In European Conference on Computer Vision, 2022.
  8. Cycada: Cycle-consistent adversarial domain adaptation. ArXiv, abs/1711.03213, 2017.
  9. Fsdr: Frequency space domain randomization for domain generalization. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6887–6898, 2021.
  10. Restnet: Boosting cross-domain few-shot segmentation with residual transformation network, 2023.
  11. Cross-domain few-shot semantic segmentation. In European Conference on Computer Vision, 2022.
  12. Adaptive prototype learning and allocation for few-shot segmentation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8330–8339, 2021.
  13. Crnet: Cross-reference networks for few-shot segmentation. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4164–4172, 2020.
  14. Part-aware prototype network for few-shot semantic segmentation. ArXiv, abs/2007.06309, 2020.
  15. Fully convolutional networks for semantic segmentation, 2015.
  16. Conditional adversarial domain adaptation. In Neural Information Processing Systems, 2017.
  17. Cross-domain few-shot segmentation with transductive fine-tuning. ArXiv, abs/2211.14745, 2022.
  18. Hypercorrelation squeeze for few-shot segmenation. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 6921–6932, 2021.
  19. Atsuro Okazawa. Interclass prototype relation for few-shot segmentation. In European Conference on Computer Vision, 2022.
  20. Switchable whitening for deep representation learning. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1863–1871, 2019.
  21. Global and local texture randomization for synthetic-to-real semantic segmentation. IEEE Transactions on Image Processing, 30:6594–6608, 2021.
  22. Semantic-aware domain generalized segmentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2584–2595, 2022.
  23. Conditional networks for few-shot semantic segmentation. In International Conference on Learning Representations, 2018.
  24. Playing for data: Ground truth from computer games. ArXiv, abs/1608.02192, 2016.
  25. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3234–3243, 2016.
  26. Task-adaptive feature transformer for few-shot segmentation. ArXiv, abs/2010.11437, 2020.
  27. One-shot learning for semantic segmentation. ArXiv, abs/1709.03410, 2017.
  28. Amp: Adaptive masked proxies for few-shot segmentation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 5248–5257, 2019.
  29. Indoor segmentation and support inference from rgbd images. In European Conference on Computer Vision, 2012.
  30. Prototypical networks for few-shot learning. In Neural Information Processing Systems, 2017.
  31. Pixel-by-pixel cross-domain alignment for few-shot semantic segmentation. 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1959–1968, 2021.
  32. Prior guided feature enrichment network for few-shot segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44:1050–1065, 2020.
  33. Matching networks for one shot learning. In Neural Information Processing Systems, 2016.
  34. Panet: Few-shot image semantic segmentation with prototype alignment. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9196–9205, 2019.
  35. Remember the difference: Cross-domain few-shot semantic segmentation via meta-memory transfer. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7055–7064, 2022.
  36. Fss-1000: A 1000-class dataset for few-shot segmentation. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2866–2875, 2019.
  37. Segformer: Simple and efficient design for semantic segmentation with transformers. In Neural Information Processing Systems, 2021.
  38. Prototype mixture models for few-shot semantic segmentation. ArXiv, abs/2008.03898, 2020.
  39. Sg-one: Similarity guidance network for one-shot semantic segmentation. IEEE Transactions on Cybernetics, 50:3855–3865, 2018.
  40. Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9586–9594, 2019.
  41. Canet: Class-agnostic segmentation networks with iterative refinement and attentive few-shot learning. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5212–5221, 2019.
  42. Few-shot segmentation via cycle-consistent transformer. In Neural Information Processing Systems, 2021.
  43. Pyramid scene parsing network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6230–6239, 2016.
  44. Domain adaptation for semantic segmentation via class-balanced self-training. ArXiv, abs/1810.07911, 2018.
  45. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In European Conference on Computer Vision, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Jiayi Chen (63 papers)
  2. Rong Quan (5 papers)
  3. Jie Qin (68 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets