Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Robust Unsupervised Domain Adaptation by Retaining Confident Entropy via Edge Concatenation (2310.07149v1)

Published 11 Oct 2023 in cs.CV

Abstract: The generalization capability of unsupervised domain adaptation can mitigate the need for extensive pixel-level annotations to train semantic segmentation networks by training models on synthetic data as a source with computer-generated annotations. Entropy-based adversarial networks are proposed to improve source domain prediction; however, they disregard significant external information, such as edges, which have the potential to identify and distinguish various objects within an image accurately. To address this issue, we introduce a novel approach to domain adaptation, leveraging the synergy of internal and external information within entropy-based adversarial networks. In this approach, we enrich the discriminator network with edge-predicted probability values within this innovative framework to enhance the clarity of class boundaries. Furthermore, we devised a probability-sharing network that integrates diverse information for more effective segmentation. Incorporating object edges addresses a pivotal aspect of unsupervised domain adaptation that has frequently been neglected in the past -- the precise delineation of object boundaries. Conventional unsupervised domain adaptation methods usually center around aligning feature distributions and may not explicitly model object boundaries. Our approach effectively bridges this gap by offering clear guidance on object boundaries, thereby elevating the quality of domain adaptation. Our approach undergoes rigorous evaluation on the established unsupervised domain adaptation benchmarks, specifically in adapting SYNTHIA $\rightarrow$ Cityscapes and SYNTHIA $\rightarrow$ Mapillary. Experimental results show that the proposed model attains better performance than state-of-the-art methods. The superior performance across different unsupervised domain adaptation scenarios highlights the versatility and robustness of the proposed method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (89)
  1. Agrisegnet: Deep aerial semantic segmentation framework for iot-assisted precision agriculture. IEEE Sensors Journal, 21, 17581–17590. https://doi.org/10.1109/JSEN.2021.3071290.
  2. Deep semantic segmentation of natural and medical images: a review. Artificial Intelligence Review, 54, 137–178. https://doi.org/10.1007/s10462-020-09854-1.
  3. Deepedge: A multi-scale bifurcated deep network for top-down contour detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4380–4389).
  4. Adabins: Depth estimation using adaptive bins. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4009–4018).
  5. Scene labeling with lstm recurrent neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3547–3555).
  6. Dlnet with training task conversion stream for precise semantic segmentation in actual traffic scene. IEEE Transactions on Neural Networks and Learning Systems, 33, 6443–6457. https://doi.org/10.1109/TNNLS.2021.3080261.
  7. Adaptive refining-aggregation-separation framework for unsupervised domain adaptation semantic segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 33, 3822–3832. https://doi.org/10.1109/TCSVT.2023.3243402.
  8. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 834–848. https://doi.org/10.1109/TPAMI.2017.2699184.
  9. Domain adaptation for semantic segmentation with maximum squares loss. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2090–2099).
  10. Enhanced feature alignment for unsupervised domain adaptation of semantic segmentation. IEEE Transactions on Multimedia, 24, 1042–1054. https://doi.org/10.1109/TMM.2021.3106095.
  11. Learning semantic segmentation from synthetic data: A geometrically guided input-output adaptation approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1841–1850).
  12. No more discrimination: Cross city adaptation of road scene segmenters. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1992–2001).
  13. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3213–3223).
  14. Cscl: Critical semantic-consistent learning for unsupervised domain adaptation. In Proceedings of the European Conference on Computer Vision (pp. 745–762). Springer.
  15. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, . https://doi.org/10.48550/arXiv.2010.11929.
  16. Ssf-dan: Separated semantic feature based domain adaptation network for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 982–991).
  17. Deep ordinal regression network for monocular depth estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2002–2011).
  18. Unsupervised domain adaptation by backpropagation. In Proceedings of the International Conference on Machine Learning (pp. 1180–1189). PMLR.
  19. Cross-domain correlation distillation for unsupervised domain adaptation in nighttime semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9913–9923).
  20. Addressing domain gap via content invariant representation for semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence (pp. 7528–7536). volume 35.
  21. Generative adversarial networks. Communications of the ACM, 63, 139–144. https://doi.org/10.1145/3422622.
  22. Sotr: Segmenting objects with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 7157–7166).
  23. Metacorrection: Domain-aware meta loss correction for unsupervised domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3927–3936).
  24. Deep learning for 3d point clouds: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 4338–4364. https://doi.org/10.1109/TPAMI.2020.3005434.
  25. Semantic contours from inverse detectors. In 2011 International Conference on Computer Vision (pp. 991–998). IEEE.
  26. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778).
  27. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2. https://doi.org/10.48550/arXiv.1503.02531.
  28. Cycada: Cycle-consistent adversarial domain adaptation. In Proceedings of the International Conference on Machine Learning (ICML) (pp. 1989–1998). PMLR.
  29. Fcns in the wild: Pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649, . https://doi.org/10.48550/arXiv.1612.02649.
  30. Adversarial learning for semi-supervised semantic segmentation. arXiv preprint arXiv:1802.07934, . https://doi.org/10.48550/arXiv.1802.07934.
  31. Jiang, J. (2008). A literature survey on domain adaptation of statistical classifiers. URL: http://sifaka.cs.uiuc.edu/jiang4/domainadaptation/survey, 3, 3.
  32. Prototypical contrast adaptation for domain adaptive semantic segmentation. In Proceedings of the European Conference on Computer Vision (pp. 36–54). Springer.
  33. Weakly supervised object boundaries. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 183–192).
  34. Continual batchnorm adaptation (cbna) for semantic segmentation. IEEE Transactions on Intelligent Transportation Systems, 23, 20899–20911. https://doi.org/10.1109/TITS.2022.3190263.
  35. Constraining pseudo-label in self-training unsupervised domain adaptation with energy-based model. International Journal of Intelligent Systems, 37, 8092–8112. https://doi.org/10.1002/int.22930.
  36. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60, 84–90. https://doi.org/10.1145/3065386.
  37. Semantic segmentation of uav images based on transformer framework with context information. Mathematics, 10, 4735. https://doi.org/10.3390/math10244735.
  38. Lee, D.-G. (2021). Fast drivable areas estimation with multi-task learning for real-time autonomous driving assistant. Applied Sciences, 11, 10713. https://doi.org/10.3390/app112210713.
  39. Joint semantic understanding with a multilevel branch for driving perception. Applied Sciences, 12, 2877. https://doi.org/10.3390/app12062877.
  40. Spigan: Privileged adversarial learning from simulation. International Conference on Learning Representations, .
  41. Feature re-representation and reliable pseudo label retraining for cross-domain semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, . https://doi.org/10.1109/TPAMI.2022.3154933.
  42. Constructing self-motivated pyramid curriculums for cross-domain semantic segmentation: A non-adversarial approach. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 6758–6767).
  43. Semantic object parsing with graph lstm. In Proceedings of the European Conference on Computer Vision (pp. 125–143). Springer.
  44. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3431–3440).
  45. Learning transferable features with deep adaptation networks. In Proceedings of the International Conference on Machine Learning (pp. 97–105). PMLR.
  46. Cross-domain semantic segmentation via domain-invariant interactive relation transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4334–4343).
  47. Deep retinal image understanding. In International Conference on Medical Image Computing and Computer-assisted Intervention (pp. 140–148). Springer.
  48. Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44, 3523–3542. https://doi.org/10.1109/TPAMI.2021.3059968.
  49. Image to image translation for domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4500–4509).
  50. Semantically adaptive image-to-image translation for domain adaptation of semantic segmentation. arXiv preprint arXiv:2009.01166, . https://doi.org/10.48550/arXiv.1511.06434.
  51. The mapillary vistas dataset for semantic understanding of street scenes. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) (pp. 4990–4999).
  52. Unsupervised intra-domain adaptation for semantic segmentation through self-supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3764–3773).
  53. Preserving semantic and temporal consistency for unpaired video-to-video translation. In Proceedings of the 27th ACM International Conference on Multimedia (pp. 1248–1257).
  54. Dense extreme inception network: Towards a robust cnn model for edge detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 1923–1932).
  55. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, . https://doi.org/10.48550/arXiv.1511.06434.
  56. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 12179–12188).
  57. Playing for data: Ground truth from computer games. In Proceedings of the European Conference on Computer Vision (pp. 102–118). Springer.
  58. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-assisted Intervention (pp. 234–241). Springer.
  59. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3234–3243).
  60. Learning to relate depth and semantics for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8197–8207).
  61. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3723–3732).
  62. Esl: Entropy-guided self-supervised learning for domain adaptation in semantic segmentation. arXiv preprint arXiv:2006.08658, . https://doi.org/10.48550/arXiv.2006.08658.
  63. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, . https://doi.org/10.48550/arXiv.1409.1556.
  64. Edge detection in gray level images based on the shannon entropy. Journal of Computer Science, 4, 186–191. https://doi.org/10.3844/jcssp.2008.186.191.
  65. Unsupervised model adaptation for continual semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence (pp. 2593–2601). volume 35.
  66. Segmenter: Transformer for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 7262–7272).
  67. Learning from scale-invariant examples for domain adaptation in semantic segmentation. In Proceedings of the European Conference on Computer Vision (pp. 290–306). Springer.
  68. Gated-scnn: Gated shape cnns for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 5229–5238).
  69. Unsupervised domain adaptation in semantic segmentation via orthogonal and clustered embeddings. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 1358–1368).
  70. Learning to adapt structured output space for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7472–7481).
  71. Domain adaptation for structured output via discriminative patch representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 1456–1465).
  72. Reseg: A recurrent neural network-based model for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 41–48).
  73. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2517–2526).
  74. Dada: Depth-aware domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 7364–7373).
  75. Classes matter: A fine-grained adversarial approach to cross-domain semantic segmentation. In Proceedings of the European Conference on Computer Vision (pp. 642–659). Springer.
  76. Segformer: Simple and efficient design for semantic segmentation with transformers. In Advances in Neural Information Processing Systems (pp. 12077–12090). Curran Associates, Inc. volume 34.
  77. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122, . https://doi.org/10.48550/arXiv.1511.0712.
  78. Dast: Unsupervised domain adaptation in semantic segmentation based on discriminator attention and self-training. In Proceedings of the AAAI Conference on Artificial Intelligence (pp. 10754–10762). volume 35.
  79. Casenet: Deep category-aware semantic edge detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5964–5973).
  80. Simultaneous edge alignment and learning. In Proceedings of the European Conference on Computer Vision (pp. 388–404).
  81. Confidence-and-refinement adaptation model for cross-domain semantic segmentation. IEEE Transactions on Intelligent Transportation Systems, 23, 9529–9542. https://doi.org/10.1109/TITS.2022.3140481.
  82. Zhang, Y. (2021). A survey of unsupervised domain adaptation for visual recognition. arXiv preprint arXiv:2112.06745, . https://doi.org/10.48550/arXiv.2112.06745.
  83. A curriculum domain adaptation approach to the semantic segmentation of urban scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 1823–1841. https://doi.org/10.1109/TPAMI.2019.2903401.
  84. Curriculum domain adaptation for semantic segmentation of urban scenes. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2020–2030).
  85. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2881–2890).
  86. Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems, 30, 3212–3232. https://doi.org/10.1109/TNNLS.2018.2876865.
  87. Affinity space adaptation for semantic segmentation across domains. IEEE Transactions on Image Processing, 30, 2549–2561. https://doi.org/10.1109/TIP.2020.3018221.
  88. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2223–2232).
  89. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European Conference on Computer Vision (pp. 289–305).
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now