Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Online Continual Domain Adaptation for Semantic Image Segmentation Using Internal Representations (2401.01035v1)

Published 2 Jan 2024 in cs.CV

Abstract: Semantic segmentation models trained on annotated data fail to generalize well when the input data distribution changes over extended time period, leading to requiring re-training to maintain performance. Classic Unsupervised domain adaptation (UDA) attempts to address a similar problem when there is target domain with no annotated data points through transferring knowledge from a source domain with annotated data. We develop an online UDA algorithm for semantic segmentation of images that improves model generalization on unannotated domains in scenarios where source data access is restricted during adaptation. We perform model adaptation is by minimizing the distributional distance between the source latent features and the target features in a shared embedding space. Our solution promotes a shared domain-agnostic latent feature space between the two domains, which allows for classifier generalization on the target dataset. To alleviate the need of access to source samples during adaptation, we approximate the source latent feature distribution via an appropriate surrogate distribution, in this case a Gassian mixture model (GMM). We evaluate our approach on well established semantic segmentation datasets and demonstrate it compares favorably against state-of-the-art (SOTA) UDA semantic segmentation methods.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (94)
  1. Wasserstein GAN. arXiv preprint arXiv:1701.07875
  2. Unsupervised domain adaptation using generative adversarial networks for semantic segmentation of aerial images. Remote Sensing 11, 1369
  3. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of the European Conference on Computer Vision (ECCV). 447–463
  4. Optimization methods for large-scale machine learning. Siam Review 60, 223–311
  5. Unsupervised pixel-level domain adaptation with generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3722–3731
  6. Semantic-aware generative adversarial nets for unsupervised domain adaptation in chest x-ray segmentation. In International workshop on machine learning in medical imaging (Springer), 143–151
  7. Rethinking atrous convolution for semantic image segmentation. In arXiv:1706.05587 [cs.CV]
  8. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV
  9. Learning semantic segmentation from synthetic data: A geometrically guided input-output adaptation approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1841–1850
  10. No more discrimination: Cross city adaptation of road scene segmenters. In arXiv:1704.08509 [cs.CV]
  11. Self-ensembling with gan-based data augmentation for domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 6830–6840
  12. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3213–3223
  13. Online methods for multi-domain learning and adaptation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (Association for Computational Linguistics), 689–697
  14. Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems 22, 1341–1360
  15. Learning a domain-invariant embedding for unsupervised domain adaptation using class-conditioned distribution alignment. In 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton). 352–359
  16. Deep Learning (MIT Press). http://www.deeplearningbook.org
  17. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672–2680
  18. Generative adversarial networks. Communications of the ACM 63, 139–144
  19. Domain adaptation for medical image analysis: a survey. IEEE Transactions on Biomedical Engineering 69, 1173–1185
  20. Deep residual learning for image recognition. CoRR abs/1512.03385
  21. Cycada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning (Pmlr), 1989–1998
  22. CyCADA: Cycle-consistent adversarial domain adaptation. In International Conference on Machine Learning. 1989–1998
  23. Fcns in the wild: Pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649
  24. Conditional generative adversarial network for structured domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1335–1344
  25. Adversarial learning for semi-supervised semantic segmentation. arXiv preprint arXiv:1802.07934
  26. Online domain adaptation of a pre-trained cascade of classifiers. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. 577–584
  27. Unsupervised domain adaptation for training event-based networks using contrastive learning and uncorrelated conditioning. arXiv preprint arXiv:2303.12424
  28. Analysis based on recent deep learning approaches applied in real-time multi-object tracking: a review. IEEE Access 9, 32650–32671
  29. Domain adaptation without source data. IEEE Transactions on Artificial Intelligence 2, 508–518
  30. Generalized sliced wasserstein distances. Advances in neural information processing systems 32
  31. Sliced wasserstein kernels for probability distributions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5258–5267
  32. Generalize then adapt: Source-free domain adaptive semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 7046–7056
  33. Universal source-free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4544–4553
  34. Universal source-free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  35. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks 3361, 1995
  36. Sliced wasserstein discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 10285–10295
  37. Enhanced transport distance for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13936–13944
  38. Multi-site fmri analysis using privacy-preserving federated learning and domain adaptation: Abide results. Medical Image Analysis 65, 101765
  39. Feature pyramid networks for object detection
  40. Source-free domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1215–1224
  41. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431–3440
  42. Semantic segmentation using adversarial networks. In NIPS Workshop on Adversarial Training
  43. UMAP: Uniform manifold approximation and projection. Journal of Open Source Software 3, 861
  44. Image to image translation for domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
  45. Theoretical analysis of domain adaptation with optimal transport. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (Springer), 737–753
  46. Playing for data: Ground truth from computer games. In European conference on computer vision (Springer), 102–118
  47. Bridging the day and night domain gap for semantic segmentation. In 2019 IEEE Intelligent Vehicles Symposium (IV) (IEEE), 1312–1318
  48. U-net: Convolutional networks for biomedical image segmentation
  49. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3234–3243
  50. Rostami, M. (2019). Learning Transferable Knowledge Through Embedding Spaces. Ph.D. thesis, University of Pennsylvania
  51. Rostami, M. (2021). Lifelong domain adaptation via consolidated internal distribution. Advances in neural information processing systems 34, 11172–11183
  52. Rostami, M. (2022). Increasing model generalizability for unsupervised visual domain adaptation. In Conference on Lifelong Learning Agents (PMLR), 281–293
  53. Domain adaptation for sentiment analysis using robust internal representations. In Findings of the Association for Computational Linguistics: EMNLP 2023. 11484–11498
  54. Overcoming concept shift in domain-aware settings through consolidated internal distributions. In Proceedings of the AAAI conference on artificial intelligence. vol. 37, 9623–9631
  55. Thirty-second international joint conference on artificial intelligence. In arXiv preprint arXiv:2110.04662
  56. A crowdsourcing triage algorithm for geopolitical event forecasting. In Proceedings of the 12th ACM Conference on Recommender Systems. 377–381
  57. Deep transfer learning for few-shot sar image classification. Remote Sensing 11, 1374
  58. Sar image classification using few-shot cross-domain transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 0–0
  59. Generative continual concept learning. In Proceedings of the AAAI conference on artificial intelligence. vol. 34, 5545–5552
  60. Complementary learning for overcoming catastrophic forgetting using experience replay. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (AAAI Press), 3339–3345
  61. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3723–3732
  62. Model adaptation with synthetic and real data for semantic dense foggy scene understanding. In Proceedings of the European Conference on Computer Vision (ECCV). 687–704
  63. Sf-uda 3d: Source-free unsupervised domain adaptation for lidar-based 3d object detection. In 2020 International Conference on 3D Vision (3DV) (IEEE), 771–780
  64. Learning from synthetic data: Addressing domain shift for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3752–3761
  65. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
  66. Convolutional wasserstein distances: Efficient optimal transportation on geometric domains. ACM Transactions on Graphics (TOG) 34, 66
  67. Privacy preserving domain adaptation for semantic segmentation of medical images. arXiv preprint arXiv:2101.00522 1
  68. Unsupervised model adaptation for continual semantic segmentation. In Proceedings of the AAAI conference on artificial intelligence. vol. 35, 2593–2601
  69. Secure domain adaptation with multiple sources. Transactions on Machine Learning Research
  70. Segmenter: Transformer for semantic segmentation
  71. Aerial-pass: Panoramic annular scene segmentation in drone videos. In 2021 European Conference on Mobile Robots (ECMR). 1–6. 10.1109/ECMR50962.2021.9568802
  72. Hierarchical multi-scale attention for semantic segmentation
  73. Wasserstein auto-encoders. arXiv preprint arXiv:1711.01558
  74. Learning to adapt structured output space for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7472–7481
  75. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2517–2526
  76. Axial-deeplab: Stand-alone axial-attention for panoptic segmentation. In European Conference on Computer Vision (ECCV)
  77. Deep visual domain adaptation: A survey. Neurocomputing 312, 135–153
  78. Weakly supervised adversarial domain adaptation for semantic segmentation in urban scenes. IEEE Transactions on Image Processing 28, 4376–4386. 10.1109/TIP.2019.2910667
  79. Revisiting dilated convolution: A simple approach for weakly- and semi-supervised semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  80. A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST) 11, 1–46
  81. Wu, D. (2016). Online and offline domain adaptation for reducing bci calibration effort. IEEE Transactions on Human-Machine Systems 47, 550–563
  82. Unsupervised domain adaptation for graph-structured data using class-conditional distribution alignment. arXiv preprint arXiv:2301.12361
  83. Dcan: Dual channel-wise alignment networks for unsupervised scene adaptation. In Proceedings of the European Conference on Computer Vision (ECCV). 518–534
  84. Federated-learning-based client scheduling for low-latency wireless communications. IEEE Wireless Communications 28, 32–38. 10.1109/MWC.001.2000252
  85. Reliable weighted optimal transport for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4394–4403
  86. Generalized source-free domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 8978–8987
  87. Phase consistent ecological domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9011–9020
  88. Fda: Fourier domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4085–4095
  89. Domain adaptive semantic segmentation without source data. In Proceedings of the 29th ACM International Conference on Multimedia (New York, NY, USA: Association for Computing Machinery), MM ’21, 3293–3302. 10.1145/3474085.3475482
  90. Divergence optimization for noisy universal domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2515–2524
  91. Category anchor-guided unsupervised domain adaptation for semantic segmentation. In Advances in Neural Information Processing Systems. 435–445
  92. Curriculum domain adaptation for semantic segmentation of urban scenes. In Proceedings of the IEEE International Conference on Computer Vision. 2020–2030
  93. Instance-level segmentation for autonomous driving with deep densely connected mrfs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 669–677
  94. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV. 2223–2232
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Serban Stan (6 papers)
  2. Mohammad Rostami (64 papers)