DART: A Principled Approach to Adversarially Robust Unsupervised Domain Adaptation
Abstract: Distribution shifts and adversarial examples are two major challenges for deploying machine learning models. While these challenges have been studied individually, their combination is an important topic that remains relatively under-explored. In this work, we study the problem of adversarial robustness under a common setting of distribution shift - unsupervised domain adaptation (UDA). Specifically, given a labeled source domain $D_S$ and an unlabeled target domain $D_T$ with related but different distributions, the goal is to obtain an adversarially robust model for $D_T$. The absence of target domain labels poses a unique challenge, as conventional adversarial robustness defenses cannot be directly applied to $D_T$. To address this challenge, we first establish a generalization bound for the adversarial target loss, which consists of (i) terms related to the loss on the data, and (ii) a measure of worst-case domain divergence. Motivated by this bound, we develop a novel unified defense framework called Divergence Aware adveRsarial Training (DART), which can be used in conjunction with a variety of standard UDA methods; e.g., DANN [Ganin and Lempitsky, 2015]. DART is applicable to general threat models, including the popular $\ell_p$-norm model, and does not require heuristic regularizers or architectural changes. We also release DomainRobust: a testbed for evaluating robustness of UDA models to adversarial attacks. DomainRobust consists of 4 multi-domain benchmark datasets (with 46 source-target pairs) and 7 meta-algorithms with a total of 11 variants. Our large-scale experiments demonstrate that on average, DART significantly enhances model robustness on all benchmarks compared to the state of the art, while maintaining competitive standard accuracy. The relative improvement in robustness from DART reaches up to 29.2% on the source-target domain pairs considered.
- f-domain adversarial learning: Theory and algorithms. In International Conference on Machine Learning. PMLR, 2021.
- Threat of adversarial attacks on deep learning in computer vision: A survey. Ieee Access, 2018.
- Square attack: a query-efficient black-box adversarial attack via random search. In European conference on computer vision. Springer, 2020.
- Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning. PMLR, 2018.
- Adversarial robustness for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
- Recent advances in adversarial training for adversarial robustness. arXiv preprint arXiv:2102.01356, 2021.
- Analysis of representations for domain adaptation. Advances in neural information processing systems, 2006.
- A theory of learning from different domains. Machine learning, 2010.
- Dimitri Bertsekas. Nonlinear Programming. Athena Scientific, 2016.
- Adversarial attacks and defences: A survey. arXiv preprint arXiv:1810.00069, 2018.
- Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learning. PMLR, 2020a.
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning. PMLR, 2020b.
- Randaugment: Practical data augmentation with no separate search. arXiv preprint arXiv:1909.13719, 2019.
- Pac-learning in the presence of adversaries. Advances in Neural Information Processing Systems, 31, 2018.
- Impossibility theorems for domain adaptation. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010.
- Unsupervised domain adaptation by backpropagation. In International conference on machine learning. PMLR, 2015.
- Domain-adversarial training of neural networks. The journal of machine learning research, 2016.
- Transfer learning for domain adaptation in mri: Application in brain lesion segmentation. In Medical Image Computing and Computer Assisted Intervention. Springer, 2017.
- Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
- A kernel two-sample test. The Journal of Machine Learning Research, 2012.
- In search of lost domain generalization. arXiv preprint arXiv:2007.01434, 2020.
- Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019.
- Jonathan J. Hull. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 1994.
- Adversarial risk bounds via function transformation. arXiv preprint arXiv:1810.09519, 2018.
- A method for stochastic optimization. In International conference on learning representations, 2015.
- Co-regularized alignment for unsupervised domain adaptation. Advances in neural information processing systems, 2018.
- Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
- Mmd gan: towards deeper understanding of moment matching network. In Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017a.
- Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, 2017b.
- Subtype-aware unsupervised domain adaptation for medical diagnosis. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
- Deep unsupervised domain adaptation: A review of recent advances and perspectives. APSIPA Transactions on Signal and Information Processing, 2022.
- Exploring adversarially robust training for unsupervised domain adaptation. In Proceedings of the Asian Conference on Computer Vision, 2022.
- Transfer feature learning with joint distribution adaptation. In Proceedings of the IEEE international conference on computer vision, 2013.
- Learning transferable features with deep adaptation networks. In International conference on machine learning. PMLR, 2015.
- Deep transfer learning with joint adaptation networks. In International conference on machine learning. PMLR, 2017.
- Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.
- Domain adaptation with multiple sources. Advances in neural information processing systems, 2008.
- Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430, 2009.
- Reading digits in natural images with unsupervised feature learning. In Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
- Kl guided domain adaptation. arXiv preprint arXiv:2106.07780, 2021.
- Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924, 2017.
- Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, 2019.
- Meta pseudo labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021.
- A survey on domain adaptation theory: learning bounds and theoretical guarantees. arXiv preprint arXiv:2004.11829, 2020.
- A distributional robustness perspective on adversarial training with the ∞\infty∞-wasserstein distance. 2021.
- Adversarial attacks and defenses in deep learning. Engineering, 2020.
- Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
- Adversarially robust transfer learning. arXiv preprint arXiv:1905.08232, 2019.
- Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
- Distributionally robust deep learning as a generalization of adversarial training. In NIPS workshop on Machine Learning and Computer Security, 2017.
- Benchmarking robustness to adversarial image obfuscations. arXiv preprint arXiv:2301.12993, 2023.
- Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision–ECCV 2016 Workshops. Springer, 2016.
- Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
- Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999.
- Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
- Improving adversarial robustness requires revisiting misclassified examples. In International conference on learning representations, 2019.
- Generative adversarial guided learning for domain adaptation. In BMVC, 2018.
- A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST), 2020.
- Exploring robustness of unsupervised domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
- Central moment discrepancy (cmd) for domain-invariant representation learning. In International Conference on Learning Representations, 2017.
- Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning. PMLR, 2019a.
- Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 2020.
- Bridging theory and algorithm for domain adaptation. In International conference on machine learning. PMLR, 2019b.
- Srouda: Meta self-training for robust unsupervised domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
