Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MedBN: Robust Test-Time Adaptation against Malicious Test Samples (2403.19326v1)

Published 28 Mar 2024 in cs.LG, cs.CR, and cs.CV

Abstract: Test-time adaptation (TTA) has emerged as a promising solution to address performance decay due to unforeseen distribution shifts between training and test data. While recent TTA methods excel in adapting to test data variations, such adaptability exposes a model to vulnerability against malicious examples, an aspect that has received limited attention. Previous studies have uncovered security vulnerabilities within TTA even when a small proportion of the test batch is maliciously manipulated. In response to the emerging threat, we propose median batch normalization (MedBN), leveraging the robustness of the median for statistics estimation within the batch normalization layer during test-time inference. Our method is algorithm-agnostic, thus allowing seamless integration with existing TTA frameworks. Our experimental results on benchmark datasets, including CIFAR10-C, CIFAR100-C and ImageNet-C, consistently demonstrate that MedBN outperforms existing approaches in maintaining robust performance across different attack scenarios, encompassing both instant and cumulative attacks. Through extensive experiments, we show that our approach sustains the performance even in the absence of attacks, achieving a practical balance between robustness and performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. A little is enough: Circumventing defenses for distributed learning. Advances in Neural Information Processing Systems, 32:8632–8642, 2019.
  2. Poisoning attacks against support vector machines. In International Conference on Machine Learning, pages 1467–1474. PMLR, 2012.
  3. Diverse feature visualizations reveal invariances in early layers of deep neural networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 217–232, 2018.
  4. Tinytl: Reduce memory, not parameters for efficient on-device learning. Advances in Neural Information Processing Systems, 33:11285–11297, 2020.
  5. Poisoning and backdooring contrastive learning. In International Conference on Learning Representations, 2021.
  6. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 295–305, 2022.
  7. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):834–848, 2017.
  8. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 1(2):1–25, 2017.
  9. Robustnet: Improving domain generalization in urban-scene segmentation via instance selective whitening. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11580–11590, 2021.
  10. Improving test-time adaptation via shift-agnostic weight regularization and nearest source prototypes. In Proceedings of the European Conference on Computer Vision (ECCV), pages 440–458. Springer, 2022.
  11. Test-time poisoning attacks against test-time adaptation models. In 2024 IEEE Symposium on Security and Privacy (SP), pages 72–72. IEEE Computer Society, 2023.
  12. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3213–3223, 2016.
  13. RobustBench: a standardized adversarial robustness benchmark. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
  14. Robust mean teacher for continual and gradual test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7704–7714, 2023.
  15. Collaborative learning in the jungle (decentralized, byzantine, heterogeneous, asynchronous and nonconvex learning). Advances in Neural Information Processing Systems, 34:25044–25057, 2021.
  16. Byzantine machine learning made easy by resilient averaging of momentums. In International Conference on Machine Learning, pages 6246–6283. PMLR, 2022.
  17. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2020.
  18. What doesn’t kill you makes you robust (er): Adversarial training against poisons and backdoors. corr, 2021.
  19. Note: Robust continual test-time adaptation against temporal correlation. Advances in Neural Information Processing Systems, 35:27253–27266, 2022.
  20. Sotta: Robust test-time adaptation on noisy data streams. Advances in Neural Information Processing Systems, 36, 2023.
  21. Test time adaptation via conjugate pseudo-labels. Advances in Neural Information Processing Systems, 35:6204–6218, 2022.
  22. The hidden vulnerability of distributed learning in byzantium. In International Conference on Machine Learning, pages 3521–3530. PMLR, 2018.
  23. Byzantine fault-tolerant distributed machine learning with norm-based comparative gradient elimination. In 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), pages 175–181. IEEE, 2021.
  24. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  25. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2018.
  26. Augmix: A simple data processing method to improve robustness and uncertainty. In International Conference on Learning Representations, 2019.
  27. Mecta: Memory-economic continual test-time model adaptation. In International Conference on Learning Representations, 2023.
  28. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448–456. PMLR, 2015.
  29. Leveraging proxy of training data for test-time adaptation. In International Conference on Machine Learning, pages 15737–15752. PMLR, 2023.
  30. Sita: Single image test-time adaptation. arXiv preprint arXiv:2112.02355, 2021.
  31. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pages 5637–5664. PMLR, 2021.
  32. The byzantine generals problem. In Concurrency: the works of leslie lamport, pages 203–226. 2019.
  33. Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896. Atlanta, 2013.
  34. Ttn: A domain-shift aware batch normalization in test-time adaptation. In International Conference on Learning Representations, 2022.
  35. Ttt++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34:21808–21820, 2021.
  36. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
  37. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006.10963, 2020.
  38. Exploiting machine learning to subvert your spam filter. In Proceedings of the 1st Usenix Workshop on Large-Scale Exploits and Emergent Threats, pages 1–9, 2008.
  39. Efficient test-time model adaptation without forgetting. In International Conference on Machine Learning, pages 16888–16905. PMLR, 2022.
  40. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations, 2022.
  41. Feature visualization. Distill, 2(11):e7, 2017.
  42. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3234–3243, 2016.
  43. If your data distribution shifts, use self-learning. arXiv preprint arXiv:2104.12928, 2021.
  44. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems, 33:11539–11551, 2020.
  45. Poison frogs! targeted clean-label poisoning attacks on neural networks. Advances in Neural Information Processing Systems, 31:6106–6116, 2018.
  46. Evalnorm: Estimating batch normalization statistics for evaluation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3633–3641, 2019.
  47. Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11920–11929, 2023.
  48. Certified defenses for data poisoning attacks. Advances in Neural Information Processing Systems, 30:3520–3532, 2017.
  49. Fault-tolerant multi-agent optimization: optimal iterative distributed algorithms. In Proceedings of the 2016 ACM Symposium on Principles of Distributed Computing, pages 425–434, 2016.
  50. Four things everyone should know to improve batch normalization. In International Conference on Learning Representations, 2019.
  51. Test-time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning, pages 9229–9248. PMLR, 2020.
  52. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2020.
  53. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7201–7211, 2022.
  54. Uncovering adversarial risks of test-time adaptation. In International Conference on Machine Learning, pages 37456–37495. PMLR, 2023.
  55. Generalized byzantine-tolerant sgd. arXiv preprint arXiv:1802.10116, 2018.
  56. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation. In Uncertainty in Artificial Intelligence, pages 261–270. PMLR, 2020.
  57. Rep-net: Efficient on-device learning via feature reprogramming. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12277–12286, 2022.
  58. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning, pages 5650–5659. PMLR, 2018.
  59. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15922–15932, 2023.
  60. Visualizing and understanding convolutional networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 818–833. Springer, 2014.
  61. Memo: Test time robustness via adaptation and augmentation. Advances in Neural Information Processing Systems, 35:38629–38642, 2022.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now