Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ARBiBench: Benchmarking Adversarial Robustness of Binarized Neural Networks (2312.13575v1)

Published 21 Dec 2023 in cs.CV and cs.LG

Abstract: Network binarization exhibits great potential for deployment on resource-constrained devices due to its low computational cost. Despite the critical importance, the security of binarized neural networks (BNNs) is rarely investigated. In this paper, we present ARBiBench, a comprehensive benchmark to evaluate the robustness of BNNs against adversarial perturbations on CIFAR-10 and ImageNet. We first evaluate the robustness of seven influential BNNs on various white-box and black-box attacks. The results reveal that 1) The adversarial robustness of BNNs exhibits a completely opposite performance on the two datasets under white-box attacks. 2) BNNs consistently exhibit better adversarial robustness under black-box attacks. 3) Different BNNs exhibit certain similarities in their robustness performance. Then, we conduct experiments to analyze the adversarial robustness of BNNs based on these insights. Our research contributes to inspiring future research on enhancing the robustness of BNNs and advancing their application in real-world scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (85)
  1. Square attack: A query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision (ECCV), pages 484–501, 2020.
  2. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML), pages 274–283, 2018.
  3. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013.
  4. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In International Conference on Learning Representations (ICLR), 2018.
  5. Xnor-net++: Improved binary neural networks. In British Machine Vision Conference (BMVC), page 62, 2019.
  6. End-to-end object detection with transformers. In European Conference on Computer Vision (ECCV), pages 213–229, 2020.
  7. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pages 39–57, 2017.
  8. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.06085, 2018.
  9. Low-bit quantization of neural networks for efficient inference. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pages 3009–3018, 2019.
  10. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830, 2016.
  11. Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670, 2020.
  12. Exploiting linear structure within convolutional networks for efficient evaluation. In Advances in neural information processing systems (NIPS), pages 1269–1277, 2014.
  13. Regularizing activation distribution for training binarized deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11408–11417, 2019a.
  14. Global sparse momentum SGD for pruning very deep neural networks. In Advances in neural information processing systems (NIPS), pages 6379–6391, 2019b.
  15. Boosting adversarial attacks with momentum. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9185–9193, 2018.
  16. Evading defenses to transferable adversarial examples by translation-invariant attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4312–4321, 2019a.
  17. Efficient decision-based black-box adversarial attacks on face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7714–7722, 2019b.
  18. Benchmarking adversarial robustness on image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 318–328, 2020.
  19. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021.
  20. A study of the effect of jpg compression on adversarial images. arXiv preprint arXiv:1608.00853, 2016.
  21. Exploring the landscape of spatial robustness. In International Conference on Machine Learning (ICML), pages 1802–1811. PMLR, 2019.
  22. Learned step size quantization. arXiv preprint arXiv:1902.08153, 2019.
  23. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision (IJCV), 88(2):303–338, 2010.
  24. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410, 2017.
  25. Attacking binarized neural networks. In International Conference on Learning Representations (ICLR), 2018.
  26. Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115, 2014.
  27. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
  28. Improved gradient-based adversarial attacks for quantized networks. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 6810–6818, 2022.
  29. Learning both weights and connections for efficient neural network. In Advances in neural information processing systems (NIPS), pages 1135–1143, 2015.
  30. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  31. Proxybnn: Learning binarized neural networks via proxy matrices. In European Conference on Computer Vision (ECCV), pages 223–241. Springer, 2020.
  32. Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1398–1406, 2017.
  33. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
  34. Binarized neural networks. In Advances in neural information processing systems (NIPS), pages 4107–4115, 2016.
  35. Speeding up convolutional neural networks with low rank expansions. In British Machine Vision Conference (BMVC), 2014.
  36. Delving into multimodal prompting for fine-grained visual classification. arXiv preprint arXiv:2309.08912, 2023.
  37. Combinatorial attacks on binarized neural networks. In International Conference on Learning Representations (ICLR), 2019.
  38. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (NIPS), pages 1106–1114, 2012.
  39. Functional adversarial attacks. Advances in neural information processing systems (NIPS), 32, 2019.
  40. NATTACK: learning the distributions of adversarial examples for an improved black-box attack on deep neural networks. In International Conference on Machine Learning (ICML), pages 3866–3876, 2019.
  41. Defensive quantization: When efficiency meets robustness. In International Conference on Learning Representations (ICLR), 2019.
  42. Nesterov accelerated gradient and scale invariance for adversarial attacks. In International Conference on Learning Representations (ICLR), 2020a.
  43. Rotated binary neural network. Advances in neural information processing systems (NIPS), 33:7474–7485, 2020b.
  44. Deep learning for generic object detection: A survey. International Journal of Computer Vision (IJCV), 128(2):261–318, 2020a.
  45. Bi-real net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm. In European Conference on Computer Vision (ECCV), pages 747–763, 2018.
  46. Reactnet: Towards precise binary neural network with generalized activation functions. In European Conference on Computer Vision (ECCV), pages 143–159, 2020b.
  47. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3431–3440, 2015.
  48. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018a.
  49. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018b.
  50. Training binary neural networks with real-to-binary convolutions. arXiv preprint, 2020.
  51. Deepfool: A simple and accurate method to fool deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2574–2582, 2016.
  52. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning (ICML), pages 4970–4979. PMLR, 2019.
  53. Forward and backward information retention for accurate binary neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2250–2259, 2020.
  54. Bibench: Benchmarking and analyzing network binarization. In International Conference on Machine Learning (ICML), pages 28351–28388, 2023.
  55. Certified defenses against adversarial examples. arXiv preprint arXiv:1801.09344, 2018.
  56. Xnor-net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision (ECCV), pages 525–542, 2016.
  57. You only look once: Unified, real-time object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 779–788, 2016.
  58. Adversarial attacks and defenses in deep learning. Engineering, 6(3):346–360, 2020.
  59. Faster R-CNN: towards real-time object detection with region proposal networks. In Advances in neural information processing systems (NIPS), pages 91–99, 2015.
  60. Network binarization via contrastive learning. In European Conference on Computer Vision (ECCV), pages 586–602, 2022.
  61. Svnet: Where so (3) equivariance meets binarization on point cloud representation. In 2022 International Conference on 3D Vision (3DV), pages 547–556. IEEE, 2022.
  62. Lightweight pixel difference networks for efficient visual representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023.
  63. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014.
  64. Robustart: Benchmarking robustness on architecture design and training techniques. arXiv preprint arXiv:2109.05211, 2021.
  65. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204, 2017.
  66. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference on Machine Learning (ICML), pages 5032–5041, 2018.
  67. Breakingbed–breaking binary and efficient deep neural networks by adversarial attacks. arXiv preprint arXiv:2103.08031, 2021.
  68. Benchmarking adversarial robustness of compressed deep learning models. arXiv preprint arXiv:2308.08160, 2023.
  69. Sparsity-inducing binarized neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 12192–12199, 2020.
  70. Gradient matters: Designing binarized neural networks via enhanced information-flow. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(11):7551–7562, 2021.
  71. Wasserstein adversarial examples via projected sinkhorn iterations. In International Conference on Machine Learning (ICML), pages 6808–6817. PMLR, 2019.
  72. Quantized convolutional neural networks for mobile devices. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4820–4828, 2016.
  73. Robustmq: Benchmarking robustness of quantized models. arXiv preprint arXiv:2308.02350, 2023.
  74. Mitigating adversarial effects through randomization. In International Conference on Learning Representations (ICLR), 2018.
  75. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2730–2739, 2019.
  76. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155, 2017.
  77. Learning frequency domain approximation for binary neural networks. Advances in neural information processing systems (NIPS), 34:25553–25565, 2021a.
  78. Accurate and compact convolutional neural networks with trained binarization. arXiv preprint, 2019.
  79. Training shallow and thin networks for acceleration via knowledge distillation with conditional adversarial networks. In International Conference on Learning Representations (ICLR) Workshops, 2018.
  80. Recu: Reviving the dead weights in binary neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5178–5188, 2021b.
  81. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7130–7138, 2017.
  82. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (ICML), pages 7472–7482, 2019.
  83. Towards compact 1-bit cnns via bayesian learning. International Journal of Computer Vision (IJCV), pages 1–25, 2022.
  84. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160, 2016.
  85. Towards effective low-bitwidth convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7920–7928, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now