Is ReLU Adversarially Robust? (2405.03777v1)
Abstract: The efficacy of deep learning models has been called into question by the presence of adversarial examples. Addressing the vulnerability of deep learning models to adversarial examples is crucial for ensuring their continued development and deployment. In this work, we focus on the role of rectified linear unit (ReLU) activation functions in the generation of adversarial examples. ReLU functions are commonly used in deep learning models because they facilitate the training process. However, our empirical analysis demonstrates that ReLU functions are not robust against adversarial examples. We propose a modified version of the ReLU function, which improves robustness against adversarial examples. Our results are supported by an experiment, which confirms the effectiveness of our proposed modification. Additionally, we demonstrate that applying adversarial training to our customized model further enhances its robustness compared to a general model.
- Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39–57. IEEE, 2017.
- Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206–2216. PMLR, 2020.
- Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255, 2009. doi: 10.1109/CVPR.2009.5206848.
- Deng, L. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141–142, 2012.
- Adversarial and clean data are not twins. arXiv preprint arXiv:1704.04960, 2017.
- Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
- Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15262–15271, 2021.
- Black-box adversarial attacks with limited queries and information. In International Conference on Machine Learning, pp. 2137–2146. PMLR, 2018.
- Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- Adversarial examples for generative models. In 2018 ieee security and privacy workshops (spw), pp. 36–42. IEEE, 2018.
- Learning multiple layers of features from tiny images. 2009.
- Adversarial examples in the physical world, 2016.
- Adversarial examples in the physical world. In Artificial intelligence safety and security, pp. 99–112. Chapman and Hall/CRC, 2018.
- Defensive quantization: When efficiency meets robustness. arXiv preprint arXiv:1904.08444, 2019.
- Towards imperceptible and robust adversarial example attacks against neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
- Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.
- Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277, 2016a.
- The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on security and privacy (EuroS&P), pp. 372–387. IEEE, 2016b.
- Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp. 506–519, 2017.
- Defend deep neural networks against adversarial examples via fixed and dynamic quantized activation functions. arXiv preprint arXiv:1807.06714, 2018.
- Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510–4520, 2018.
- Adversarial training for free! Advances in Neural Information Processing Systems, 32, 2019.
- Enhancing adversarial examples on deep q networks with previous information. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 01–07. IEEE, 2021.
- Evaluation of adversarial attacks sensitivity of classifiers with occluded input data. Neural Computing and Applications, 34(20):17615–17632, 2022.
- Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204, 2017.
- Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2019.
- Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.03994, 2020.
- Generating adversarial examples with adversarial networks. arXiv preprint arXiv:1801.02610, 2018.
- Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472–7482. PMLR, 2019.
- Generating natural adversarial examples. arXiv preprint arXiv:1710.11342, 2017.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.