Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Tropical Decision Boundaries for Neural Networks Are Robust Against Adversarial Attacks (2402.00576v1)

Published 1 Feb 2024 in cs.LG, cs.CR, cs.CV, and math.CO

Abstract: We introduce a simple, easy to implement, and computationally efficient tropical convolutional neural network architecture that is robust against adversarial attacks. We exploit the tropical nature of piece-wise linear neural networks by embedding the data in the tropical projective torus in a single hidden layer which can be added to any model. We study the geometry of its decision boundary theoretically and show its robustness against adversarial attacks on image datasets using computational experiments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Adversarial attacks on neural network policies. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings, pages dblp computer science bibliography, https://dblp.org. OpenReview.net, 2017. URL https://openreview.net/forum?id=ryvlRyBKl.
  2. Technical report on the cleverhans v2.1.0 adversarial examples library. arXiv preprint arXiv:1610.00768, 2018.
  3. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, page https://openreview.net/forum?id=rJzIBfZAb, 2018.
  4. S Kotyan and DV Vargas. Adversarial robustness assessment: Why in evaluation both l0subscript𝑙0l_{0}italic_l start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT and l∞subscript𝑙l_{\infty}italic_l start_POSTSUBSCRIPT ∞ end_POSTSUBSCRIPT attacks are necessary. PLoS One, 14(17(4)):e0265723, 2022. doi: 10.1371/journal.pone.0265723.
  5. Towards evaluating the robustness of neural networks, 2017a.
  6. Provable robustness of relu networks via maximization of linear regions. AISTATS 2019, 2019.
  7. Provable robustness against all adversarial lpsubscript𝑙𝑝l_{p}italic_l start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT-perturbations for p≥1𝑝1p\geq 1italic_p ≥ 1, 2020.
  8. A survey of robust adversarial training in pattern recognition: Fundamental, theory, and methodologies. Pattern Recognition, 131:108889, 2022. ISSN 0031-3203. doi: https://doi.org/10.1016/j.patcog.2022.108889. URL https://www.sciencedirect.com/science/article/pii/S0031320322003703.
  9. Theoretical analysis of adversarial learning: a minimax approach. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, page 11, Red Hook, NY, USA, 2019. Curran Associates Inc.
  10. ME-Net: Towards effective adversarial robustness with matrix estimation. In Proceedings of the 36th International Conference on Machine Learning (ICML), page https://arxiv.org/abs/1905.11971, 2019.
  11. Cstar: Towards compact and structured deep neural networks with adversarial robustness. CoRR, abs/2212.01957, 2022. URL https://doi.org/10.48550/arXiv.2212.01957.
  12. Transformed low-rank parameterization can help robust generalization for tensor neural networks. In Thirty-seventh Conference on Neural Information Processing Systems, page https://openreview.net/forum?id=rih3hsSWx8, 2023.
  13. D. Maclagan and B. Sturmfels. Introduction to Tropical Geometry, volume 161 of Graduate Studies in Mathematics. Graduate Studies in Mathematics, 161, American Mathematical Society, Providence, RI, 2015.
  14. Michael Joswig. Essentials of tropical combinatorics. Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2022.
  15. Tropical support vector machines: Evaluations and extension to function spaces. Neural Networks, 157:77–89, 2023a. ISSN 0893-6080.
  16. K. Miura and R. Yoshida. Plücker coordinates of the best-fit stiefel tropical linear space to a mixture of gaussian distributions. Info. Geo., 6:171–201, 2023.
  17. Tropical logistic regression model on space of phylogenetic trees, 2023. Available at https://arxiv.org/abs/2306.08796.
  18. Tropical neural networks and its applications to classifying phylogenetic trees, 2023b. Available at https://arxiv.org/abs/2309.13410.
  19. Deep sparse rectifier neural networks. In Geoffrey Gordon, David Dunson, and Miroslav Dudík, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 315–323, Fort Lauderdale, FL, USA, 11–13 Apr 2011. PMLR. URL https://proceedings.mlr.press/v15/glorot11a.html.
  20. Tropical geometry of deep neural networks. In International Conference on Machine Learning, pages 5824–5832. PMLR, 2018.
  21. Maxout networks. In Sanjoy Dasgupta and David McAllester, editors, Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 1319–1327, Atlanta, Georgia, USA, 17–19 Jun 2013a. PMLR. URL https://proceedings.mlr.press/v28/goodfellow13.html.
  22. A tropical approach to neural networks with piecewise linear activations. ArXiv, abs/1805.08749, 2018. URL https://api.semanticscholar.org/CorpusID:46895499.
  23. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39–57, 2017b. doi: 10.1109/SP.2017.49.
  24. Tropical foundations for probability & statistics on phylogenetic tree space, 2019.
  25. Tropical bisectors and voronoi diagrams. Found Comput Math, 22:1923–1960, 2022. https://doi.org/10.1007/s10208-021-09538-4.
  26. Tropical fermat-weber polytopes, 2024. In progress.
  27. Ding-Xuan Zhou. Universality of deep convolutional neural networks. Applied and Computational Harmonic Analysis, 48(2):787–794, 2020. ISSN 1063-5203. doi: https://doi.org/10.1016/j.acha.2019.06.004. URL https://www.sciencedirect.com/science/article/pii/S1063520318302045.
  28. Maximum inscribed and minimum enclosing tropical balls of tropical polytopes and applications to volume estimation and uniform sampling, 2023. Available at https://arxiv.org/abs/2303.02539.
  29. Multivariate volume, ehrhart, and h*superscriptℎh^{*}italic_h start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT-polynomials of polytropes. J. Symb. Comput., 114(C):209–230, jan 2023. ISSN 0747-7171. doi: 10.1016/j.jsc.2022.04.011. URL https://doi.org/10.1016/j.jsc.2022.04.011.
  30. Tropical bisectors and voronoi diagrams. Foundations of Computational Mathematics, 22(6):1923–1960, September 2021. ISSN 1615-3383. doi: 10.1007/s10208-021-09538-4. URL http://dx.doi.org/10.1007/s10208-021-09538-4.
  31. Ngoc Mai Tran. Enumerating polytropes. Journal of Combinatorial Theory, Series A, 151:1–22, 2017. ISSN 0097-3165. doi: https://doi.org/10.1016/j.jcta.2017.03.011. URL https://www.sciencedirect.com/science/article/pii/S0097316517300389.
  32. Michael Joswig. Essentials of Tropical Combinatorics. Springer, New York, NY, 2021.
  33. Ngoc M Tran. The tropical geometry of causal inference for extremes, 2022. URL https://arxiv.org/abs/2207.10227.
  34. On the distributions of the range and mean range for samples from a normal distribution. Biometrika, 53(1/2):245–248, 1966. ISSN 00063444. URL http://www.jstor.org/stable/2334072.
  35. H. Leon Harter. Expected values of normal order statistics. Biometrika, 48(1/2):151–165, 1961.
  36. Mnist handwritten digit database, 2010.
  37. Reading digits in natural images with unsupervised feature learning. In Advances in Neural Information Processing Systems (NIPS), page https://api.semanticscholar.org/CorpusID:16852518, 2011.
  38. Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
  39. Towards deep learning models resistant to adversarial attacks, 2019.
  40. Explaining and harnessing adversarial examples, 2015.
  41. Maxout networks, 2013b.
  42. Deep residual learning for image recognition, 2015.
  43. Adversarial training and robustness for multiple perturbations, 2019.
  44. Adversarial risk and the dangers of evaluating against weak attacks, 2018.
  45. Robustness via curvature regularization, and vice versa, 2018.
  46. James C. Spall. Introduction to Stochastic Search and Optimization. John Wiley & Sons, Inc, 2003.
  47. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples, 2018.
  48. Provable defenses against adversarial examples via the convex outer adversarial polytope, 2018.
  49. Text analysis using deep neural networks in digital humanities and information science, 2023. Available at arxiv.org/abs/2307.16217.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now