L-AutoDA: Leveraging Large Language Models for Automated Decision-based Adversarial Attacks (2401.15335v2)

Abstract: In the rapidly evolving field of machine learning, adversarial attacks present a significant challenge to model robustness and security. Decision-based attacks, which only require feedback on the decision of a model rather than detailed probabilities or scores, are particularly insidious and difficult to defend against. This work introduces L-AutoDA (LLM-based Automated Decision-based Adversarial Attacks), a novel approach leveraging the generative capabilities of LLMs to automate the design of these attacks. By iteratively interacting with LLMs in an evolutionary framework, L-AutoDA automatically designs competitive attack algorithms efficiently without much human effort. We demonstrate the efficacy of L-AutoDA on CIFAR-10 dataset, showing significant improvements over baseline methods in both success rate and computational efficiency. Our findings underscore the potential of LLMs as tools for adversarial attack generation and highlight new avenues for the development of robust AI systems.