Evolving Assembly Code in an Adversarial Environment (2403.19489v2)
Abstract: In this work, we evolve Assembly code for the CodeGuru competition. The goal is to create a survivor -- an Assembly program that runs the longest in shared memory, by resisting attacks from adversary survivors and finding their weaknesses. For evolving top-notch solvers, we specify a Backus Normal Form (BNF) for the Assembly language and synthesize the code from scratch using Genetic Programming (GP). We evaluate the survivors by running CodeGuru games against human-written winning survivors. Our evolved programs found weaknesses in the programs they were trained against and utilized them. To push evolution further, we implemented memetic operators that utilize machine learning to explore the solution space effectively. This work has important applications for cyber-security as we utilize evolution to detect weaknesses in survivors. The Assembly BNF is domain-independent; thus, by modifying the fitness function, it can detect code weaknesses and help fix them. Finally, the CodeGuru competition offers a novel platform for analyzing GP and code evolution in adversarial environments. To support further research in this direction, we provide a thorough qualitative analysis of the evolved survivors and the weaknesses found.
- David G. Andersen. 2001. The Garden: Evolving Warriors in Core Wars. https://api.semanticscholar.org/CorpusID:17099745.
- Wolfgang Banzhaf. 2018. Some Remarks on Code Evolution with Genetic Programming. In Inspired by Nature. Springer, 145–156.
- Exploiting co-evolution and a modified island model to climb the core war hill. In The 2003 Congress on Evolutionary Computation, 2003. CEC’03., Vol. 3. IEEE, Torino, Italy, 2217–2221.
- Alexander Keewatin Dewdney. 1984. Recreational Mathematics–Core Wars. Scientific American 2 (1984), 43–98.
- Franz Rothlauf Dominik Sobania, Dirk Schweim. 2021. Recent Developments in Program Synthesis with Evolutionary Algorithms. https://arxiv.org/pdf/2108.12227.pdf.
- Wildor Ferrel and Luis Alfaro. 2020. Genetic Programming-Based Code Generation for Arduino. https://thesai.org/Downloads/Volume11No11/Paper_68-Genetic_Programming_Based_Code_Generation.pdf. International Journal of Advanced Computer Science and Applications 11 (01 2020). https://doi.org/10.14569/IJACSA.2020.0111168
- Ulya R. Karpuzcu. 2005. Automatic Verilog Code Generation through Grammatical Evolution. https://dl.acm.org/doi/pdf/10.1145/1102256.1102346. In Proceedings of the 7th Annual Workshop on Genetic and Evolutionary Computation (Washington, D.C.) (GECCO ’05). Association for Computing Machinery, New York, NY, USA, 394––397. https://doi.org/10.1145/1102256.1102346
- John R Koza et al. 1994. Genetic programming II. Vol. 17. MIT press Cambridge.
- Danny Leshem and Tomer Eyzenberg. 2012. CodeGuru repository. https://github.com/codeguru-il.
- Faster sorting algorithms discovered using deep reinforcement learning. Nature 618 (06 2023), 257–263. https://doi.org/10.1038/s41586-023-06004-9
- Michael Orlov and Moshe Sipper. 2011. Flight of the FINCH through the Java wilderness. IEEE Transactions on Evolutionary Computation 15, 2 (2011), 166–182.
- On the Difficulty of Benchmarking Inductive Program Synthesis Methods. https://doi.org/10.1145/3067695.3082533. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (Berlin, Germany) (GECCO ’17). Association for Computing Machinery, New York, NY, USA, 1589––1596. https://doi.org/10.1145/3067695.3082533
- Dimitris Papailiopoulos. 2023. GPT-4 “discovered” the same sorting algorithm as AlphaDev. https://twitter.com/DimitrisPapail/status/1666843952824168465, last accessed on 2024-01-31.
- Genetic Improvement of Software: A Comprehensive Survey. IEEE Transactions on Evolutionary Computation 22, 3 (June 2018), 415–432. https://doi.org/10.1109/tevc.2017.2693219
- Christopher D. Rosin. 2019. Stepping Stones to Inductive Synthesis of Low-Level Looping Programs. https://doi.org/10.1609/aaai.v33i01.33012362. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence (AAAI’19/IAAI’19/EAAI’19). AAAI Press, Honolulu, Hawaii, USA, Article 292, 9 pages. https://doi.org/10.1609/aaai.v33i01.33012362
- Wildor Ferrel Serruto and Luis Alfaro Casas. 2017. Automatic Code Generation for Microcontroller-Based System Using Multi-objective Linear Genetic Programming. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8560802. In 2017 International Conference on Computational Science and Computational Intelligence (CSCI). IEEE, Las Vegas, NV, USA, 279–285. https://doi.org/10.1109/CSCI.2017.47
- EC-KitY: Evolutionary computation tool kit in Python with seamless machine learning integration. SoftwareX 22 (2023), 101381. https://doi.org/10.1016/j.softx.2023.101381
- Dean Sysman and Amir Leibu. 2009. Darwin8086. https://code.google.com/archive/p/darwin8086/.