Algorithm Evolution Using Large Language Model (2311.15249v1)

Abstract: Optimization can be found in many real-life applications. Designing an effective algorithm for a specific optimization problem typically requires a tedious amount of effort from human experts with domain knowledge and algorithm design skills. In this paper, we propose a novel approach called Algorithm Evolution using LLM (AEL). It utilizes a LLM to automatically generate optimization algorithms via an evolutionary framework. AEL does algorithm-level evolution without model training. Human effort and requirements for domain knowledge can be significantly reduced. We take constructive methods for the salesman traveling problem as a test example, we show that the constructive algorithm obtained by AEL outperforms simple hand-crafted and LLM-generated heuristics. Compared with other domain deep learning model-based algorithms, these methods exhibit excellent scalability across different problem sizes. AEL is also very different from previous attempts that utilize LLMs as search operators in algorithms.

The Evolution of Optimization Algorithms with AI

Introduction to Optimization

Optimization is a fundamental aspect of problem-solving in various fields such as logistics, manufacturing, resource allocation, and many other areas that require decision-making. Typically, creating an algorithm to effectively solve a specific optimization problem takes a considerable amount of work and expertise, an undertaking that is both time-consuming and requires deep knowledge in both the domain of the problem and algorithm design.

A Novel Approach: Algorithm Evolution using LLM (AEL)

In light of the challenges presented in traditional optimization methods, a novel approach titled Algorithm Evolution using LLM (AEL) has been introduced. This approach leverages the capabilities of LLMs like the well-known GPT-3.5-turbo and GPT-4 to automatically construct and evolve optimization algorithms without the need for training models specific to the domain. This framework presents the possibility of significantly reducing the effort and specialized knowledge traditionally required in algorithm design.

Testing the AEL Approach

The proposed AEL method was applied to the Traveling Salesman Problem (TSP), a classic combinatorial problem. The TSP involves finding the most efficient route that visits a list of locations once before returning to the starting point. AEL's performance was benchmarked against three different design strategies: a hand-crafted greedy algorithm by human experts, a domain-specific model that learned heuristics through training, and algorithms generated directly via instruction to LLMs.

Advantages and Potential of AEL

In comparative tests, the algorithm produced by AEL demonstrated superior performance over the simple greedy algorithm developed by humans. It outperformed the human-crafted algorithm across various TSP instances, providing better generalization across different problem sizes compared to domain-specific models trained on TSP instances.

Moreover, AEL proved more capable than LLMs that were directly prompted to generate algorithms for TSP, displaying the importance of the evolutionary methods integrated into the AEL framework. The use of advanced models, such as GPT-4, showed that AEL could harness the power of cutting-edge AI to evolve high-quality optimization strategies with minimal human input.

Potential Future Enhancements

The AEL framework is seen as a starting point for future exploration, with vast potential for extensions and improvements. The incorporation of other heuristic optimization tools, the introduction of additional problem-specific information to enhance algorithm evolution, and the application of AEL to more complex, real-world, and multi-objective optimization tasks are areas ripe for development. Moreover, the current effectiveness of AEL in exploiting LLMs raises intriguing questions about the potential of these models as catalysts for revolutionizing algorithm design.

Conclusion

The introduction of Algorithm Evolution using LLM marks a significant advancement in the field of optimization. By effectively reducing the need for domain-specific model training and extensive expert knowledge, AEL can streamline the algorithm design process and potentially open up new possibilities for tackling complex optimization problems across numerous industries and research domains. As AEL evolves, it may redefine the landscape of optimization, making it more accessible, efficient, and adaptable to change.