How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems (2403.01757v1)
Abstract: Recently, LLMs have notably positioned them as capable tools for addressing complex optimization challenges. Despite this recognition, a predominant limitation of existing LLM-based optimization methods is their struggle to capture the relationships among decision variables when relying exclusively on numerical text prompts, especially in high-dimensional problems. Keeping this in mind, we first propose to enhance the optimization performance using multimodal LLM capable of processing both textual and visual prompts for deeper insights of the processed optimization problem. This integration allows for a more comprehensive understanding of optimization problems, akin to human cognitive processes. We have developed a multimodal LLM-based optimization framework that simulates human problem-solving workflows, thereby offering a more nuanced and effective analysis. The efficacy of this method is evaluated through extensive empirical studies focused on a well-known combinatorial optimization problem, i.e., capacitated vehicle routing problem. The results are compared against those obtained from the LLM-based optimization algorithms that rely solely on textual prompts, demonstrating the significant advantages of our multimodal approach.
- Simulated annealing. Statistical science, 8(1):10–15, 1993.
- Christian Blum. Ant colony optimization: Introduction and recent trends. Physics of Life reviews, 2(4):353–373, 2005.
- Enhancing genetic improvement mutations using large language models. In International Symposium on Search Based Software Engineering, pages 153–159. Springer, 2023.
- Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
- Andrew Cox. Power, value and supply chain management. Supply chain management: An international journal, 4(4):167–175, 1999.
- Large language models for compiler optimization. arXiv preprint arXiv:2309.07062, 2023.
- Kalyanmoy Deb. Optimization for engineering design: Algorithms and examples. PHI Learning Pvt. Ltd., 2012.
- Towards optimizing with large language models. arXiv preprint arXiv:2310.05204, 2023.
- Chatgpt for shaping the future of dentistry: the potential of multi-modal large language model. International Journal of Oral Science, 15(1):29, 2023.
- Douglas B Lenat. The nature of heuristics. Artificial intelligence, 19(2):189–249, 1982.
- Large language models for supply chain optimization. arXiv preprint arXiv:2307.03875, 2023.
- Large language model for multi-objective evolutionary optimization. arXiv preprint arXiv:2310.12541, 2023.
- Language model crossover: Variation through few-shot prompting. arXiv preprint arXiv:2302.12170, 2023.
- H Chr Pfohl. Logistics systems. Springer, 2010.
- Leveraging large language models for the generation of novel metaheuristic optimization algorithms. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation, pages 1812–1820, 2023.
- Mathematical discoveries from program search with large language models. Nature, pages 1–3, 2023.
- A novel algorithm for capacitated vehicle routing problem for smart cities. Symmetry, 13(10):1923, 2021.
- Optimization for machine learning. Mit Press, 2012.
- Vehicle routing: problems, methods, and applications. SIAM, 2014.
- No free lunch theorems for optimization. IEEE transactions on evolutionary computation, 1(1):67–82, 1997.
- Large language models as optimizers. arXiv preprint arXiv:2309.03409, 2023.
- Introduction to evolutionary algorithms. Springer Science & Business Media, 2010.
- Multi-objective medical supplies distribution open vehicle routing problem with fairness and timeliness under major public health emergencies. Management System Engineering, 2(1):5, 2023.