Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation (2403.01131v2)

Published 2 Mar 2024 in math.OC, cs.AI, cs.CL, cs.LG, cs.NE, and cs.SE

Abstract: Recent research explores optimization using LLMs by either iteratively seeking next-step solutions from LLMs or directly prompting LLMs for an optimizer. However, these approaches exhibit inherent limitations, including low operational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. Specifically, we establish a comprehensive instruction set containing well-described problem prompts and effective optimization codes. We then develop a novel two-phase learning strategy that incorporates a contrastive learning-based warm-up procedure before the instruction-tuning phase to enhance the convergence behavior during model fine-tuning. The experiment results demonstrate that a CodeGen (350M) model fine-tuned by our LLaMoCo achieves superior optimization performance compared to GPT-4 Turbo and the other competitors across both synthetic and realistic problem sets. The fine-tuned model and the usage instructions are available at https://anonymous.4open.science/r/LLaMoCo-722A.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (94)
  1. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
  2. Optimus: Optimization modeling using mip solvers and large language models. arXiv preprint arXiv:2310.06116, 2023.
  3. Amari, S.-i. Backpropagation and stochastic gradient descent method. Neurocomputing, 1993.
  4. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021.
  5. A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explorations Newsletter, 2004.
  6. Improving differential evolution through bayesian hyperparameter optimization. In 2021 IEEE Congress on evolutionary computation (CEC), 2021.
  7. Biswas, S. S. Role of chat gpt in public health. Annals of Biomedical Engineering, 2023a.
  8. Biswas, S. S. Potential use of chat gpt in global warming. Annals of Biomedical Engineering, 2023b.
  9. A progressive batching l-bfgs method for machine learning. In International Conference on Machine Learning, 2018.
  10. Convex optimization. Cambridge university press, 2004.
  11. Language models are few-shot learners. Advances in Neural Information Processing Systems, 2020.
  12. Evoprompting: Language models for code-level neural architecture search. arXiv preprint arXiv:2302.14838, 2023.
  13. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
  14. Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems, 2017.
  15. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022.
  16. Natural selection fails to optimize mutation rates for long-term adaptation on rugged fitness landscapes. PLoS Computational Biology, 2008.
  17. Trust region methods. SIAM, 2000.
  18. Pypop7: A pure-python library for population-based black-box optimization. arXiv preprint arXiv:2212.05652, 2022.
  19. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 2020.
  20. A hybrid particle swarm optimization and simulated annealing algorithm for the job shop scheduling problem with transport resources. European Journal of Operational Research, 2023.
  21. Deap: Evolutionary algorithms made easy. The Journal of Machine Learning Research, 2012.
  22. Incoder: A generative model for code infilling and synthesis. In The Eleventh International Conference on Learning Representations, 2023.
  23. Genetic learning particle swarm optimization. IEEE transactions on cybernetics, 2015.
  24. Unixcoder: Unified cross-modal pre-training for code representation. arXiv preprint arXiv:2203.03850, 2022.
  25. Metabox: A benchmark platform for meta-black-box optimization with reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023a.
  26. Towards optimizing with large language models. arXiv preprint arXiv:2310.05204, 2023b.
  27. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532, 2023c.
  28. Instruction tuned models are quick learners. arXiv preprint arXiv:2306.05539, 2023.
  29. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), 2006.
  30. Hansen, N. Benchmarking a bi-population cma-es on the bbob-2009 function testbed. In Proceedings of the 11th annual conference companion on genetic and evolutionary computation conference: late breaking papers, 2009.
  31. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 2001.
  32. Mmes: Mixture model-based evolution strategy for large-scale optimization. IEEE Transactions on Evolutionary Computation, 2020.
  33. Holland, J. H. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. 1992.
  34. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
  35. Lawyer llama technical report. arXiv preprint arXiv:2305.15062, 2023.
  36. Phi-2: The surprising power of small language models, 2023.
  37. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024.
  38. Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645, 2023.
  39. Particle swarm optimization. In Proceedings of ICNN’95-International Conference on Neural Networks, 1995.
  40. Optimization by simulated annealing. science, 1983.
  41. Kraft, D. A software package for sequential quadratic programming. Forschungsbericht- Deutsche Forschungs- und Versuchsanstalt fur Luft- und Raumfahrt, 1988.
  42. A test-suite of non-convex constrained optimization problems from the real-world and some baseline results. Swarm and Evolutionary Computation, 2020.
  43. Ds-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning, 2023.
  44. Lange, R. T. evosax: Jax-based evolution strategies. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation, 2023.
  45. Evolution through large models. In Handbook of Evolutionary Machine Learning, 2023.
  46. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161, 2023.
  47. Competition-level code generation with alphacode. Science, 2022.
  48. Large language model for multi-objective evolutionary optimization. arXiv preprint arXiv:2310.12541, 2023a.
  49. Large language models as evolutionary optimizers. arXiv preprint arXiv:2310.19046, 2023b.
  50. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019.
  51. Scikit-optimize, 2016. URL https://github.com/scikit-optimize/scikit-optimize.
  52. Double-track particle swarm optimizer for nonlinear constrained optimization problems. Information Sciences, 2023.
  53. Chatting about chatgpt: how may ai and gpt impact academia and libraries? Library Hi Tech News, 2023.
  54. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837, 2022.
  55. Problem definitions and evaluation criteria for the cec 2021 on single objective bound constrained numerical optimization. In Proceedings of the IEEE Congress of Evolutionary Computation, 2021.
  56. Remark on “algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound constrained optimization”. ACM Transactions on Mathematical Software (TOMS), 2011.
  57. Codegen: An open large language model for code with multi-turn program synthesis. In The Eleventh International Conference on Learning Representations, 2023.
  58. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 2022.
  59. Leveraging large language models for the generation of novel metaheuristic optimization algorithms. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation, 2023.
  60. Powell, M. J. A view of algorithms for optimization without derivatives. Mathematics Today-Bulletin of the Institute of Mathematics and its Applications, 2007.
  61. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023.
  62. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 2020.
  63. A simple modification in cma-es achieving linear time and space complexity. In International conference on parallel problem solving from nature, 2008.
  64. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023.
  65. R.Turner and D.Eriksson. Bayesmark: Benchmark framework to easily compare bayesian optimization methods on real machine learning tasks, 2019. URL https://github.com/uber/bayesmark.
  66. Multitask prompted training enables zero-shot task generalization. In International Conference on Learning Representations, 2022.
  67. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  68. Large language models encode clinical knowledge. Nature, 2023.
  69. Practical bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, 2012.
  70. A new taxonomy of global optimization algorithms. Natural Computing, 2022.
  71. Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 1997.
  72. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
  73. Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020. In NeurIPS 2020 Competition and Demonstration Track, 2021.
  74. Simulated annealing. Springer, 1987.
  75. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
  76. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 2020.
  77. An improved genetic algorithm for constrained optimization problems. IEEE Access, 2023.
  78. Learning search space partition for black-box optimization using monte carlo tree search. Advances in Neural Information Processing Systems, 2020.
  79. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2022.
  80. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019.
  81. Employing reinforcement learning to enhance particle swarm optimization methods. Engineering Optimization, 2022.
  82. Problem definitions and evaluation criteria for the cec 2017 competition on constrained real-parameter optimization. National University of Defense Technology, Changsha, Hunan, PR China and Kyungpook National University, Daegu, South Korea and Nanyang Technological University, Singapore, Technical Report, 2017.
  83. Generalized simulated annealing algorithm and its application to the thomson model. Physics Letters A, 1997.
  84. UnifiedSKG: Unifying and multi-tasking structured knowledge grounding with text-to-text language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022.
  85. Helper and equivalent objectives: Efficient approach for constrained optimization. IEEE transactions on cybernetics, 2020.
  86. Large language models as optimizers. arXiv preprint arXiv:2309.03409, 2023.
  87. Differential evolution with alternation between steady monopoly and transient competition of mutation strategies. Swarm and Evolutionary Computation, 2023.
  88. Large language models meet nl2code: A survey. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, 2023.
  89. A survey on evolutionary computation for complex continuous optimization. Artificial Intelligence Review, 2022.
  90. Recommendation as instruction following: A large language model empowered recommendation approach. arXiv preprint arXiv:2305.07001, 2023.
  91. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023.
  92. Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023.
  93. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206, 2023.
  94. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Zeyuan Ma (17 papers)
  2. Hongshu Guo (15 papers)
  3. Jiacheng Chen (37 papers)
  4. Guojun Peng (8 papers)
  5. Zhiguang Cao (48 papers)
  6. Yining Ma (31 papers)
  7. Yue-Jiao Gong (23 papers)
Citations (9)
View on Semantic Scholar