OptiMUS: Optimization Modeling Using MIP Solvers and large language models (2310.06116v2)

Abstract: Optimization problems are pervasive across various sectors, from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the-art solvers, as the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. We introduce OptiMUS, a LLM-based agent designed to formulate and solve MILP problems from their natural language descriptions. OptiMUS is capable of developing mathematical models, writing and debugging solver code, developing tests, and checking the validity of generated solutions. To benchmark our agent, we present NLP4LP, a novel dataset of linear programming (LP) and mixed integer linear programming (MILP) problems. Our experiments demonstrate that OptiMUS solves nearly twice as many problems as a basic LLM prompting strategy. OptiMUS code and NLP4LP dataset are available at \href{https://github.com/teshnizi/OptiMUS}{https://github.com/teshnizi/OptiMUS}

• The paper presents OptiMUS, a novel framework that automates MILP modeling by combining LLM-driven formulation with MIP solvers.
• It introduces the NLP4LP dataset with 52 expert-annotated instances to standardize natural language descriptions for optimization.
• It demonstrates a 91% improvement over conventional LLM prompting through automated testing, debugging, and problem rephrasing.

Overview of "OptiMUS: Optimization Modeling Using MIP Solvers and LLMs"

This paper introduces OptiMUS, an innovative framework that integrates LLMs with optimization solvers to automate the process of solving mixed integer linear programming (MILP) problems from natural language descriptions. The authors, Ali AhmadiTeshnizi, Wenzhi Gao, and Madeleine Udell, present this work within the broader context of making optimization techniques more accessible across various sectors by reducing the need for significant domain expertise.

Contributions

The primary contributions of the paper are threefold:

1. Introduction of NLP4LP Dataset: The authors introduce NLP4LP, a novel dataset constructed to facilitate the formulation and solving of linear programming (LP) and MILP problems. Comprising 52 instances, the dataset includes expert annotations and representations of problems in structured natural language (referred to as SNOP) alongside their respective solver codes and optimal solutions.
2. Development of OptiMUS: OptiMUS is crafted as an LLM-based agent that streamlines the process of optimization modeling. The methodology encompasses generating mathematical formulations, solver code, and validity checks for solutions. Additionally, OptiMUS can autonomously augment data by rephrasing problem statements to improve robustness across varied representations.
3. Empirical Demonstration of Effectiveness: Through extensive experimentation, OptiMUS is shown to significantly outperform basic LLM prompting strategies, with a reported solve rate increase of 91%. Critical to this success is the integration of automated testing, debugging, and problem rephrasing, which together enhance both execution and success rates.

Methodology

OptiMUS leverages the structured representation of optimization problems to interact with optimization solvers effectively. The system utilizes a multi-step process:

• SNOP Representation: A problem is defined using a standardized structure separating the natural language description from the data. This approach mitigates the limitations of LLMs concerning context length and facilitates more efficient computation.
• Formulation and Code Generation: The LLM interprets the problem formulation from SNOP and generates the corresponding solver code. Error correction mechanisms are employed to handle runtime or syntax issues iteratively.
• Automated Testing and Revision: Automatic and supervised test generation checks output validity, guided by error feedback for iterative correction. This feature ensures adherence to the problem constraints and output specifications.
• Augmentation via Rephrasing: OptiMUS employs problem statement rephrasing to address representational variances across problem instances, further improving solve rates by diversifying problem approaches.

Implications and Future Directions

The paper underscores the potential of LLM-enhanced optimization systems to democratize access to sophisticated modeling techniques traditionally reserved for experts. By lowering the barrier to entry, diverse applications in fields such as logistics, healthcare, and energy can benefit from optimization's efficiency gains.

From a theoretical standpoint, the research demonstrates a significant step forward in augmenting LLM capabilities using domain-specific knowledge and processes. By leveraging solver tools alongside LLMs, the paper highlights synergy potentials which can inspire further exploration into hybrid AI systems.

Looking ahead, several avenues warrant exploration. Firstly, expanding the dataset and extending the approach to other types of optimization problems could enhance OptiMUS's applicability. Additionally, fine-tuning LLMs for optimization-specific tasks and improving input handling for unstructured natural language inputs represent pathways for future enhancement.

In conclusion, this paper presents a compelling case for integrating LLMs with MIP solvers to automate optimization modeling. Through thoughtful methodological design and empirical validation, OptiMUS emerges as a promising tool in the evolving landscape of AI-driven optimization solutions.