Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PySCIPOpt-ML: Embedding Trained Machine Learning Models into Mixed-Integer Programs (2312.08074v2)

Published 13 Dec 2023 in math.OC and cs.AI

Abstract: A standard tool for modelling real-world optimisation problems is mixed-integer programming (MIP). However, for many of these problems, information about the relationships between variables is either incomplete or highly complex, making it difficult or even impossible to model the problem directly. To overcome these hurdles, ML predictors are often used to represent these relationships and are then embedded in the MIP as surrogate models. Due to the large amount of available ML frameworks and the complexity of many ML predictors, formulating such predictors into MIPs is a highly non-trivial task. In this paper, we introduce PySCIPOpt-ML, an open-source tool for the automatic formulation and embedding of trained ML predictors into MIPs. By directly interfacing with a broad range of commonly used ML frameworks and an open-source MIP solver, PySCIPOpt-ML provides a way to easily integrate ML constraints into optimisation problems. Alongside PySCIPOpt-ML, we introduce, SurrogateLIB, a library of MIP instances with embedded ML constraints, and present computational results over SurrogateLIB, providing intuition on the scale of ML predictors that can be practically embedded. The project is available at https://github.com/Opt-Mucca/PySCIPOpt-ML.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (28)
  1. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems, 2015. Software available from tensorflow.org.
  2. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  3. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pages 785–794. ACM, 2016.
  4. LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30:3146–3154, 2017.
  5. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems, 32:8024–8035, 2019.
  6. Enabling Research through the SCIP Optimization Suite 8.0. ACM Trans. Math. Softw., 49(2), 2023.
  7. An Analytics Approach to Designing Combination Chemotherapy Regimens for Cancer. Management Science, 62(5):1511–1531, 2016.
  8. Assessment of carbon-aware flexibility measures from data centres using machine learning. IEEE Transactions on Industry Applications, 59(1):70–80, 2023.
  9. Beyond price taker: Conceptual design and optimization of integrated energy systems using machine learning market surrogates. Applied Energy, 351:121767, 2023.
  10. Hybrid machine learning-mathematical programming approach for optimizing gas production and water management in shale gas fields. ACS Sustainable Chemistry & Engineering, 11, 2023.
  11. Analytics for an online retailer: Demand forecasting and price optimization. Manufacturing & Service Operations Management, 18, 1 2015.
  12. Predictive and prescriptive analytics for location selection of add-on retail products. Production and Operations Management, 28, 4 2018.
  13. Boosting combinatorial problem modeling with machine learning. arXiv preprint arXiv:1807.05517, 2018.
  14. Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Computers & Chemical Engineering, 108:250–267, 2018.
  15. Overview of surrogate modeling in chemical process engineering. Chemie Ingenieur Technik, 91(3):228–239, 2019.
  16. Gurobi Optimization. Gurobi-machinelearning, 2023.
  17. JANOS: an integrated predictive and prescriptive modeling framework. INFORMS Journal on Computing, 34(2):807–816, 2022.
  18. ENTMOOT: a framework for optimization over ensemble tree models. Computers & Chemical Engineering, 151:107343, 2021.
  19. reluMIP: Open Source Tool for MILP Optimization of ReLU Neural Networks, 2021.
  20. Pyomo–optimization modeling in Python, volume 67. Springer Science & Business Media, third edition, 2021.
  21. OMLT: Optimization & machine learning toolkit. The Journal of Machine Learning Research, 23(1):15829–15836, 2022.
  22. ReLU networks as surrogate models in mixed-integer linear programs. Computers & Chemical Engineering, 131:106580, 2019.
  23. Deterministic global optimization with artificial neural networks embedded. Journal of Optimization Theory and Applications, 180(3):925–948, 2019.
  24. Strong mixed-integer programming formulations for trained neural networks. Mathematical Programming, 183(1-2):3–39, 2020.
  25. Between steps: Intermediate relaxations between big-M and convex hull formulations. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pages 299–314. Springer, 2021.
  26. Mixed-integer convex nonlinear optimization with gradient-boosted trees embedded. INFORMS Journal on Computing, 33(3):1103–1119, 2021.
  27. Linear Model Decision Trees as Surrogates in Optimization of Engineering Applications. Computers & Chemical Engineering, 178, 2023.
  28. Branch-and-cut for linear programs with overlapping sos1 constraints. Mathematical Programming Computation, 10:33–68, 2018.
Citations (5)

Summary

We haven't generated a summary for this paper yet.