Deep Learning for Two-Stage Robust Integer Optimization (2310.04345v3)
Abstract: Robust optimization is an established framework for modeling optimization problems with uncertain parameters. While static robust optimization is often criticized for being too conservative, two-stage (or adjustable) robust optimization (2RO) provides a less conservative alternative by allowing some decisions to be made after the uncertain parameters have been revealed. Unfortunately, in the case of integer decision variables, existing solution methods for 2RO typically fail to solve large-scale instances, limiting the applicability of this modeling paradigm to simple cases. We propose Neur2RO, a deep-learning-augmented instantiation of the column-and-constraint-generation (CCG) algorithm, which expands the applicability of the 2RO framework to large-scale instances with integer decisions in both stages. A custom-designed neural network is trained to estimate the optimal value and feasibility of the second-stage problem. The network can be incorporated into CCG, leading to more computationally tractable subproblems in each of its iterations. The resulting algorithm enjoys approximation guarantees which depend on the neural network's prediction error. In our experiments, Neur2RO produces high-quality solutions quickly, outperforming state-of-the-art methods on two-stage knapsack, capital budgeting, and facility location problems. Compared to existing methods, which often run for hours, Neur2RO finds better solutions in a few seconds or minutes. Our code is available at https://github.com/khalil-research/Neur2RO.
- Mixed integer programming: Analyzing 12 years of progress. In Facets of combinatorial optimization: Festschrift for martin grötschel, pp. 449–481. Springer, 2013.
- Strong mixed-integer programming formulations for trained neural networks. Mathematical Programming, pp. 1–37, 2020.
- Decomposition-based approaches for a class of two-stage robust binary optimization problems. INFORMS journal on computing, 34(2):857–871, 2022.
- Min-sup-min robust combinatorial optimization with few recourse solutions. INFORMS Journal on Computing, 34(4):2212–2228, 2022.
- Adjustable robust solutions of uncertain linear programs. Mathematical programming, 99(2):351–376, 2004.
- Robust optimization, volume 28. Princeton university press, 2009.
- Machine learning for combinatorial optimization: a methodological tour d’horizon. European Journal of Operational Research, 290(2):405–421, 2021.
- JANOS: an integrated predictive and prescriptive modeling framework. INFORMS Journal on Computing, 34(2):807–816, 2022.
- Finite adaptability in multistage linear optimization. IEEE Transactions on Automatic Control, 55(12):2751–2766, 2010.
- Design of near optimal decision rules in multistage adaptive mixed-integer optimization. Operations Research, 63(3):610–627, 2015.
- Binary decision rules for multistage adaptive mixed-integer optimization. Mathematical Programming, 167:395–433, 2018.
- Data-driven robust optimization. Mathematical Programming, 167:235–292, 2018.
- Robust combinatorial optimization under convex and discrete cost uncertainty. EURO Journal on Computational Optimization, 6(3):211–238, 2018.
- Bayesian nonparametric set construction for robust optimization. In 2015 American Control Conference (ACC), pp. 4216–4221. IEEE, 2015.
- OMLT: Optimization & machine learning toolkit. arXiv preprint arXiv:2202.02414, 2022.
- Maximum resilience of artificial neural networks. In International Symposium on Automated Technology for Verification and Analysis, pp. 251–268. Springer, 2017.
- Adjustable robust optimization with objective uncertainty. European Journal of Operational Research, 312(1):373–384, 2024.
- Neur2SP: Neural two-stage stochastic programming. Advances in Neural Information Processing Systems, 35, 2022.
- Neur2BiLO: Neural bilevel optimization. arXiv preprint arXiv:2402.02552, 2024.
- Deep neural networks and mixed integer linear optimization. Constraints, 23(3):296–309, 2018.
- A double-oracle, logic-based benders decomposition approach to solve the k-adaptability problem. Computers & Operations Research, 155:106243, 2023.
- Data-driven prediction of relevant scenarios for robust optimization. arXiv e-prints, pp. arXiv–2203, 2022.
- Data-driven robust optimization using deep neural networks. Computers & Operations Research, 151:106087, 2023.
- ReLU networks as surrogate models in mixed-integer linear programs. Computers & Chemical Engineering, 131:106580, 2019.
- Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2023. URL https://www.gurobi.com.
- K-adaptability in two-stage robust binary programming. Operations Research, 63(4):877–891, 2015.
- Machine learning for k𝑘kitalic_k-adaptability in two-stage robust optimization. arXiv preprint arXiv:2210.11152, 2022.
- Oracle-based algorithms for binary two-stage robust optimization. Computational Optimization and Applications, 77:539–569, 2020.
- The integration of explicit MPC and ReLU based neural networks. IFAC-PapersOnLine, 53(2):11350–11355, 2020. ISSN 2405-8963. doi:https://doi.org/10.1016/j.ifacol.2020.12.544. URL https://www.sciencedirect.com/science/article/pii/S2405896320308429. 21st IFAC World Congress.
- Modeling the AC power flow equations with optimally compact neural networks: Application to unit commitment. Electric Power Systems Research, 213:108282, 2022. ISSN 0378-7796. doi:https://doi.org/10.1016/j.epsr.2022.108282. URL https://www.sciencedirect.com/science/article/pii/S0378779622004771.
- Alternating mixed-integer programming and neural network training for approximating stochastic two-stage problems. arXiv preprint arXiv:2305.06785, 2023.
- Jannis Kurtz. Approximation algorithms for min-max-min robust optimization and k-adaptability under objective uncertainty. arXiv preprint arXiv:2106.03107, 2023.
- Adjustable robust optimization with discrete uncertainty. INFORMS Journal on Computing, 2023.
- On the number of linear regions of deep neural networks. Advances in neural information processing systems, 27, 2014.
- Neural networks for encoding dynamic security-constrained optimal power flow. arXiv preprint arXiv:2003.07939, 2020.
- Cutting-set methods for robust convex optimization with pessimizing oracles. Optimization Methods & Software, 24(3):381–406, 2009.
- Data-driven adaptive nested robust optimization: general modeling framework and efficient computational algorithm for decision making under uncertainty. AIChE Journal, 63(9):3790–3817, 2017.
- Data-driven decision making under uncertainty integrating robust optimization with principal component analysis and kernel smoothing methods. Computers & Chemical Engineering, 112:190–210, 2018.
- PyTorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024–8035. Curran Associates, Inc., 2019. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
- Operational research: Methods and applications. arXiv preprint arXiv:2303.14217, 2023.
- Multistage adjustable robust mixed-integer optimization via iterative splitting of the uncertainty set. INFORMS Journal on Computing, 28(3):553–574, 2016.
- Nonlinear hybrid planning with deep net learned transition models and mixed-integer linear programming. In IJCAI, pp. 750–756, 2017.
- Bounding and counting linear regions of deep neural networks. In International Conference on Machine Learning, pp. 4558–4566. PMLR, 2018.
- A data-driven robust optimization approach to scenario-based stochastic model predictive control. Journal of Process Control, 75:24–39, 2019.
- Data-driven robust optimization based on kernel learning. Computers & Chemical Engineering, 106:464–479, 2017.
- Large-scale industrial energy systems optimization under uncertainty: A data-driven robust optimization approach. Applied Energy, 259:114199, 2020.
- Anirudh Subramanyam. A lagrangian dual method for two-stage robust optimization with binary uncertainties. Optimization and Engineering, 23(4):1831–1871, 2022.
- K-adaptability in two-stage mixed-integer robust optimization. Mathematical Programming Computation, 12:193–224, 2020.
- Evaluating robustness of neural networks with mixed integer programming. arXiv preprint arXiv:1711.07356, 2017.
- An inexact column-and-constraint generation method to solve two-stage robust optimization problems. Operations Research Letters, 51(1):92–98, 2023.
- Robust optimization using machine learning for uncertainty sets. arXiv preprint arXiv:1407.1097, 2014.
- Learning for robust optimization. arXiv preprint arXiv:2305.19225, 2023a.
- Optimizing over an ensemble of trained neural networks. INFORMS Journal on Computing, 35(3):652–674, 2023b.
- Laurence A Wolsey. Integer programming. John Wiley & Sons, 2020.
- A survey of adjustable robust optimization. European Journal of Operational Research, 277(3):799–813, 2019.
- Deep sets. Advances in neural information processing systems, 30, 2017.
- Bo Zeng and Long Zhao. Solving two-stage robust optimization problems using a column-and-constraint generation method. Operations Research Letters, 41(5):457–461, 2013.
- A survey for solving mixed integer programming via machine learning. Neurocomputing, 519:205–217, 2023.
- Long Zhao and Bo Zeng. An exact algorithm for two-stage robust optimization with mixed integer recourse problems. submitted, available on Optimization-Online. org, 2012.