Learning to Compare Nodes in Branch and Bound with Graph Neural Networks (2210.16934v1)
Abstract: Branch-and-bound approaches in integer programming require ordering portions of the space to explore next, a problem known as node comparison. We propose a new siamese graph neural network model to tackle this problem, where the nodes are represented as bipartite graphs with attributes. Similar to prior work, we train our model to imitate a diving oracle that plunges towards the optimal solution. We evaluate our method by solving the instances in a plain framework where the nodes are explored according to their rank. On three NP-hard benchmarks chosen to be particularly primal-difficult, our approach leads to faster solving and smaller branch- and-bound trees than the default ranking function of the open-source solver SCIP, as well as competing machine learning methods. Moreover, these results generalize to instances larger than used for training. Code for reproducing the experiments can be found at https://github.com/ds4dm/learn2comparenodes.
- Combinatorial auctions. Annals of Operations Research, 153:131–164, 2007.
- Tobias Achterberg. Constraint Integer Programming. PhD thesis, ZIB, Berlin, 2007.
- Constraint integer programming: Techniques and applications. ZIB-Report 08-43, Zuse Institute Berlin, 2008.
- A simulation scenario based mixed integer programming approach to airline reserve crew scheduling under uncertainty. Annals of Operations Research, 252:335–363, 2017.
- Machine learning for combinatorial optimization: A methodological tour d’horizon. European Journal of Operational Research, 290:405–421, 2021.
- Experiments in mixed-integer linear programming. Mathematical Programming, 1:76–94, 1971.
- Signature verification using a “siamese” time delay neural network. In Advances in Neural Information Processing Systems, volume 6, pages 737–744, 1993.
- Learning to rank using gradient descent. In Proceedings of the 22nd International Conference on Machine Learning, pages 89–96, 2005.
- Generating hard instances for MaxSAT. In Proceedings of the 2009 39th International Symposium on Multiple-Valued Logic, pages 191–195, 2009.
- Learning to schedule heuristics in branch and bound. In Advances in Neural Information Processing Systems, volume 34, pages 24235–24246, 2021.
- The generalized independent set problem: Polyhedral analysis and solution approaches. European Journal of Operational Research, 260:41–55, 2017.
- Robert J. Dakin. A tree-search algorithm for mixed integer programming problems. The Computer Journal, 8:250–255, 1965.
- Reinforcement learning for variable selection in a branch and bound algorithm. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pages 176–185, 2020.
- Mixed integer linear programming in process scheduling: Modeling, algorithms, and applications. Annals of Operations Research, 139:131–162, 2005.
- Practical solution of large mixed integer programming problems with UMPIRE. Management Science, 20:736–773, 1974.
- The SCIP Optimization Suite 7.0. ZIB-Report 20-10, Zuse Institute Berlin, 2020.
- Exact combinatorial optimization with graph convolutional neural networks. In Advances in Neural Information Processing Systems, volume 32, pages 15580–15592, 2019.
- A new model for learning in graph domains. In Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, volume 2, pages 729–734, 2005.
- Hybrid models for learning to branch. In Advances in Neural Information Processing Systems, volume 33, pages 18087–18097, 2020.
- Gurobi Optimization LLC. Gurobi Optimizer Reference Manual, 2020.
- A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4:100–107, 1968.
- Learning to search in branch-and-bound algorithms. In Advances in Neural Information Processing Systems, volume 27, pages 3293–3301, 2014.
- Combining exact and heuristic approaches for the capacitated fixed-charge network flow problem. INFORMS Journal on Computing, 22:314–325, 2010.
- Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- An automatic method for solving discrete programming problems. In 50 Years of Integer Programming 1958-2008, pages 105–132. Springer, 2010.
- Empirical decision model learning. Artificial Intelligence, 244:343–367, 2017.
- Solving mixed integer programs using neural networks. arXiv preprint arXiv:2012.13349, 2020.
- Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32, 2019.
- Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
- Dean A. Pomerleau. Efficient training of artificial neural networks for autonomous navigation. Neural Computation, 3:88–97, 1991.
- Efficient reductions for imitation learning. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 661–668. JMLR Workshop and Conference Proceedings, 2010.
- A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 627–635, 2011.
- Learning to search via retrospective imitation. arXiv preprint arXiv:1804.00846, 2018.
- Vladimir Vapnik. The Nature of Statistical Learning Theory. Springer Science & Business Media, 1999.
- A study of learning search approximation in mixed integer branch and bound: Node selection in SCIP. AI, 2:150–178, 2021.
- Parameterizing branch-and-bound search trees to learn branching policies. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 3931–3939, 2021.