CAMBranch: Contrastive Learning with Augmented MILPs for Branching (2402.03647v1)
Abstract: Recent advancements have introduced machine learning frameworks to enhance the Branch and Bound (B&B) branching policies for solving Mixed Integer Linear Programming (MILP). These methods, primarily relying on imitation learning of Strong Branching, have shown superior performance. However, collecting expert samples for imitation learning, particularly for Strong Branching, is a time-consuming endeavor. To address this challenge, we propose \textbf{C}ontrastive Learning with \textbf{A}ugmented \textbf{M}ILPs for \textbf{Branch}ing (CAMBranch), a framework that generates Augmented MILPs (AMILPs) by applying variable shifting to limited expert data from their original MILPs. This approach enables the acquisition of a considerable number of labeled expert samples. CAMBranch leverages both MILPs and AMILPs for imitation learning and employs contrastive learning to enhance the model's ability to capture MILP features, thereby improving the quality of branching decisions. Experimental results demonstrate that CAMBranch, trained with only 10\% of the complete dataset, exhibits superior performance. Ablation studies further validate the effectiveness of our method.
- Branching rules revisited. Oper. Res. Lett., 33(1):42–54, 2005. doi: 10.1016/j.orl.2004.04.002. URL https://doi.org/10.1016/j.orl.2004.04.002.
- Abien Fred Agarap. Deep learning using rectified linear units (relu). CoRR, abs/1803.08375, 2018. URL http://arxiv.org/abs/1803.08375.
- Design of virtual topologies for large optical networks through an efficient MILP formulation. Opt. Switch. Netw., 3(1):2–10, 2006. doi: 10.1016/j.osn.2005.10.002. URL https://doi.org/10.1016/j.osn.2005.10.002.
- A machine learning-based approximation of strong branching. INFORMS J. Comput., 29(1):185–195, 2017. doi: 10.1287/ijoc.2016.0723. URL https://doi.org/10.1287/ijoc.2016.0723.
- Finding cuts in the TSP (A preliminary report), volume 95. Citeseer, 1995.
- A theoretical analysis of contrastive unsupervised representation learning. arXiv preprint arXiv:1902.09229, 2019.
- E BALAS. Set covering algorithms using cutting planes, heuristics, and subgradient optimization: A computational study. Mathematical Programming Study, 12:37–60, 1980.
- Learning to branch. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 353–362. PMLR, 2018. URL http://proceedings.mlr.press/v80/balcan18a.html.
- Cyclic scheduling of multi-cluster tools based on mixed integer programming. IEEE Transactions on Semiconductor Manufacturing, 30(4):515–525, 2017.
- Machine learning for combinatorial optimization: A methodological tour d’horizon. Eur. J. Oper. Res., 290(2):405–421, 2021. doi: 10.1016/j.ejor.2020.07.063. URL https://doi.org/10.1016/j.ejor.2020.07.063.
- Unsupervised learning of visual features by contrasting cluster assignments. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https://proceedings.neurips.cc/paper/2020/hash/70feb62b69f16e0238f741fab228fec2-Abstract.html.
- A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 1597–1607. PMLR, 2020a. URL http://proceedings.mlr.press/v119/chen20j.html.
- Big self-supervised models are strong semi-supervised learners. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020b. URL https://proceedings.neurips.cc/paper/2020/hash/fcbc95ccdd551da181207c0c1400c655-Abstract.html.
- Exploring simple siamese representation learning. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 15750–15758. Computer Vision Foundation / IEEE, 2021. doi: 10.1109/CVPR46437.2021.01549. URL https://openaccess.thecvf.com/content/CVPR2021/html/Chen_Exploring_Simple_Siamese_Representation_Learning_CVPR_2021_paper.html.
- Improved baselines with momentum contrastive learning. CoRR, abs/2003.04297, 2020c. URL https://arxiv.org/abs/2003.04297.
- Cire and Andre Augusto. Decision diagrams for optimization. Constraints, 20(4):494–495, 2015.
- A comparison of heuristics and relaxations for the capacitated plant location problem. European Journal of Operational Research, 50(3):280–297, 1991.
- A theoretical and computational analysis of full strong-branching. Mathematical Programming, pp. 1–34, 2023.
- A framework for contrastive self-supervised learning and designing A new approach. CoRR, abs/2009.00104, 2020. URL https://arxiv.org/abs/2009.00104.
- CERT: contrastive self-supervised learning for language understanding. CoRR, abs/2005.12766, 2020. URL https://arxiv.org/abs/2005.12766.
- Fast graph representation learning with pytorch geometric. CoRR, abs/1903.02428, 2019. URL http://arxiv.org/abs/1903.02428.
- An exploratory computational analysis of dual degeneracy in mixed-integer programming. EURO J. Comput. Optim., 8(3):241–261, 2020. doi: 10.1007/s13675-020-00130-z. URL https://doi.org/10.1007/s13675-020-00130-z.
- Exact combinatorial optimization with graph convolutional neural networks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 15554–15566, 2019. URL https://proceedings.neurips.cc/paper/2019/hash/d14c2267d848abeb81fd590f371d39bd-Abstract.html.
- Neural message passing for quantum chemistry. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1263–1272. PMLR, 2017. URL http://proceedings.mlr.press/v70/gilmer17a.html.
- Declutr: Deep contrastive learning for unsupervised textual representations. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pp. 879–895. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.acl-long.72. URL https://doi.org/10.18653/v1/2021.acl-long.72.
- The SCIP Optimization Suite 6.0. Technical report, Optimization Online, July 2018. URL http://www.optimization-online.org/DB_HTML/2018/07/6692.html.
- Milp for a variant of pickup & delivery problem for both passengers and goods transportation. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2692–2698. IEEE, 2018.
- Bootstrap your own latent - A new approach to self-supervised learning. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https://proceedings.neurips.cc/paper/2020/hash/f3ada80d5c4ee70142b17b8192b2958e-Abstract.html.
- Hybrid models for learning to branch. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https://proceedings.neurips.cc/paper/2020/hash/d1e946f4e67db4b362ad23818a6fb78a-Abstract.html.
- A hybrid cp/milp method for scheduling with energy costs. European Journal of Industrial Engineering, 5(4):471–489, 2011.
- Momentum contrast for unsupervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 9726–9735. Computer Vision Foundation / IEEE, 2020. doi: 10.1109/CVPR42600.2020.00975. URL https://doi.org/10.1109/CVPR42600.2020.00975.
- Pretraining with contrastive sentence objectives improves discourse performance of language models. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 4859–4870. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.acl-main.439. URL https://doi.org/10.18653/v1/2020.acl-main.439.
- Learning combinatorial optimization algorithms over graphs. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6348–6358, 2017. URL https://proceedings.neurips.cc/paper/2017/hash/d9896106ca98d3d05b8cbdf4fd8b13a1-Abstract.html.
- Learning to branch in mixed integer programming. In Dale Schuurmans and Michael P. Wellman (eds.), Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA, pp. 724–731. AAAI Press, 2016. doi: 10.1609/aaai.v30i1.10080. URL https://doi.org/10.1609/aaai.v30i1.10080.
- Information-based branching schemes for binary linear mixed integer problems. Math. Program. Comput., 1(4):249–293, 2009. doi: 10.1007/s12532-009-0009-1. URL https://doi.org/10.1007/s12532-009-0009-1.
- Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6980.
- Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum?id=SJU4ayYgl.
- AH Land and AG Doig. An automatic method of solving discrete programming problems. Econometrica, 28(3):497–520, 1960.
- Towards a universal test suite for combinatorial auction algorithms. In Anant Jhingran, Jeff MacKie-Mason, and Doug J. Tygar (eds.), Proceedings of the 2nd ACM Conference on Electronic Commerce (EC-00), Minneapolis, MN, USA, October 17-20, 2000, pp. 66–76. ACM, 2000. doi: 10.1145/352871.352879. URL https://doi.org/10.1145/352871.352879.
- Learning to branch with tree-aware branching transformers. Knowl. Based Syst., 252:109455, 2022. doi: 10.1016/j.knosys.2022.109455. URL https://doi.org/10.1016/j.knosys.2022.109455.
- Michele Monaci Matteo Fischetti. Branching on nonchimerical fractionalities. Operations Research Letters, 40(3):159–164, 2012.
- Distributed representations of words and phrases and their compositionality. In Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger (eds.), Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pp. 3111–3119, 2013. URL https://proceedings.neurips.cc/paper/2013/hash/9aa42b31882ec039965f3c4923ce901b-Abstract.html.
- Solving mixed integer programs using neural networks. CoRR, abs/2012.13349, 2020. URL https://arxiv.org/abs/2012.13349.
- J Orbach. Principles of neurodynamics. perceptrons and the theory of brain mechanisms. Archives of General Psychiatry, 7(3):218–219, 1962.
- Supervised contrastive learning. Advances in Neural Information Processing Systems, 33:18661–18673, 2020.
- Pytorch: An imperative style, high-performance deep learning library. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 8024–8035, 2019. URL https://proceedings.neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html.
- Dean Pomerleau. Efficient training of artificial neural networks for autonomous navigation. Neural Comput., 3(1):88–97, 1991. doi: 10.1162/neco.1991.3.1.88. URL https://doi.org/10.1162/neco.1991.3.1.88.
- Learning to branch with tree mdps. In NeurIPS, 2022. URL http://papers.nips.cc/paper_files/paper/2022/hash/756d74cd58592849c904421e3b2ec7a4-Abstract-Conference.html.
- Synergy analysis of collaborative supply chain management in energy systems using multi-period milp. European Journal of Operational Research, 174(1):387–403, 2006.
- Representation learning with contrastive predictive coding. CoRR, abs/1807.03748, 2018. URL http://arxiv.org/abs/1807.03748.
- Pisces: A multi-modal data augmentation approach for drug combination synergy prediction. bioRxiv, pp. 2022–11, 2022.
- Are graph augmentations necessary?: Simple graph contrastive learning for recommendation. In Enrique Amigó, Pablo Castells, Julio Gonzalo, Ben Carterette, J. Shane Culpepper, and Gabriella Kazai (eds.), SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022, pp. 1294–1303. ACM, 2022. doi: 10.1145/3477495.3531937. URL https://doi.org/10.1145/3477495.3531937.
- Parameterizing branch-and-bound search trees to learn branching policies. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 3931–3939. AAAI Press, 2021. doi: 10.1609/aaai.v35i5.16512. URL https://doi.org/10.1609/aaai.v35i5.16512.
- A survey for solving mixed integer programming via machine learning. Neurocomputing, 519:205–217, 2023. doi: 10.1016/j.neucom.2022.11.024. URL https://doi.org/10.1016/j.neucom.2022.11.024.
- Jiacheng Lin (22 papers)
- Meng Xu (52 papers)
- Zhihua Xiong (5 papers)
- Huangang Wang (1 paper)