Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Mixed-Integer Optimisation of Graph Neural Networks for Computer-Aided Molecular Design (2312.01228v1)

Published 2 Dec 2023 in math.OC and cs.NE

Abstract: ReLU neural networks have been modelled as constraints in mixed integer linear programming (MILP), enabling surrogate-based optimisation in various domains and efficient solution of machine learning certification problems. However, previous works are mostly limited to MLPs. Graph neural networks (GNNs) can learn from non-euclidean data structures such as molecular structures efficiently and are thus highly relevant to computer-aided molecular design (CAMD). We propose a bilinear formulation for ReLU Graph Convolutional Neural Networks and a MILP formulation for ReLU GraphSAGE models. These formulations enable solving optimisation problems with trained GNNs embedded to global optimality. We apply our optimization approach to an illustrative CAMD case study where the formulations of the trained GNNs are used to design molecules with optimal boiling points.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. F. A. de Lima Ribeiro, M. M. C. Ferreira, QSPR models of boiling point, octanol–water partition coefficient and retention time index of polycyclic aromatic hydrocarbons, Journal of Molecular Structure: THEOCHEM 663 (2003) 109–126.
  2. QSPR: the correlation and quantitative prediction of chemical and physical properties from structure, Chemical Society Reviews 24 (1995) 279–287.
  3. Quantitative structure- property relationship (QSPR) models for boiling points, specific gravities, and refraction indices of hydrocarbons, Energy & Fuels 19 (2005) 152–163.
  4. B. F. Begam, J. S. Kumar, Computer assisted QSAR/QSPR approaches–a review, Indian Journal of Science and Technology 9 (2016) 1–8.
  5. M. D. Wessel, P. C. Jurs, Prediction of normal boiling points of hydrocarbons from molecular structure, Journal of Chemical Information and Computer Sciences 35 (1995) 68–76.
  6. Prediction of boiling points and critical temperatures of industrially important organic compounds from molecular structure, Journal of Chemical Information and Computer Sciences 34 (1994) 947–956.
  7. Quantitative structure–property relationships for the normal boiling temperatures of acyclic carbonyl compounds, Internet Electronic Journal of Molecular Design 1 (2002) 252–268.
  8. Prediction of the vapor pressure boiling point, heat of vaporization and diffusion coefficient of organic compounds, QSAR & Combinatorial Science 22 (2003) 565–574.
  9. A group contribution approach to computer-aided molecular design, AIChE Journal 37 (1991) 1318–1332.
  10. Generic mathematical programming formulation and solution for computer-aided molecular design, Computers & Chemical Engineering 78 (2015) 79–84.
  11. Deep learning and knowledge-based methods for computer-aided molecular design—toward a unified approach: State-of-the-art and future directions, Computers & Chemical Engineering 141 (2020) 107005.
  12. R. Gani, Group contribution-based property estimation methods: advances and perspectives, Current Opinion in Chemical Engineering 23 (2019) 184–196.
  13. Computer-aided molecular design: An introduction and review of tools, applications, and solution techniques, Chemical Engineering Research and Design 116 (2016) 2–26.
  14. Computer-aided molecular design in the continuous-molecular targeting framework using group-contribution pc-saft, Computers & Chemical Engineering 81 (2015) 278–287.
  15. R. H. Herring III, M. R. Eden, Evolutionary algorithm for de novo molecular design with multi-dimensional constraints, Computers & Chemical Engineering 83 (2015) 267–277.
  16. Computer-aided molecular design using genetic algorithms, Computers & Chemical Engineering 18 (1994) 833–844.
  17. A hybrid stochastic–deterministic optimization approach for integrated solvent and process design, Chemical Engineering Science 159 (2017) 207–216.
  18. Computer-aided molecular design using tabu search, Computers & Chemical Engineering 29 (2005) 337–347.
  19. Design of ionic liquids via computational molecular design, Computers & chemical engineering 34 (2010) 1476–1480.
  20. A comprehensive survey on graph neural networks, IEEE Transactions on Neural Networks and Learning Systems 32 (2020) 4–24.
  21. A compact review of molecular property prediction with graph neural networks, Drug Discovery Today: Technologies 37 (2020) 1–12.
  22. Graph machine learning for design of high-octane fuels, arXiv preprint arXiv:2206.00619 (2022).
  23. Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for drug-like molecules, Journal of Chemical Information and Modeling 53 (2013) 1563–1575.
  24. The graph neural network model, IEEE Transactions on Neural Networks 20 (2008) 61–80.
  25. Molecular geometry prediction using a deep generative graph neural network, Scientific reports 9 (2019) 1–13.
  26. Building attention and edge message passing neural networks for bioactivity and physical–chemical property prediction, Journal of Cheminformatics 12 (2020) 1–18.
  27. Low data drug discovery with one-shot learning, ACS Central Science 3 (2017) 283–293.
  28. Improving graph neural network expressivity via subgraph isomorphism counting, IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).
  29. Lanczosnet: Multi-scale deep graph convolutional networks, arXiv preprint arXiv:1901.01484 (2019).
  30. Deep convolutional networks on graph-structured data, arXiv preprint arXiv:1506.05163 (2015).
  31. Convolutional networks on graphs for learning molecular fingerprints, Advances in Neural Information Processing Systems 28 (2015).
  32. A fair comparison of graph neural networks for graph classification, arXiv preprint arXiv:1912.09893 (2019).
  33. Strategies for pre-training graph neural networks, arXiv preprint arXiv:1905.12265 (2019).
  34. Neural message passing for quantum chemistry, in: International conference on machine learning, PMLR, 2017, pp. 1263–1272.
  35. Strong mixed-integer programming formulations for trained neural networks, Mathematical Programming 183 (2020) 3–39.
  36. M. Fischetti, J. Jo, Deep neural networks and mixed integer linear optimization, Constraints 23 (2018) 296–309.
  37. When deep learning meets polyhedral theory: A survey, arXiv preprint arXiv:2305.00241 (2023).
  38. Partition-based formulations for mixed-integer optimization of trained relu neural networks, Advances in Neural Information Processing Systems 34 (2021) 3068–3080.
  39. A. M. Schweidtmann, A. Mitsos, Deterministic global optimization with artificial neural networks embedded, Journal of Optimization Theory and Applications 180 (2019) 925–948.
  40. Evaluating robustness of neural networks with mixed integer programming, arXiv preprint arXiv:1711.07356 (2017).
  41. Piecewise linear neural network verification: A comparative study, arXiv preprint arXiv:1711.00455 (2017).
  42. Output range analysis for deep neural networks, arXiv preprint arXiv:1709.09130 (2017).
  43. Equivalent and approximate transformations of deep neural networks, arXiv preprint arXiv:1905.11428 (2019).
  44. Lossless compression of deep neural networks, in: Proc. of International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research (CPAIOR’20), 2020, pp. 417–430.
  45. B. Grimstad, H. Andersson, Relu networks as surrogate models in mixed-integer linear programs, Computers & Chemical Engineering 131 (2019) 106580.
  46. A neural network based superstructure optimization approach to reverse osmosis desalination plants, Membranes 12 (2022) 199.
  47. Modeling the AC power flow equations with optimally compact neural networks: Application to unit commitment, Electric Power Systems Research 213 (2022) 108282.
  48. Data-driven process optimization considering surrogate model prediction uncertainty: A mixture density network-based approach, Industrial & Engineering Chemistry Research 60 (2021) 2206–2222.
  49. Continuous-molecular targeting for integrated solvent and process design, Industrial & Engineering Chemistry Research 49 (2010) 2834–2840.
  50. T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, arXiv preprint arXiv:1609.02907 (2016).
  51. Inductive representation learning on large graphs, Advances in Neural Information Processing Systems 30 (2017).
  52. A. Micheli, Neural network for graphs: A contextual constructive approach, IEEE Transactions on Neural Networks 20 (2009) 498–511.
  53. Learning convolutional neural networks for graphs, in: International conference on machine learning, PMLR, 2016, pp. 2014–2023.
  54. Large-scale learnable graph convolutional networks, in: Proc. of 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 1416–1424.
  55. Graph attention networks, arXiv preprint arXiv:1710.10903 (2017).
  56. Gaan: Gated attention networks for learning on large and spatiotemporal graphs, arXiv preprint arXiv:1803.07294 (2018).
  57. Geometric deep learning on graphs and manifolds using mixture model cnns, in: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR’17), 2017, pp. 5115–5124.
  58. Relational inductive biases, deep learning, and graph networks, arXiv preprint arXiv:1806.01261 (2018).
  59. Convolutional neural networks on graphs with fast localized spectral filtering, Advances in Neural Information Processing Systems 29 (2016).
  60. Three enhancements for optimization-based bound tightening, Journal of Global Optimization 67 (2017) 731–757.
  61. Graph neural networks for the prediction of molecular structure-property relationships, arXiv preprint arXiv:2208.04852 (2022).
  62. Pytorch: An imperative style, high-performance deep learning library, in: Advances in Neural Information Processing Systems 32, 2019, pp. 8024–8035. URL: http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
  63. M. Fey, J. E. Lenssen, Fast graph representation learning with pytorch geometric, arXiv preprint arXiv:1903.02428 (2019).
  64. Computer-assisted molecular design (camd)—an overview, Angewandte Chemie International Edition in English 26 (1987) 403–418.
  65. 1,1-Ethenediol: The long elusive enol of acetic acid, Angewandte Chemie International Edition 59 (2020) 5577–5580.
  66. M. Araki, N. Kuze, Laboratory detection of a linear carbon chain alcohol: Hc4oh and its deuterated species, The Astrophysical Journal 680 (2008) L93.
  67. Synthesis of hydrogen polyoxides H2O4 and H2O3 and their characterization by Raman spectroscopy, European Journal of Inorganic Chemistry 33 (2011) 5144–5150.
  68. A study of methanetetraol dehydration to carbonic acid, International Journal of Quantum Chemistry 62 (1997) 315–322.
  69. Acceleration techniques for optimization over trained neural network ensembles, arXiv preprint arXiv:2112.07007 (2021).
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now