Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep reinforcement learning uncovers processes for separating azeotropic mixtures without prior knowledge (2310.06415v1)

Published 10 Oct 2023 in cs.LG

Abstract: Process synthesis in chemical engineering is a complex planning problem due to vast search spaces, continuous parameters and the need for generalization. Deep reinforcement learning agents, trained without prior knowledge, have shown to outperform humans in various complex planning problems in recent years. Existing work on reinforcement learning for flowsheet synthesis shows promising concepts, but focuses on narrow problems in a single chemical system, limiting its practicality. We present a general deep reinforcement learning approach for flowsheet synthesis. We demonstrate the adaptability of a single agent to the general task of separating binary azeotropic mixtures. Without prior knowledge, it learns to craft near-optimal flowsheets for multiple chemical systems, considering different feed compositions and conceptual approaches. On average, the agent can separate more than 99% of the involved materials into pure components, while autonomously learning fundamental process engineering paradigms. This highlights the agent's planning flexibility, an encouraging step toward true generality.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. R.W.H. Sargent “Advances in modelling and analysis of chemical process systems” In Comput. Chem. Eng. 7, 1983, pp. 219–237
  2. V. Venkatasubramanian “The Promise of Artificial Intelligence in Chemical Engineering: Is It Here, Finally?” In AIChE J. 65, 2018, pp. 225502
  3. J.H. Lee, J. Shin and M.J. Realff “Machine learning: Overview of the recent progresses and implications for the process systems engineering field” In Comput. Chem. Eng. 114, 2018, pp. 111–121
  4. “Machine Learning in Chemical Engineering: Strengths, Weaknesses, Opportunities, and Threats” In Engineering 7.9, 2021, pp. 1201–1211
  5. “Machine Learning in Chemical Engineering: A Perspective” In Chem. Ing. Tech. 93, 2021, pp. 2029–2039
  6. “Process systems engineering - The generation next?” In Comput. Chem. Eng. 147, 2021, pp. 107252
  7. “Analyzing Learned Molecular Representations for Property Prediction” In J. Chem. Inf. Model. 59.8, 2019, pp. 3370–3388
  8. “A graph-convolutional neural network model for the prediction of chemical reactivity” In Chem. Sci. 10, 2019, pp. 370–377
  9. “Large-scale chemical language representations capture molecular structure and properties” In Nat. Mach. Intell. 4, 2022, pp. 1256–1264
  10. “Mapping the space of chemical reactions using attention-based neural networks” In Nat. Mach. Intell. 3, 2021, pp. 144–152
  11. Z. Zhou, X. Li and R.N. Zare “Optimizing Chemical Reactions with Deep Reinforcement Learning” In ACS Cent. Sci. 3.12, 2017, pp. 1337–1344
  12. “Overview of Surrogate Modeling in Chemical Process Engineering” In Chem. Ing. Tech. 91.3, 2019, pp. 1–13
  13. J.J. Siirola “Strategic process synthesis: advances in the hierarchical approach” In Comput. Chem. Eng. 20.2, 1996, pp. 1637–1643
  14. A.W. Westerberg “A retrospective on design and process synthesis” In Comput. Chem. Eng. 28.4, 2004, pp. 447–458
  15. “From conceptual design to process design optimization: a review on flowsheet synthesis” In Oil Gas Sci. Technol. 74.80, 2019
  16. R.L. Kirkwood, M.H. Locke and J.M. Douglas “A prototype expert system for synthesizing chemical process flowsheets” In Comput. Chem. Eng. 12.4, 1988, pp. 329–343
  17. “A knowledge based system for the selection of thermodynamic models” In Comput. Chem. Eng. 13.4-5, 1989, pp. 397–404
  18. “A review on superstructure optimization approaches in process system engineering” In Comput. Chem. Eng. 136.106808, 2020
  19. “Mastering the game of Go without human knowledge” In Nature 550, 2017, pp. 354–359
  20. “A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play” In Science 362, 2018, pp. 1140–1144
  21. “Discovering faster matrix multiplication algorithms with reinforcement learning” In Nature 610.7930 Nature Publishing Group, 2022, pp. 47–53
  22. “Faster sorting algorithms discovered using deep reinforcement learning” In Nature 618.7964 Nature Publishing Group UK London, 2023, pp. 257–263
  23. L.I. Midgley “Deep Reinforcement Learning for Process Synthesis”, 2020 URL: https://doi.org/10.48550/arXiv.2009.13265
  24. “Searching for optimal process routes: A reinforcement learning approach” In Comput. Chem. Eng. 141, 2020, pp. 107027
  25. Q. Göttl, D.G. Grimm and J. Burger “Automated synthesis of steady-state continuous processes using reinforcement learning” In Front. Chem. Sci. Eng. 16, 2022, pp. 288–302
  26. Q. Göttl, D.G. Grimm and J. Burger “Automated Process Synthesis Using Reinforcement Learning” In Comput. Aided Chem. Eng. 50, 2021, pp. 209–214
  27. “Automated Flowsheet Synthesis Using Hierarchical Reinforcement Learning: Proof of Concept” In Chem. Ing. Tech. 93.12, 2021, pp. 2010–2018
  28. Q. Göttl, D.G. Grimm and J. Burger “Using Reinforcement Learning in a Game-like Setup for Automated Process Synthesis without Prior Process Knowledge” In Comput. Aided Chem. Eng. 49, 2022, pp. 1555–1560
  29. “Designing the process designer: Hierarchical reinforcement learning for optimisation-based process design” In Chem. Eng. Process. 180, 2022, pp. 108885
  30. J.R. Seidenberg, A.A. Khan and A.A. Lapkin “Boosting autonomous process design and intensification with formalized domain knowledge” In Comput. Chem. Eng. 169, 2023, pp. 108097
  31. “Flowsheet generation through hierarchical reinforcement learning and graph neural networks” In AIChE J. 69.1, 2023, pp. 17938
  32. “Proximal Policy Optimization Algorithms”, 2017 URL: https://doi.org/10.48550/arXiv.1707.06347
  33. “Transfer learning for process design with reinforcement learning” In Comput. Aided Chem. Eng. 52, 2023, pp. 2005–2010
  34. “Policy improvement by planning with gumbel” In ICLR, 2022
  35. “MLP-mixer: An all-MLP architecture for vision” In NeurIPS 34, 2021, pp. 24261–24272
  36. “Reinforcement learning: An introduction” MIT press, 2018
  37. “Attention is all you need” In NeurIPS, 2017, pp. 5998–6008
  38. “Unique Design Considerations for Maximum-Boiling Azeotropic Systems via Extractive Distillation: Acetone/Chloroform Separation” In Ind. Eng. Chem. Res. 57.38, 2018, pp. 12884–12894
  39. “Techno-economic comparison of energy usage between azeotropic distillation and hybrid system for water-ethanol separation” In Renew. Energ. 51, 2013, pp. 310–316
  40. W.L. Luyben “Control of the Heterogeneous Azeotropic n-Butanol/Water Distillation System” In Energ. Fuel. 22, 2008, pp. 4249–4258
  41. “Design and Control of a Hybrid Extraction/Distillation System for the Separation of Pyridine and Water” In Ind. Eng. Chem. 54, 2015, pp. 7715–7727
  42. O. Ryll, S. Blagov and H. Hasse “Convex envelope method for the determination of fluid phase diagrams” In Fluid Ph. Equilibria 324, 2012, pp. 108–116
  43. O. Ryll, S. Blagov and H. Hasse “Inf/Inf Analysis of homogeneous distillation processes” In Chem. Eng. Sci. 84, 2012, pp. 315–332
  44. “Convex Envelope Method for determining liquid multi-phase equilibria in systems with arbitrary number of components” In Comput. Chem. Eng. 177, 2023, pp. 108321
  45. “Learning values across many orders of magnitude” In Advances in Neural Information Processing Systems 29, 2016
  46. “Observe and Look Further: Achieving Consistent Performance on Atari” In arXiv:1805.11593, 2018
  47. “Policy-Based Self-Competition for Planning Problems” In International Conference on Learning Representations, 2023
  48. C.M. Bishop “Pattern Recognition and Machine Learning” New York: Springer, 2006
  49. O. Ryll “Thermodynamische Analyse gekoppelter Reaktions-Destillations-Prozesse: konzeptioneller Entwurf, Modellierung, Simulation und experimentelle Validierung (in German)”, 2009
  50. “Multiple Steady States in Distillation: Inf/Inf Predictions, Extensions, and Implications for Design, Synthesis, and Simulation” In Ind. Eng. Chem. Res. 35, 1996, pp. 4264–4280
  51. “Chemical Engineering Design: Principles, Practice and Economics of Plant and Process Design” Butterworth-Heinemann, 2022
  52. “NIST Chemistry WebBook” Gaithersburg MD, 20899: National Institute of StandardsTechnology, 2023
  53. DDBST GmbH “Dortmund Data Bank”, 2023 URL: www.ddbst.com
  54. Aspen Technology Inc. “Aspen Plus”, 2015
  55. “Basic Theoretical Physics” Berlin, Heidelberg: Springer, 2007
  56. J.I. Yellott Jr. “The relationship between Luce’s choice axiom, Thurstone’s theory of comparative judgment, and the double exponential distribution” In Journal of Mathematical Psychology 15.2 Elsevier, 1977, pp. 109–144
  57. T. Vieira “Gumbel-max trick and weighted reservoir sampling”, http://timvieira.github.io/blog/post/2014/08/01/gumbel-max-trick-and-weighted-reservoir-sampling/, 2014
  58. Z. Karnin, T. Koren and O. Somekh “Almost Optimal Exploration in Multi-Armed Bandits” In ICML, 2013
  59. J.M. Prausnitz, R.N. Lichtenthaler and E.G.d. Azevedo “Molecular Thermodynamics of Fluid-Phase Equilibria” Prentice Hall PTR, 1999
  60. F.B. Petlyuk “Distillation Theory and Its Application to Optimal Design of Separation Units” Cambridge University Press, 2004
  61. L.T. Biegler, I.E. Grossmann and A.W. Westerberg “Systematic Methods of Chemical Process Design” Prentice Hall PTR, 1997
  62. “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python” In Nature Methods 17, 2020, pp. 261–272

Summary

We haven't generated a summary for this paper yet.