AdsorbRL: Deep Multi-Objective Reinforcement Learning for Inverse Catalysts Design (2312.02308v1)

Abstract: A central challenge of the clean energy transition is the development of catalysts for low-emissions technologies. Recent advances in Machine Learning for quantum chemistry drastically accelerate the computation of catalytic activity descriptors such as adsorption energies. Here we introduce AdsorbRL, a Deep Reinforcement Learning agent aiming to identify potential catalysts given a multi-objective binding energy target, trained using offline learning on the Open Catalyst 2020 and Materials Project data sets. We experiment with Deep Q-Network agents to traverse the space of all ~160,000 possible unary, binary and ternary compounds of 55 chemical elements, with very sparse rewards based on adsorption energy known for only between 2,000 and 3,000 catalysts per adsorbate. To constrain the actions space, we introduce Random Edge Traversal and train a single-objective DQN agent on the known states subgraph, which we find strengthens target binding energy by an average of 4.1 eV. We extend this approach to multi-objective, goal-conditioned learning, and train a DQN agent to identify materials with the highest (respectively lowest) adsorption energies for multiple simultaneous target adsorbates. We experiment with Objective Sub-Sampling, a novel training scheme aimed at encouraging exploration in the multi-objective setup, and demonstrate simultaneous adsorption energy improvement across all target adsorbates, by an average of 0.8 eV. Overall, our results suggest strong potential for Deep Reinforcement Learning applied to the inverse catalysts design problem.

• The paper introduces AdsorbRL, a novel deep multi-objective reinforcement learning framework for inverse catalyst design that efficiently explores a 160,000-compound chemical space.
• It employs innovative techniques like Random Edge Traversal and Objective Sub-Sampling to focus on compounds with known data and improve binding energies by an average of 0.8 eV.
• The study demonstrates AI-driven methodologies can expedite catalyst development for clean energy and environmental applications by leveraging extensive public datasets.

Researchers at Stanford University have developed a sophisticated technology called AdsorbRL, which applies Deep Multi-Objective Reinforcement Learning (DRL) to the field of catalyst design. The technology is aimed at speeding up the identification of optimal catalysts for reactions pivotal to low-emissions and clean energy technologies. Catalysts are substances that can accelerate chemical reactions, and designing effective ones is key to various industrial processes, including the production of fuels and reduction of harmful emissions.

AdsorbRL utilizes a form of machine learning known as Reinforcement Learning (RL), where an agent learns to make decisions by interacting with an environment to achieve certain objectives. Here, the "environment" is a dataset comprising all possible combinations of unary, binary, and ternary compounds, with the goal of locating materials that have specific target adsorption energies. Adsorption energy is a critical parameter indicating how strongly a molecule, known as an adsorbate, binds to a catalyst's surface.

The significance of AdsorbRL lies in its ability to navigate a substantial chemical space—approximately 160,000 potential compounds—using only known adsorption energies from existing datasets like the Open Catalyst 2020 and the Materials Project. The system accomplishes this by training a DRL model to recognize catalysts that exhibit the strongest or weakest binding energies to a range of target adsorbates, particularly those crucial to the advancement of clean energy.

To make the problem tractable, the researchers devised innovative approaches like Random Edge Traversal and Objective Sub-Sampling. Random Edge Traversal limits the number of potential actions by the learning agent, allowing it to focus on compounds with known data. On the other hand, Objective Sub-Sampling is a training schema designed to enhance material discovery across various target adsorbates, encouraging a robust exploration of the chemical space. The technology exhibits a promising capability to simultaneously improve binding energies for multiple adsorbates by an average of 0.8 electronvolts (eV).

The paper illustrates the potential of DRL to tackle complex chemical spaces and suggests that it could be critical in developing new materials for applications in areas such as heterogeneous catalysis and electrochemical energy conversion. AdsorbRL represents a significant step forward in utilizing AI-driven methodologies for materials science, potentially expediting the discovery and manufacturing of new catalysts while conserving computational resources.

In summary, the AdsorbRL system introduces advanced methods in DRL for the purpose of inverse catalyst design. By navigating vast datasets to predict and optimize adsorption energies, this technology stands to make a substantial impact on the creation of efficient catalysts for environmental and energy-related applications. The researchers have also made their code and datasets publicly available, fostering further innovation in this exciting intersection of artificial intelligence and materials science.