From Words to Molecules: A Survey of Large Language Models in Chemistry (2402.01439v1)

Abstract: In recent years, LLMs have achieved significant success in NLP and various interdisciplinary areas. However, applying LLMs to chemistry is a complex task that requires specialized domain knowledge. This paper provides a thorough exploration of the nuanced methodologies employed in integrating LLMs into the field of chemistry, delving into the complexities and innovations at this interdisciplinary juncture. Specifically, our analysis begins with examining how molecular information is fed into LLMs through various representation and tokenization methods. We then categorize chemical LLMs into three distinct groups based on the domain and modality of their input data, and discuss approaches for integrating these inputs for LLMs. Furthermore, this paper delves into the pretraining objectives with adaptations to chemical LLMs. After that, we explore the diverse applications of LLMs in chemistry, including novel paradigms for their application in chemistry tasks. Finally, we identify promising research directions, including further integration with chemical knowledge, advancements in continual learning, and improvements in model interpretability, paving the way for groundbreaking developments in the field.

Overview

The integration of LLMs into the chemistry sector represents a significant leap forward in the computational understanding and manipulation of chemical data. This paper, authored by Chang Liao, Yemin Yu, Yu Mei, and Ying Wei, delves deeply into how LLMs are being tailored and applied within the field of chemistry. It elucidates various methodologies for feeding molecular information into LLMs, examines adaptations required to suit the chemical context, and highlights a range of applications where LLMs are making significant contributions.

Molecule Encoding and Tokenization

A critical step in leveraging LLMs for chemical applications involves the representation and tokenization of molecular data to make it understandable for these models. The paper outlines two main approaches for molecular representation: Sequential Representations and Graph Representations, with a focus on SMILES (Simplified Molecular Input Line Entry System) and molecular graphs, respectively. Diverse tokenization methods—character-level, atom-level, and motif-level—are explored, each offering unique advantages and limitations in terms of capturing chemical structures' granularity and complexity.

Methodological Taxonomy

A comprehensive taxonomy categorizes the current LLM methodologies into three distinct groups based on the modality of input data: single-domain, multi-domain, and multi-modal approaches. Each category is dissected to illustrate how they contribute to learning molecular properties and behaviors, with single-domain approaches focusing purely on chemical data while multi-domain and multi-modal strategies incorporate broader data types, including textual descriptions and even images.

Pretraining Objectives

The paper discusses various pretraining objectives unique to chemical LLMs, such as Masked LLMing (MLM), Molecule Property Prediction (MPP), and Autoregressive Token Generation (ATG). It presents a detailed comparison of how these objectives are implemented across different models and highlights their contribution to enhancing model performance in chemical tasks.

Novel Paradigms and Applications

LLMs in chemistry are not limited to traditional computational chemistry tasks. The survey touches upon novel paradigms such as utilizing LLMs as chatbots for chemical inquiries, in-context learners that adapt and learn from real-time data without explicit retraining, and representation learners that encode complex molecular information into simpler, analyzable formats. These applications demonstrate the versatility and potential of LLMs beyond just data analysis, encompassing educational tools, adaptive learning systems, and intuitive interfaces for complex chemical data.

Future Directions

In the conclusive section, the paper outlines several promising research paths. These include a deeper integration of LLMs with chemical knowledge to enhance their understanding and generation capabilities, advancements in continual learning methodologies to keep pace with evolving chemical science, and improvements in model interpretability. Such directions are critical for harnessing the full potential of LLMs in the chemistry domain, ensuring their practicality, reliability, and usefulness in real-world applications.

Conclusion

The application of LLMs within chemistry stands as a testament to the interdisciplinary nature of modern scientific inquiry. This survey by Liao et al. not only offers a structured overview of the current landscape but also charts a course for future research, emphasizing the vast, untapped potential of merging AI with chemical science. As LLMs continue to evolve, their integration into chemical research and applications is poised to revolutionize our approach to solving complex chemical problems, ultimately accelerating discoveries and innovations in this pivotal field.