An All-Atom Generative Model for Designing Protein Complexes

Abstract: Proteins typically exist in complexes, interacting with other proteins or biomolecules to perform their specific biological roles. Research on single-chain protein modeling has been extensively and deeply explored, with advancements seen in models like the series of ESM and AlphaFold2. Despite these developments, the study and modeling of multi-chain proteins remain largely uncharted, though they are vital for understanding biological functions. Recognizing the importance of these interactions, we introduce APM (All-Atom Protein Generative Model), a model specifically designed for modeling multi-chain proteins. By integrating atom-level information and leveraging data on multi-chain proteins, APM is capable of precisely modeling inter-chain interactions and designing protein complexes with binding capabilities from scratch. It also performs folding and inverse-folding tasks for multi-chain proteins. Moreover, APM demonstrates versatility in downstream applications: it achieves enhanced performance through supervised fine-tuning (SFT) while also supporting zero-shot sampling in certain tasks, achieving state-of-the-art results. We released our code at https://github.com/bytedance/apm.

An All-Atom Generative Model for Designing Protein Complexes

The study of proteins, especially their complex formations, has long been pivotal in understanding biological functions. However, while significant advancements have been made with models like ESM and AlphaFold in single-chain protein modeling, the multi-chain context remains relatively uncharted. Addressing this gap, the paper introduces the All-Atom Protein Generative Model (APM), a tool for designing protein complexes at the atomic level. This model aims to enhance our understanding and modeling of multi-chain proteins, which are integral for elucidating many biological processes.

Overview and Contributions

APM stands out by precisely modeling inter-chain interactions and designing protein complexes with specified binding capabilities. It is unique in its integration of atom-level data, facilitating complex structure modeling and multiple downstream applications, including protein folding, inverse folding, and functional design. APM supports zero-shot learning and achieves state-of-the-art (SOTA) results in several contexts without necessitating initial transformations often used in other models, such as pseudo sequences or residue-level information.

Key Contributions:

1. Native Multi-Chain Protein Modeling: Unlike previous models that rely on a continuous pseudo sequence connection, APM models multi-chain proteins with inherent inter-chain interactions.
2. All-Atom Structural Representation: APM accommodates residue-level information and sidechain conformations, maintaining computational efficiency while providing essential atom-level data for precise interaction modeling.
3. Sequence-Structure Dependency: The model addresses the challenge of disrupted sequence-structure dependency in hybrid generation tasks, enhancing dependencies through careful modulation of noising processes and training strategies.
4. Experimental Validation: APM's capabilities are extensively validated. It achieved SOTA performance in designing antibodies and binding peptides, alongside excelling in traditional single-chain tasks like folding and inverse-folding.

Model Architecture and Training

APM consists of three modules:

• Seq Module: Handles the co-generation of sequences and backbone structures at a residue level using flow-matching.
• Sidechain Module: Predicts sidechain conformations to provide the model with all-atom data.
• Refine Module: Refines sequences and structures based on all-atom data, optimizing outputs to more closely resemble natural proteins.

The integration of protein LLMs (PLMs), such as ESM2-650M, enhances APM's sequence understanding capabilities, aiding its predictions' accuracy.

Training occurs in two distinct phases. Phase I focuses on independent training of the Seq and Sidechain modules, while Phase II sees the introduction of the Refine module, with modules trained jointly using a novel consistency loss to ensure alignment and continuous improvement.

Experimental Results

APM diversifies its applicability across several domains:

• Single-Chain Tasks: It matches or exceeds the performance of existing models on standard folding and inverse-folding tasks, per self-consistency metrics such as scTM and AAR.
• Multi-Chain Protein Generation: APM's novel ability to generate multi-chain complexes is validated by demonstrating high binding affinity between chains, underscoring its potential for applications in drug design and synthetic biology.
• Downstream Functional Design Tasks: The paper details the model's success in designing functional proteins, specifically antibodies and binding peptides, achieving superior binding energy in both zero-shot and supervised fine-tuning settings.

Implications and Future Directions

APM sets the stage for several future explorations, most pertinently in its potential to enhance understanding of protein interactions at a molecular level. Its atomic precision supports advancements in rational drug design, bioengineering, and synthetic biology, especially in creating tailored proteins for therapeutic purposes.

However, as with any novel model, further refinements such as improved integration of biological constraints and better scaling may enhance future iterations of APM. Broader testing on a diverse range of proteins can also unveil more nuanced applications.

In summary, this work underlines a significant advance in modeling protein complexes, highlighting the importance of atomic-level precision in forecasting and designing complex biological functions.