ABodyBuilder3: Improved and scalable antibody structure predictions (2405.20863v1)

Abstract: Accurate prediction of antibody structure is a central task in the design and development of monoclonal antibodies, notably to understand both their developability and their binding properties. In this article, we introduce ABodyBuilder3, an improved and scalable antibody structure prediction model based on ImmuneBuilder. We achieve a new state-of-the-art accuracy in the modelling of CDR loops by leveraging LLM embeddings, and show how predicted structures can be further improved through careful relaxation strategies. Finally, we incorporate a predicted Local Distance Difference Test into the model output to allow for a more accurate estimation of uncertainties.

• The paper presents enhanced computational efficiency and accuracy by reengineering the model architecture and implementing advanced physics-based refinement.
• The paper integrates ProtT5 language model embeddings to improve CDR loop predictions, notably reducing CDRH3 RMSD from 2.54Å to 2.40Å.
• The paper introduces robust uncertainty estimation with pLDDT, providing reliable per-residue confidence scores for therapeutic antibody design.

Improved and Scalable Antibody Structure Predictions with ABodyBuilder3

The paper "ABodyBuilder3: Improved and scalable antibody structure predictions" introduces ABodyBuilder3, an enhanced model for predicting antibody structures, building on the foundation laid by ABodyBuilder2. This paper seeks to address a significant challenge in rational antibody design: the accurate prediction of complementarity-determining region (CDR) loops, particularly the CDRH3 loop, which is pivotal for antigen recognition.

Key Improvements and Methodological Advances

The improvements in ABodyBuilder3 are multifaceted, encompassing implementation efficiencies, enhanced data curation, and advanced structure refinement techniques. These advancements aim to significantly bolster the accuracy and scalability of antibody structure predictions.

1. Enhanced Implementation: The authors reengineer the ABodyBuilder2 architecture to enhance computational efficiency. By adopting vectorization and leveraging optimizations from the OpenFold project, they achieve a threefold increase in training speed. Notably, the model now employs mixed precision training, utilizing bfloat16 precision for weights, which leads to a more efficient memory footprint and faster computational throughput. This improves hardware utilization and supports distributed data parallel strategies for training across multiple GPUs.
2. Refinement Strategies: The paper evaluates two physics-based refinement methods, OpenMM and YASARA, for their impact on structural accuracy. The authors identify that minimization using the YASARA2 forcefield in explicit water significantly enhances the accuracy of the predicted structures, especially in the framework regions. This delicate balance of accurate stereochemical modelling and efficient computation is crucial for practical applications in therapeutic antibody development.
3. LLM Integration: A pivotal enhancement in ABodyBuilder3 is the incorporation of LLM embeddings derived from the ProtT5 model. By replacing the traditional one-hot-encoding with these embeddings, the model achieves improved performance in predicting the structures of CDRH3 and CDRL3 loops. The superior performance of general protein LLMs over antibody-specific models highlights the potential of leveraging extensive pre-trained models in specialized applications.
4. Uncertainty Estimation: To refine function of uncertainty estimation, the authors introduce a predicted Local Distance Difference Test (pLDDT). This model outputs per-residue uncertainty predictions that correlate more strongly with root mean squared deviation (RMSD) than previous ensemble-based uncertainty estimates. The paper demonstrated that pLDDT effectively differentiates high-confidence predictions, providing a robust framework for assessing model predictions without the computational overhead of ensemble methods.

Numerical Results and Comparative Analysis

The empirical advancements of ABodyBuilder3 are evidenced by rigorous evaluation against prior iterations. As detailed in Table~\ref{table:rmsd}, the model exhibits reduced RMSD across various antibody regions compared to ABodyBuilder2. Specifically, the RMSD for the critical CDRH3 loop decreases from 2.54Å to 2.40Å when using LLM embeddings.

Additionally, improvements in uncertainty estimation are quantified through Pearson correlation coefficients, as shown in Table~\ref{table:pearson}. The correlation between pLDDT scores and RMSD indicates a substantial alignment, significantly outperforming the ensemble-based uncertainty scores of ABodyBuilder2.

Implications and Future Developments

The enhancements introduced in ABodyBuilder3 present several impactful implications for both theoretical research and practical applications in antibody design:

• Scalability and Efficiency:

The improved computational efficiency enables the model to scale across multiple GPUs, facilitating the processing of larger datasets and accelerating the discovery pipeline for therapeutic antibodies.

• Accuracy in CDR Modelling:

Enhanced predictions for CDR loops, particularly CDRH3, pave the way for more precise antigen-binding predictions. This is a critical step towards the development of effective monoclonal antibodies.

• Robust Uncertainty Estimations:

Accurate uncertainty predictions allow researchers to assess the reliability of the predicted structures, crucial for downstream applications in drug design and immune response studies.

Future research could explore self-distillation techniques to marginally improve the model accuracy further and test the potential of a combined pLDDT and ensemble-based approach for nuanced uncertainty estimations. Additionally, expansion of training datasets using synthetic structures predicted from the Observed Antibody Space (OAS) could further enhance model robustness and generalization capabilities.

In conclusion, ABodyBuilder3 represents a significant step forward in the field of antibody structure prediction, combining computational efficiency with advanced accuracy measures and robust uncertainty estimations to support and accelerate therapeutic antibody development.