Steering Generative Models with Experimental Data for Protein Fitness Optimization (2505.15093v1)

Abstract: Protein fitness optimization involves finding a protein sequence that maximizes desired quantitative properties in a combinatorially large design space of possible sequences. Recent developments in steering protein generative models (e.g diffusion models, LLMs) offer a promising approach. However, by and large, past studies have optimized surrogate rewards and/or utilized large amounts of labeled data for steering, making it unclear how well existing methods perform and compare to each other in real-world optimization campaigns where fitness is measured by low-throughput wet-lab assays. In this study, we explore fitness optimization using small amounts (hundreds) of labeled sequence-fitness pairs and comprehensively evaluate strategies such as classifier guidance and posterior sampling for guiding generation from different discrete diffusion models of protein sequences. We also demonstrate how guidance can be integrated into adaptive sequence selection akin to Thompson sampling in Bayesian optimization, showing that plug-and-play guidance strategies offer advantages compared to alternatives such as reinforcement learning with protein LLMs.

• The paper presents the Steered Generation for Protein Optimization (SGPO) framework to guide discrete diffusion models with experimental fitness data for protein engineering.
• It evaluates strategies like classifier guidance and posterior sampling, showing they effectively enhance protein variant generation even with limited labeled data.
• This research offers a plug-and-play approach for efficient protein design, significantly reducing computational overhead and experimental iterations.

Steering Generative Models with Experimental Data for Protein Fitness Optimization

This paper presents an innovative paper focused on optimizing protein fitness through the use of generative models conditioned on experimental data. The core challenge addressed is the identification of protein sequences that exhibit optimal properties, like binding affinity or stability, in a vast combinatorial sequence space. The paper illustrates advanced methodologies for guiding generative models, namely discrete diffusion models, using minimal amounts of labeled sequence-fitness pairs obtained from wet-lab assays.

Main Contributions

The paper primarily contributes to the field of protein engineering by exploring various strategies for steering generative models with fitness data. The authors conduct a comprehensive evaluation of methods such as classifier guidance and posterior sampling, integrating them with discrete diffusion models for protein sequences. Key contributions include:

1. Framework Development: Establishes a general framework called Steered Generation for Protein Optimization (SGPO), providing a cohesive structure for integrating experimental fitness data with generative models.
2. Adaptive Optimization Integration: Introduces adaptive optimization techniques, akin to Bayesian optimization, that use uncertainty-aware exploration to improve sequence selection and enhance model guidance effectiveness.
3. Evaluative Insights: Offers an extensive evaluation of different design choices and guidance strategies within the SGPO framework, highlighting the conditions under which each strategy performs best.

Methods and Evaluation

The paper contrasts different generative models and methods for guiding them with labeled data. It compares discrete diffusion models using various transition matrices, like those involving uniform noise and masking, to guide sequence generation effectively. The results indicate that classifier guidance and posterior sampling (e.g., decoupled annealing posterior sampling) offer potent strategies for enhancing protein variant generation, even with limited labeled data.

The evaluation was conducted using datasets for proteins such as TrpB and CreiLOV, employing thousands of sequence-fitness pairs for training predictive models. Importantly, the paper implements a plug-and-play approach, achieving guidance without the need for extensive model retraining, thus reducing computational overhead.

Implications and Future Directions

Practically, this research paves the way for more efficient protein engineering by marrying machine learning with experimental feedback, optimizing for specific protein attributes with fewer iterations. Theoretically, it suggests promising avenues for further paper on the integration of generative models with experimental data beyond protein engineering, potentially applicable to other complex biological systems.

Moving forward, further refinement of guidance strategies, especially those that can incorporate more complex fitness landscapes or multi-objective optimization scenarios, could broaden the applicability and efficiency of these models. The paper also highlights emerging interests in adapting these methodologies for broader applications in molecular design and potentially drug discovery.

In conclusion, this paper significantly propels the field of protein design by illustrating how advanced AI techniques can be integrated with experimental data to efficiently explore and exploit the vast protein sequence landscape, pushing the boundaries of what's possible in protein fitness optimization.