Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design

Abstract: Combining discrete and continuous data is an important capability for generative models. We present Discrete Flow Models (DFMs), a new flow-based model of discrete data that provides the missing link in enabling flow-based generative models to be applied to multimodal continuous and discrete data problems. Our key insight is that the discrete equivalent of continuous space flow matching can be realized using Continuous Time Markov Chains. DFMs benefit from a simple derivation that includes discrete diffusion models as a specific instance while allowing improved performance over existing diffusion-based approaches. We utilize our DFMs method to build a multimodal flow-based modeling framework. We apply this capability to the task of protein co-design, wherein we learn a model for jointly generating protein structure and sequence. Our approach achieves state-of-the-art co-design performance while allowing the same multimodal model to be used for flexible generation of the sequence or structure.

• The paper introduces DFMs that extend flow-based models to discrete state-spaces using CTMCs, enabling flexible and robust multimodal generation.
• It integrates discrete sequence generation with continuous structure prediction via the Multiflow model, achieving state-of-the-art performance in protein co-design tasks.
• Empirical results across text and protein generation demonstrate improved efficiency and fidelity, highlighting the framework's potential for advanced computational protein design.

Advanced Generative Framework for Protein Co-Design: A Deep Dive into Multiflow

Introduction to Discrete Flow Models (DFMs)

In the field of generative models, particularly within the field of protein design, the development of models that effectively handle both discrete and continuous data types holds substantial implications. Recent advancements presented in "Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design" introduce Discrete Flow Models (DFMs), a novel methodology extending flow-based generative models' capabilities to encompass both continuous and discrete data realms. The key innovation lies in the adaptation of Continuous Time Markov Chains (CTMCs) to discrete data, forming a bedrock for multimodal, flow-based generative modeling.

The Framework

DFMs operate under the principle of simulating a probability flow that transitions from noise towards actual data, emulating a continuous-time Markov Chain but in a discrete state-space. The model is elegant in its simplicity, providing a straightforward extension of flow-based models to discrete data environments, significantly expanding the versatility of generative modeling frameworks.

Central to this development is a novel discrete generative modeling method, built upon simulating CTMCs, thereby enabling an enhanced generative framework that encapsulates the generation of discrete modalities in concert with continuous data.

Multimodal Protein Co-Design with Multiflow

A pivotal application of DFMs is demonstrated through the Multiflow model, which showcases state-of-the-art performance in protein co-design tasks. Multiflow uniquely facilitates the joint generation of protein structure and sequence, leveraging the strengths of DFMs for sequence generation and incorporating continuous flow-based methods for structure prediction. This integrated approach realizes a seamless multimodal generative modeling, enabling flexible generation across protein structure and sequence modalities.

Empirical Validation on Text and Protein Generation

The model's efficacy was rigorously tested through comprehensive experiments in text and protein sequence generation. In text modeling, DFMs showcased superior performance over existing discrete diffusion alternatives, highlighting the framework's adaptability and efficiency across different modalities. Further, the application of Multiflow in protein generation tasks illuminated its profound potential, evidencing notable improvements in co-design performance, versatility in handling a wide range of protein lengths, and maintaining high fidelity in secondary structure prediction.

Insights and Future Directions

DFMs' introduction marks a significant leap in generative modeling, bridging the gap between discrete and continuous data types and paving the way for innovative applications in scientific domains. Multiflow, exemplifying this breakthrough, opens new horizons in protein design, offering a versatile tool for exploring the vast protein sequence and structure space.

The adaptability of Multiflow to perform forward and inverse folding tasks, albeit with initial baseline performances, further underscores the framework's potential as a general-purpose model for protein generation. This initial exploration sets the stage for future work aimed at refining performance across all protein generation tasks, heralding a new era of computational protein design.

As we continue to advance the frontiers of generative modeling, DFMs and Multiflow stand as testament to the transformative power of integrating discrete and continuous data paradigms, catalyzing innovation across bioinformatics and beyond.