Impact of optimizing designability on progress in generative protein design

Determine whether further optimizing the designability metric for protein structure generative models leads to meaningful progress in generative protein design. Here, designability refers to evaluating whether there exists a sequence that folds into a generated structure (typically assessed by designing sequences conditioned on the structure and testing refolding with a protein folding model).

Background

The paper argues that existing evaluation metrics for protein structure generative models—particularly designability—are becoming saturated, with some models achieving near-perfect scores despite limited practical success. The authors demonstrate that natural proteins exhibit lower designability than model-generated structures of similar lengths, raising concerns that further improvements in this metric may not reflect genuine advancement.

To address shortcomings in current evaluations, the authors introduce a protein Frechet Inception Distance (FID) that measures distributional similarity in a latent space and show it captures diversity and structural realism more robustly. In the Discussion, they explicitly state uncertainty about whether continuing to optimize designability will translate to meaningful progress, motivating the need to reassess the role of designability in guiding future model development.