Robust evaluation of novelty and diversity in generative crystal structure models

Develop standardized, quantitative evaluation metrics and protocols to assess novelty and diversity in generative models for crystalline materials, ensuring that these measures go beyond duplicate counting and align with the practical goal of new material design by capturing substantial, physically meaningful sample diversity.

Background

In this work, the authors evaluate generative models for crystals using a suite of metrics, including structural and compositional validity, Coverage and Property EMD, stability-based S.U.N., space-group-related symmetry measures, and exact-duplicate filtering for novelty. They also introduce element-agnostic Wyckoff template novelty to better reflect symmetry-conditioned diversity.

Despite these efforts, the authors note that current novelty and diversity assessments have limitations: exact-duplicate filtering can be an incomplete proxy for meaningful diversity, and some commonly used metrics may reward sampling close to the training distribution rather than promoting truly novel and useful discoveries. This motivates the need for robust, standardized evaluation methods that better capture the diversity and usefulness of generated crystals for materials design.