Continuous SUN (Stable, Unique, and Novel) Metric for Generative Modeling of Inorganic Crystals
About
To address pressing scientific challenges such as climate change, increasingly sophisticated generative models are being developed to efficiently sample the large chemical space of potential functional materials. The proliferation of these models has necessitated the establishment of rigorous evaluation metrics. While uniqueness (U), novelty (N), and stability (S) of samples serve as standard metrics, their current formulations show several limitations. U and N rely on binary comparisons of crystals, rendering them dependent on heuristic thresholds, incapable of quantifying the degree of similarity, sensitive to atomic coordinate perturbations, and not invariant to sample permutation. Similarly, the binary assessment of S risks a premature exclusion of marginally unstable yet potentially novel candidates. These limitations are addressed by making the aforementioned metrics continuous. Furthermore, we integrate them into a unified metric ``continuous SUN" (cSUN), which offers a smoother score distribution and greater tunability than the conventional binary SUN metric. Experimental results demonstrate that our continuous metrics provide granular insights into sample distributions and facilitate the identification of the most promising candidates. Finally, the use of cSUN as a reward signal in reinforcement learning is explored, showing that its adjustable weighting scheme effectively mitigates reward hacking and avoids local minima.