CliqueFlowmer: Neural MBO for Materials Discovery

• CliqueFlowmer is a neural architecture that fuses clique-based surrogate modeling, transformer encoders/decoders, and flow-based geometry decoding for computational materials discovery.
• It employs rank-based evolution strategies to optimize latent representations, yielding superior formation energies, band gaps, and S.U.N. rates.
• The framework advances CMD by systematically balancing property optimization with the generation of stable, unique, and novel crystal structures.

CliqueFlowmer is a neural model architecture and offline model-based optimization (MBO) framework for computational materials discovery (CMD), designed to produce crystal structures maximizing or minimizing specific materials properties, such as formation energy or band gap. Unlike purely generative approaches that operate via maximum likelihood training, CliqueFlowmer fuses property optimization with discrete atom generation and continuous geometry decoding by incorporating a clique-based surrogate, transformer encoders/decoders, and conditional flow-based geometry modeling. Empirical results show CliqueFlowmer achieves substantially superior property values and high rates of stability, uniqueness, and novelty relative to previous state-of-the-art generative models (Kuba et al., 6 Mar 2026).

1. Architectural Overview

CliqueFlowmer is conceptualized as an auto-encoder plus surrogate model comprising three principal phases: encoding, clique decomposition and surrogate prediction, and decoding.

• Encoding: The model receives as inputs the lattice lengths $(a,b,c)$, lattice angles $(\alpha,\beta,\gamma)$, fractional atomic positions $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$, and atom types $t\in\mathcal{A}^{N_{\mathrm{atom}}}$. Each is embedded using distinct MLPs and an atom-type embedding layer. The outputs are concatenated to a sequence that is processed by a transformer encoder with AdaLN (adaptive layer normalization), producing a contextualized representation $$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$. Attention pooling with a learned query yields a pooled vector $h^{\mathrm{pool}}$, which is mapped to mean and log-variance for a latent Gaussian $z\sim\mathcal{N}(\mu, \mathrm{diag}(\sigma^2))$.
• Clique Decomposition & Surrogate: The latent $z\in\mathbb{R}^d$ is reshaped into $N_{\text{clique}}$ overlapping cliques $z_c\in\mathbb{R}^{d_{\rm clique}}$ (with overlap $(\alpha,\beta,\gamma)$0). The property predictor is a sum of small MLPs over cliques:

$(\alpha,\beta,\gamma)$1

• Decoding: The decoder bifurcates into (i) an atom-type autoregressive transformer, and (ii) a conditional normalizing flow for geometry. The atom-type decoder maps $(\alpha,\beta,\gamma)$2 to $(\alpha,\beta,\gamma)$3 and employs a causal transformer, conditioned by AdaLN. The geometry decoder uses a denoising velocity network $(\alpha,\beta,\gamma)$4 trained using a flow-matching loss over interpolations between prior and ground truth geometry.

2. Clique-Based Model-Based Optimization

Offline MBO is enabled by the surrogate’s additive clique structure. The optimization problem is to find $(\alpha,\beta,\gamma)$5 minimizing the predicted property:

$(\alpha,\beta,\gamma)$6

Latents are regularized towards the prior $(\alpha,\beta,\gamma)$7 via AdamW weight decay, which maintains proximity to the training manifold. In-distribution clique fragments can be "stitched" to synthesize new candidate materials.

Evolution strategies (ES), specifically rank-based ES with antithetic perturbations and standardized rank differences, are used to optimize latents:

$(\alpha,\beta,\gamma)$8

where $(\alpha,\beta,\gamma)$9 are ranks of $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$0 and $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$1 are perturbations.

3. Training Objectives and Procedures

Training utilizes four objectives:

• Atom-type log-likelihood: $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$2 from the autoregressive transformer decoder.
• Flow matching loss: $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$3 for geometry, with loss

$p\in[0,1)^{N_{\mathrm{atom}}\times 3}$4

with $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$5 interpolating between prior and data.

• Prediction loss: $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$6.
• Clique-wise latent KL: $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$7, averaged over cliques.

The total loss is

$p\in[0,1)^{N_{\mathrm{atom}}\times 3}$8

with $p\in[0,1)^{N_{\mathrm{atom}}\times 3}$9 and $t\in\mathcal{A}^{N_{\mathrm{atom}}}$0 as warm-up weights.

Key training parameters: MP-20 dataset (45k crystals), batch size 1024, 700k steps, learning rate $t\in\mathcal{A}^{N_{\mathrm{atom}}}$1, dropout 0.1, latent dimension $t\in\mathcal{A}^{N_{\mathrm{atom}}}$2, $t\in\mathcal{A}^{N_{\mathrm{atom}}}$3, $t\in\mathcal{A}^{N_{\mathrm{atom}}}$4, $t\in\mathcal{A}^{N_{\mathrm{atom}}}$5.

Inference proceeds by encoding empirical lattices, optimizing $t\in\mathcal{A}^{N_{\mathrm{atom}}}$6 via ES for 2,000 steps, decoding atom types by beam search, and decoding geometry using flow-matching under classifier-free guidance (CFG, $t\in\mathcal{A}^{N_{\mathrm{atom}}}$7).

4. Algorithmic Implementation

Pseudocode implementations are provided for key subroutines:

• Clique chaining (PyTorch):

$$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$5

• Rank-ES gradient estimator (NumPy):

$$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$6

• Flow-matching training loop:

$$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$7

• Inference pipeline (pseudo):

$$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$8

5. Quantitative Evaluation and Comparative Performance

Performance is evaluated on the MP-20 dataset using metrics formalized as follows:

• Formation energy per atom: $t\in\mathcal{A}^{N_{\mathrm{atom}}}$8.
• Band gap: $t\in\mathcal{A}^{N_{\mathrm{atom}}}$9.
• Stability: Design is “strictly stable” if DFT energy above hull $$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$0.
• Uniqueness: Fraction of non-duplicate designs.
• Novelty: Not present in training set.
• S.U.N. rate: Designs that are stable, unique, and novel.
• Top-$$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$1 property: Mean of best $$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$2 designs post-MBO.

Empirical results (Table: selected metrics):

Metric	CrystalFormer	DiffCSP	DiffCSP++	MatterGen	CliqueFlowmer	Top-10%
E_form (↓)	0.71	0.59	0.65	0.60	–0.81	–0.99
Band Gap (↓)	0.52	0.63	0.48	0.57	0.03	0.07
S.U.N. rate (↑)	12.8 %	18.6 %	18.5 %	17.6 %	61.3 %	69.4 %

CliqueFlowmer achieves average formation energies of –0.81 eV/atom (–0.99 eV/atom for Top 10%). For band gap, output is driven to 0.03 eV (0.07 eV for Top 10%). In the S.U.N. metric for band-gap-optimized designs, CliqueFlowmer achieves $$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$3, whereas generative baselines attain around $$H^{\mathrm{out}}\in\mathbb{R}^{(N_{\mathrm{atom}}+2)\times d_{\mathrm{model}}}$$4.

6. Comparative Analysis

Generative baselines such as CrystalFormer, DiffCSP, MatterGen, and FlowMM sample from a maximum likelihood-trained model and subsequently select candidates with the best predicted properties, restricting exploration to the data manifold and failing to systematically push into regions of optimal property values. CliqueFlowmer, in contrast, couples property gradients (via rank-based ES in latent space) with generative decoding, traversing aggressively toward property optima. The clique-based structure further supports combinatorial recombination of in-distribution latents, balancing exploration with distributional realism.

Empirically, CliqueFlowmer consistently discovers materials with superior formation energy and band gap metrics, while maintaining or exceeding generative baselines in stability, uniqueness, and novelty of candidate materials (Kuba et al., 6 Mar 2026).

7. Significance and Applications

CliqueFlowmer demonstrates a new paradigm for computational materials optimization by integrating latent-space property surrogates that are both decomposable and amenable to ES-based gradient estimation, transformer-based sequence modeling for discrete atom prediction, and flow-based geometry generation. The architecture is fully differentiable and amenable to large-batch offline training while leveraging in-distribution regularization of latent states.

All code for CliqueFlowmer is open-sourced to facilitate replication and further research in materials optimization (https://github.com/znowu/CliqueFlowmer). This facilitates its adoption and adaptation for specialized materials optimization objectives, supporting ongoing interdisciplinary developments in offline CMD (Kuba et al., 6 Mar 2026).