CrystalICL: Few-Shot ICL for Crystal Design

• CrystalICL is a crystal-generation model that employs few-shot in-context learning through symmetry-aware serialization and demonstration-based prompts.
• It uses space-group based tokenization and hybrid instruction tuning to integrate condition-based and structure-aware learning, enhancing property prediction.
• Empirical results show that CrystalICL significantly outperforms previous models like CrystalLLM in formula accuracy, space group, formation energy, and band gap predictions.

Searching arXiv for CrystalICL and closely related crystal-generation representation papers to ground the article in current literature. CrystalICL is a crystal-generation model that explicitly targets few-shot in-context learning (ICL) for periodic materials design. It formulates crystal generation as either conditional generation, $p_\theta(\cdot \mid s)$, from target conditions such as chemical formula, space group, formation energy, or band gap, or unconditional generation, $p_\theta(\cdot)$, over crystal distributions. Its central claim is that prior LLM-based crystal generators were effectively restricted to zero-shot use, whereas practical materials design more closely resembles few-shot adaptation from relevant known structures. To address that gap, CrystalICL combines a space-group based crystal serialization, condition- and structure-aware prompt construction, and multi-task instruction tuning on top of Llama2-7b-chat (Wang et al., 27 Aug 2025).

1. Problem formulation and conceptual scope

CrystalICL represents a crystal unit cell with $N$ atoms as

$$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$

where $\mathbf{A} \in \mathbb{R}^{N \times K}$ is one-hot atom-type encoding, $\mathbf{X} \in \mathbb{R}^{N \times 3}$ are Cartesian coordinates, and $\mathbf{L} \in \mathbb{R}^{3 \times 3}$ is the lattice matrix. Because periodic crystals are more naturally handled in fractional coordinates, the model adopts

$\mathcal{M}=(\mathbf{A},\mathbf{F},\mathbf{L}),$

with $\mathbf{F}\in [0,1)^{N\times3}$. The infinite periodic crystal is written as

$$\{(\boldsymbol{a}'_i, \boldsymbol{x}'_i) \mid \boldsymbol{a}'_i = \boldsymbol{a}_i, \boldsymbol{x}'_i = \boldsymbol{x}_i + \boldsymbol{k}\mathbf{L}, \forall \boldsymbol{k} \in \mathbb{Z}^{1\times3}\}.$$

This framing keeps the problem grounded in explicit crystallographic structure rather than composition-only generation (Wang et al., 27 Aug 2025).

The model is designed for two regimes. In the conditional regime, given training pairs $p_\theta(\cdot)$0, it learns $p_\theta(\cdot)$1. In the unconditional regime, given $p_\theta(\cdot)$2, it learns $p_\theta(\cdot)$3. The intended novelty is not merely LLM-based generation, but train-time and test-time use of demonstrations so that generation quality improves when the prompt includes relevant examples. The paper motivates this by contrasting CrystalICL with CrystalLLM and by noting that, on P5, GPT-3.5 Turbo improves from 0-shot to 3-shot prompting while CrystalLLM degrades, suggesting that prior crystal-specific LLMs do not reliably realize few-shot ICL behavior (Wang et al., 27 Aug 2025).

Architecturally, CrystalICL is not a domain-specific backbone pretrained from scratch. It is an instruction-tuned generator built on Llama2-7b-chat, fine-tuned with LoRA, and specialized through a crystal-specific serialization and retrieval protocol. This places it in a distinct methodological niche relative to encoder-style crystal foundation models such as CLOUD, which rely on masked-language pretraining and downstream fine-tuning rather than prompt-time adaptation (Xu et al., 19 Jun 2025).

2. Space-group based crystal tokenization

A core contribution is space-group based crystal tokenization (SGS), introduced to reduce the difficulty of modeling crystallographic symmetry in an autoregressive text model. The paper argues that CIF is overly formatted and token-heavy, while XYZ-style coordinate serialization forces the model to infer symmetry entirely from raw atomic coordinates. SGS instead externalizes symmetry through the space group and compresses atomic content through Wyckoff-position representatives (Wang et al., 27 Aug 2025).

The key observation is that, for a given space group, atoms of the same element occupying the same Wyckoff position can be represented by a single representative atom. Under SGS, the model therefore generates three components: the space group symbol, the lattice parameters, and the element symbols with fractional coordinates for atoms at each Wyckoff position. In practical serialized form, the output contains the space-group symbol, lattice lengths $p_\theta(\cdot)$4, lattice angles $p_\theta(\cdot)$5, and then the representative atomic entries. This reduces both sequence length and the burden of enforcing full symmetry over all atoms (Wang et al., 27 Aug 2025).

The representation is conceptually

$p_\theta(\cdot)$6

where $p_\theta(\cdot)$7 is the number of occupied representative Wyckoff positions rather than the full unit-cell atom count. The paper does not provide a separate formal equation for the SGS string, nor an explicit symbolic mapping from representative coordinates to full Wyckoff orbits, but the intended effect is clear: symmetry constraints are shifted from latent coordinate regularities into an explicit textual scaffold (Wang et al., 27 Aug 2025).

This design aligns CrystalICL with a broader trend toward symmetry-aware crystal representations. WyCryst encodes crystals through space groups and Wyckoff occupancy in a VAE framework (Zhu et al., 2023), and CLOUD uses a coordinate-free sequence representation that begins with symmetry generators and then occupied Wyckoff positions (Xu et al., 19 Jun 2025). CrystalICL differs by using the symmetry-aware representation as a promptable autoregressive language interface rather than as an encoder input or latent variable.

3. In-context learning mechanism and instruction tuning

CrystalICL operationalizes few-shot learning through condition-structure aware hybrid instruction tuning. Its training set is the union

$p_\theta(\cdot)$8

where $p_\theta(\cdot)$9 contains zero-shot instructions and $N$0 contains few-shot instructions with demonstrations. The model is trained with the autoregressive objective

$N$1

where $N$2 is the pretrained LLM, $N$3 is the instruction prompt, and $N$4 is the target response (Wang et al., 27 Aug 2025).

The zero-shot prompt template provides a condition description and asks for a crystal string. The few-shot template prepends $N$5 demonstrations, each consisting of a condition description and its corresponding crystal string, and then queries the model for a new crystal conditioned on the target description. The demonstrations are drawn from the training set rather than being arbitrary prompt examples, which makes prompt construction part of the method rather than an afterthought (Wang et al., 27 Aug 2025).

Demonstration selection follows three strategies. Condition-based selection retrieves examples matching the target condition, using anonymized formula patterns for composition, exact match for discrete properties such as space group, and nearest neighbors in absolute value for continuous properties such as band gap. Structure-based selection chooses an anchor crystal and retrieves the top $N$6 nearest crystals by Euclidean distance in CrystalNN fingerprint space. Condition-structure based selection first filters by condition, then retrieves structural neighbors. The method name “condition-structure aware” refers precisely to these retrieval policies (Wang et al., 27 Aug 2025).

CrystalICL adds a second training ingredient, multi-task crystal instruction tuning with property prediction. The auxiliary task masks a property value in a prompt that includes an SGS crystal string and asks the model to generate the missing value. The paper presents this through prompts of the form “The [Property] is [MASK]” followed by the crystal string. The purpose is to make the model explicitly learn structure–property relationships rather than only property-to-structure mappings. The paper does not give a separate weighted multi-task loss decomposition, but the property-prediction instructions are mixed into the same autoregressive instruction-tuning corpus (Wang et al., 27 Aug 2025).

The ablations strongly support this design. Removing the auxiliary task in the noAux variant substantially hurts zero-shot performance on MP20 with SGS: pretty formula drops to 0.4167 versus 0.7390 for full CrystalICL, and space group drops to 0.3740 versus 0.9904. This indicates that the auxiliary property-prediction task is not incidental; it appears to be a major component of the model’s structure–property grounding (Wang et al., 27 Aug 2025).

4. Training setup, prompting protocol, and evaluation

CrystalICL fine-tunes Llama2-7b-chat using LoRA with rank 8, alpha 32, and dropout $N$7. Optimization uses cosine annealing with initial learning rate $N$8. Training batches and epochs are dataset-specific: MP20 uses batch size 1 for 10 epochs, C24 batch size 1 for 10 epochs, MP30 batch size 1 for 3 epochs, and P5 batch size 4 for 10 epochs. All training and inference are carried out on a single Nvidia L40 48G GPU (Wang et al., 27 Aug 2025).

At inference time, decoding uses top-p = 0.9 and temperature = 0.9. For conditional generation, prompts are built from test-set conditions and training-set demonstrations. Inference batch sizes are 6 for cross-domain conditional generation, 8 for conditional generation, and 32 for unconditional generation. In unconditional generation, the model samples 10,000 structures, parses generated strings into CIFs, and rejects invalid parses before resampling (Wang et al., 27 Aug 2025).

The evaluation setup covers four conditional-generation benchmarks: MP20, MP30, P5, and C24. For unconditional generation, results are reported on MP20, P5, and C24. The paper states that standard splits from prior work are used. Cross-domain conditional generation is also evaluated by training on MP20 and testing on P5 and C24, which probes whether the prompt-conditioned behavior transfers beyond the training distribution (Wang et al., 27 Aug 2025).

Conditional generation is measured by task-specific success rate. Formula success requires that the generated atomic composition match the requested composition. Space-group success is evaluated using SpacegroupAnalyzer from pymatgen. Formation-energy success requires that the MEGNet-estimated generated formation energy have the same sign as the input condition. Band-gap success requires

$N$9

where $$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$0 is the target band gap and $$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$1 is the predicted band gap from MEGNet. Unconditional generation is evaluated by compositional validity, structural validity, Recall, Precision, and Wasserstein distances for density, formation energy per atom, and number of distinct element types: $$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$2, $$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$3, and $$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$4 (Wang et al., 27 Aug 2025).

5. Empirical performance and ablation results

The central empirical claim is that CrystalICL improves over CrystalLLM in both 0-shot and 3-shot settings and, unlike CrystalLLM, benefits from few-shot prompting. The strongest reported configuration is usually SGS + few-shot CrystalICL. On MP20 with SGS and 3-shot, the success rates are 0.8868 for Pretty Formula, 0.9908 for Space Group, 0.9392 for Formation Energy, and 0.7453 for Band Gap. The corresponding CrystalLLM(SGS) results are 0.4513, 0.8726, 0.7984, and 0.6373. On MP30 with the same configuration, CrystalICL reaches 0.9641, 0.9956, 0.9789, and 0.7943, versus 0.5008, 0.9006, 0.8030, and 0.6687 for CrystalLLM(SGS) (Wang et al., 27 Aug 2025).

Benchmark	CrystalICL (SGS, 3-shot)	CrystalLLM (SGS)
MP20 Pretty Formula	0.8868	0.4513
MP20 Space Group	0.9908	0.8726
MP20 Formation Energy	0.9392	0.7984
MP20 Band Gap	0.7453	0.6373
MP30 Pretty Formula	0.9641	0.5008
MP30 Space Group	0.9956	0.9006
MP30 Formation Energy	0.9789	0.8030
MP30 Band Gap	0.7943	0.6687

These numbers indicate that the largest relative gains occur on formula-conditioned generation, while symmetry and property conditioning also improve materially. The paper further reports that, with SGS, CrystalICL exhibits almost no performance degradation in cross-domain space-group conditioned generation relative to in-domain evaluation, though the full table of cross-domain values is not reproduced in the provided material (Wang et al., 27 Aug 2025).

For unconditional generation, the main message is more modest but still substantive: CrystalICL improves substantially over CrystalLLM and is competitive with diffusion-based models on several distributional metrics. On MP20 with SGS, CrystalICL reports Valid 0.8555, Recall 0.9949, Precision 0.9926, wdist($$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$5) 0.6039, wdist($$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$6) 0.2568, and wdist($$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$7) 0.1359, improving over CrystalLLM(SGS), which reports 0.8144, 0.9944, 0.9847, 0.8356, 0.3544, and 0.1743. On C24 with SGS, CrystalICL reports Valid 0.9669, Recall 0.9921, Precision 0.8928, wdist($$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$8) 0.0593, and wdist($$\mathcal{M} = (\mathbf{A}, \mathbf{X}, \mathbf{L}),$$9) 1.3061, versus CrystalLLM(SGS) at 0.9574, 0.9916, 0.7965, 0.0707, and 3.0564 (Wang et al., 27 Aug 2025).

The appendix provides additional realism diagnostics for MP20 under SGS 3-shot CrystalICL: Atomic Overlap 0.0290, Symmetry Adherence 0.9666, and Energy-based feasibility 0.9151. These results are important because the model’s headline success-rate metrics depend partly on surrogate predictors such as MEGNet, whereas overlap and symmetry adherence probe structural plausibility more directly (Wang et al., 27 Aug 2025).

The ablation studies clarify which components matter. Under inference-time selection, condition-based prompt construction is the strongest strategy. On MP20 with SGS and 3-shot, the best C configuration reports 0.9214 for Pretty Formula, 0.9948 for Space Group, 0.9685 for Formation Energy, and 0.7687 for Band Gap. The paper also notes that shuffling example order hurts performance, which suggests that demonstration organization is semantically active rather than inert prompt padding (Wang et al., 27 Aug 2025).

The shot-count study is notable because it weakens a simple scaling interpretation of ICL. For condition-based retrieval, 1Shot-C yields 0.9376 / 0.9946 / 0.9731 / 0.8009, 2Shot-C yields 0.9410 / 0.9954 / 0.9683 / 0.7777, and 3Shot-C yields 0.9214 / 0.9948 / 0.9685 / 0.7687 across Pretty Formula, Space Group, Formation Energy, and Band Gap. The paper’s conclusion is that the number of demonstrations has insignificant impact; retrieval quality matters more than raw shot count (Wang et al., 27 Aug 2025).

6. Relation to adjacent crystal-generation paradigms, limitations, and significance

CrystalICL occupies a distinct place among recent crystal ML systems. It differs from sequence encoders such as CLOUD, which pretrain a BERT-style model on symmetry-consistent crystal strings and then rely on supervised fine-tuning rather than prompt-time few-shot adaptation (Xu et al., 19 Jun 2025). It also differs from discrete-latent or VQ-based generators such as VQCrystal, where codebook indices behave like tokenized crystal motifs but are manipulated through latent-space search and relaxation rather than in-context demonstrations (Qiu et al., 2024). Relative to symmetry-aware generators such as WyCryst, which encode space groups and Wyckoff patterns explicitly in a VAE-style framework, CrystalICL replaces latent-variable decoding with instruction-tuned autoregressive generation and prompt-based adaptation (Zhu et al., 2023).

This contrast makes CrystalICL significant less as a universal best generator than as a demonstration that LLM-style ICL can be made materially useful for crystal design. The paper’s evidence for genuine ICL is behavioral rather than mechanistic: improvement from 0-shot to 3-shot, sensitivity to retrieval strategy, degradation when demonstrations are shuffled or randomly selected, and stronger cross-domain space-group conditioning than the baseline (Wang et al., 27 Aug 2025).

The limitations are equally clear. Cross-domain performance still degrades when moving from MP20 to P5 or C24. Some datasets, particularly P5 and C24, contain many unrealistic or unstable materials, which restricts reliable property-based evaluation. Conditional evaluation on P5 and C24 is correspondingly limited to formula and space group. Generation validity is not perfect, since unconditional sampling requires parsing checks and resampling. SGS can also slightly reduce performance on chemical-formula conditioned tasks relative to XYZ in some settings. Finally, much of the physical assessment still depends on surrogate predictors such as MEGNet rather than direct DFT validation, which suggests that CrystalICL is presently strongest as a prompt-conditioned structure generator rather than as a fully validated inverse-design pipeline (Wang et al., 27 Aug 2025).

Taken together, these features define CrystalICL as a symmetry-aware, instruction-tuned LLM system that reorients crystal generation around promptable demonstrations. Its main technical contribution is not only that it generates crystals in text form, but that it makes few-shot context operationally useful through SGS serialization, retrieval-based demonstration construction, and auxiliary structure–property supervision. Within the recent crystal-ML literature, it therefore serves as a bridge between symmetry-aware crystal representation work and LLM-style prompt adaptation (Wang et al., 27 Aug 2025).