CROP: Cross-view Prefixes in Molecular LLMs

• CROP is a multimodal molecular LLM architecture that integrates SMILES, graph, and image views via fixed-length cross-view prefixes.
• It employs a SMILES-guided resampler and a structural embedding gate to efficiently fuse topological and spatial molecular information.
• Empirical evaluations demonstrate improved performance in captioning, IUPAC naming, and property prediction while reducing computational cost.

Cross-view Prefixes, abbreviated CROP, is a multimodal molecular LLM architecture that incorporates sequence, graph, and image views of a molecule into a decoder-only LLM through a compact prefixing mechanism. Introduced in "CROP: Integrating Topological and Spatial Structures via Cross-View Prefixes for Molecular LLMs" (Tang et al., 9 Aug 2025), the method addresses a central limitation of sequence-only molecular LLMs: SMILES strings fit standard language modeling pipelines, but do not adequately encode atom-level topology, global substructure arrangement, or spatial configuration. CROP compresses topological and spatial structure into fixed-length cross-view prefixes guided by the LLM’s own SMILES representations, then prepends those prefixes to upper LLM layers rather than concatenating long structural token sequences.

1. Problem formulation and molecular views

CROP is motivated by the claim that molecule understanding should not rely on SMILES alone, nor even on SMILES plus a graph alone. The framework uses three molecular views: the sequence view given by the SMILES string, the topological view given by the molecular graph, and the spatial view given by a molecular image (Tang et al., 9 Aug 2025). In this formulation, the graph view captures atom connectivity and local neighborhood structure, whereas the image view captures 2D spatial arrangement and overall shape cues. The paper explicitly argues that graph and image are complementary: graph representations are strong at connectivity and topology, while images encode spatial configuration and global shape.

The practical challenge is not merely multimodal fusion, but multimodal fusion under a strict LLM context budget. A naive concatenation of graph embeddings and image embeddings with SMILES tokens is inefficient because graph and image sequences are long, redundant, and increasingly costly as additional views are added. The paper gives concrete examples of redundancy, including blank or background regions in image features and overlapping atom- or bond-type information across views (Tang et al., 9 Aug 2025). CROP therefore frames structure injection as a compression problem: multiple structural views should be resampled into a small, fixed-length representation that can be inserted into the LLM without exhausting context length.

Formally, the input molecule is represented as a SMILES string $\boldsymbol{S}=(s_1,s_2,\dots,s_l)$, a molecular graph $\boldsymbol{G}=(V,E)$ with $V=\{v_1,\dots,v_n\}$, and an image $\boldsymbol{I}$. Given $(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$, the model supports both generative tasks that produce text $y$ and classification tasks that produce a probability distribution $p$ (Tang et al., 9 Aug 2025).

2. Architectural organization of CROP

CROP consists of four main components: a Galactica backbone split into lower and upper segments, a graph encoder, an image encoder, the SMILES Guided Resampler (SGR), and the Structural Embedding Gate (SEG) (Tang et al., 9 Aug 2025). The lower LLM segment first processes the SMILES input and produces internal hidden states used as SMILES guidance. In parallel, the graph encoder maps the molecular graph to node embeddings and the image encoder maps the molecular image to patchwise image embeddings.

The view encoders are defined as

$$Z_G = GraphEncoder(\boldsymbol{G}), \qquad Z_I = ImageEncoder(\boldsymbol{I}),$$

with $Z_G \in \mathbb{R}^{n \times d}$ and $Z_I \in \mathbb{R}^{p \times d}$, where $\boldsymbol{G}=(V,E)$0 after flattening the image feature map (Tang et al., 9 Aug 2025). The graph encoder is a 5-layer GIN initialized from moleculeSTM, and the image encoder is a pretrained ResNet18 from ImageMol.

A distinctive design choice is that the LLM backbone is partitioned into lower and upper segments. At every layer in the lower segment, CROP prepends $\boldsymbol{G}=(V,E)$1 learnable vectors that become the SMILES guidance

$\boldsymbol{G}=(V,E)$2

where $\boldsymbol{G}=(V,E)$3 is the number of lower layers, $\boldsymbol{G}=(V,E)$4 is the guidance length, and $\boldsymbol{G}=(V,E)$5 is the hidden dimension (Tang et al., 9 Aug 2025). The paper argues that these guidance vectors should not be treated as generic latent queries. Instead, they are produced by the chemically pretrained LLM itself, so they inherit chemical priors from Galactica’s SMILES modeling.

Because a standard causal decoder would prevent prepended vectors from seeing future SMILES tokens, CROP modifies the lower-segment attention mask so that the prepended guidance tokens can attend to the entire SMILES sequence, while the ordinary SMILES tokens remain causal (Tang et al., 9 Aug 2025). This mask modification is a necessary condition for the guidance vectors to summarize the full molecular sequence.

3. SMILES Guided Resampler

The SMILES Guided Resampler is the central compression mechanism. Its role is to transform long graph and image sequences into a compact fixed-length structural representation using SMILES-conditioned cross-attention. The inputs are the graph embeddings $\boldsymbol{G}=(V,E)$6, the image embeddings $\boldsymbol{G}=(V,E)$7, and the SMILES guidance $\boldsymbol{G}=(V,E)$8. The paper defines the resampler as

$\boldsymbol{G}=(V,E)$9

and

$V=\{v_1,\dots,v_n\}$0

Thus, the queries are not free learned tokens in the main model, but SMILES-conditioned latent vectors produced by the lower LLM (Tang et al., 9 Aug 2025).

This design makes the output length independent of the graph size or image resolution. No matter how many graph nodes or image patches exist, the resampler always returns exactly $V=\{v_1,\dots,v_n\}$1 vectors per lower-layer group. The paper characterizes this as an efficiency advantage because additional structural views can be added by concatenating them into the key-value sequence while keeping the output prefix length fixed (Tang et al., 9 Aug 2025).

The paper also attributes an effectiveness advantage to the query construction. Because Galactica is chemically pretrained, the hidden states used as guidance already encode domain knowledge, so resampling is chemically informed rather than generic. This is not a minor implementation detail: in ablations, replacing SMILES guidance with randomly initialized learnable vectors degrades performance across captioning, IUPAC prediction, and property prediction (Tang et al., 9 Aug 2025).

In appendix-reported settings, the number of SGR layers is 4, and the main experiments use a guidance or prefix length of $V=\{v_1,\dots,v_n\}$2. The prefix-length study reports that performance improves substantially as $V=\{v_1,\dots,v_n\}$3 increases from 2 to 10, saturates around $V=\{v_1,\dots,v_n\}$4, and may slightly decline for larger values due to overfitting (Tang et al., 9 Aug 2025).

4. Structural Embedding Gate and prefix injection

The output of SGR is

$V=\{v_1,\dots,v_n\}$5

but the upper LLM segment may contain $V=\{v_1,\dots,v_n\}$6 layers, and generally $V=\{v_1,\dots,v_n\}$7. The Structural Embedding Gate converts the $V=\{v_1,\dots,v_n\}$8 groups of structural embeddings into $V=\{v_1,\dots,v_n\}$9 groups of prefixes to be prepended to the upper LLM layers. The gate is defined by

$\boldsymbol{I}$0

$\boldsymbol{I}$1

$\boldsymbol{I}$2

Here, $\boldsymbol{I}$3 is the prefix for upper layer $\boldsymbol{I}$4 (Tang et al., 9 Aug 2025).

The interpretation given in the paper is that different groups $\boldsymbol{I}$5 may capture different semantic levels. SEG learns a layer-specific mixture over those groups so that each upper LLM layer receives the structural prefix most useful for its depth and function. This differs from a direct passthrough design in which all upper layers would consume the same ungated structural embeddings.

The final cross-view prefixes are prepended to all layers in the upper segment, not only to the input embedding layer (Tang et al., 9 Aug 2025). This is one of the method’s main distinctions from direct-concatenation or shallow fusion approaches. The fusion process is therefore staged: lower LLM layers encode SMILES and generate guidance; SGR jointly compresses graph and image structure; SEG converts the compressed structure into layer-specific prefixes; upper LLM layers process the original SMILES hidden states together with those prefixes.

The best reported LLM partition settings depend on the task. For captioning and property prediction, the best setting is $\boldsymbol{I}$6. For IUPAC prediction, the best setting is $\boldsymbol{I}$7 (Tang et al., 9 Aug 2025). The paper interprets this as a trade-off between the number of lower layers available to form SMILES guidance and the number of upper layers able to consume structural prefixes.

5. Training regime and task settings

All experiments use Galactica-1.3B as the LLM backbone (Tang et al., 9 Aug 2025). Training proceeds in two stages. In Stage 1, the model is pretrained on molecule captioning conditioned on SMILES, graph, and image. During this stage, the trainable components are the SMILES guidance parameters, SGR, and SEG, while the LLM, graph encoder, and image encoder remain frozen. In Stage 2, downstream fine-tuning is performed. For captioning and IUPAC prediction, the graph encoder and image encoder are unfrozen and LoRA is used to fine-tune the LLM. For molecule property prediction, the paper states that LoRA is not utilized (Tang et al., 9 Aug 2025).

The reported optimization configuration is AdamW, weight decay 0.01, cosine schedule, 1000 warmup steps, and peak learning rate $\boldsymbol{I}$8. Pretraining lasts 20 epochs and fine-tuning lasts 100 epochs. For captioning and IUPAC prediction, LoRA uses rank $\boldsymbol{I}$9, $(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$0, and targets the modules q_proj, v_proj, out_proj, fc1, and fc2 (Tang et al., 9 Aug 2025).

Pretraining uses a PubChem324k pretraining subset of about 300k molecule-text pairs. Downstream generative tasks are molecule captioning on PubChem324k and CheBI-20, and IUPAC name prediction on PubChem324k. These tasks are evaluated with BLEU-2, BLEU-4, ROUGE-1, ROUGE-2, ROUGE-L, and METEOR. Classification experiments use MoleculeNet datasets Tox21, ToxCast, Sider, ClinTox, BBBP, and Bace, with ROC-AUC over 5 random seeds under scaffold split (Tang et al., 9 Aug 2025).

6. Empirical performance, ablations, and efficiency

CROP’s strongest reported captioning results are obtained with all three views, $(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$1. On PubChem324k, the best model reports BLEU-2 44.9, BLEU-4 36.7, ROUGE-1 54.8, ROUGE-2 41.1, ROUGE-L 49.5, and METEOR 50.8. On CheBI-20, the corresponding scores are 64.6, 56.2, 69.8, 55.9, 63.9, and 66.7 (Tang et al., 9 Aug 2025). For IUPAC name prediction on PubChem324k, the best CROP variant with all three modalities reports BLEU-2 81.5, BLEU-4 74.3, ROUGE-1 78.5, ROUGE-2 58.6, ROUGE-L 72.9, and METEOR 78.8. The paper explicitly highlights a +6.5 BLEU-2 gain over MolCA on this task (Tang et al., 9 Aug 2025).

For molecule property prediction, the average ROC-AUC over the six MoleculeNet datasets is 76.3 for CROP with $(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$2. The best per-dataset scores with all three views are Tox21 77.5, ToxCast 61.4, Sider 67.3, ClinTox 94.6, BBBP 72.6, and Bace 84.2 (Tang et al., 9 Aug 2025). The paper notes that the ToxCast result is not state of the art relative to Mole-BERT or GIT-Mol on that specific dataset.

The modality ablations are central to the paper’s argument that graph and image are complementary rather than redundant. On PubChem324k captioning, the reported BLEU-2 scores are 36.6 for $(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$3 only, 43.4 for $(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$4, 43.1 for $(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$5, and 44.9 for $(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$6. On CheBI-20, the same progression is 58.4, 63.8, 62.8, and 64.6. On IUPAC prediction, the scores are 74.5, 80.8, 80.6, and 81.5. For property prediction, the reported mean ROC-AUC values are 72.8, 75.1, 74.9, and 76.3 (Tang et al., 9 Aug 2025). This suggests that the image view is not reducible to the graph view, and that the joint prefix formulation uses both.

The ablations on SGR and SEG isolate the role of the two main architectural additions. Replacing SMILES guidance with random learnable vectors yields the w/o sgr variant, whose all-view results drop from 44.9 to 42.4 on PubChem324k captioning, from 81.5 to 78.5 on IUPAC prediction, and from 77.5 to 75.4 on Tox21, from 94.6 to 92.1 on ClinTox, and from 84.2 to 82.7 on Bace (Tang et al., 9 Aug 2025). Removing SEG also degrades performance, though more mildly: the corresponding scores become 44.2 for captioning, 80.7 for IUPAC, 76.2 for Tox21, 94.0 for ClinTox, and 83.4 for Bace (Tang et al., 9 Aug 2025).

The efficiency analysis directly motivates the term cross-view prefixes. The paper compares three architectures on PubChem324k captioning: arch1, direct concatenation of graph and image embeddings; arch2, independent resampling of graph and image embeddings; and arch3, CROP’s joint cross-view resampling into fixed-length prefixes. The reported results are: Galactica BLEU-2 36.6 with train time 5.37 h; CROP$(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$7 BLEU-2 42.8, average prefix count 288, average FLOPs 353.52 B, train time 7.14 h; CROP$(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$8 BLEU-2 41.5, average prefix count 20, average FLOPs 24.20 B, train time 5.86 h; and CROP$(\boldsymbol{S},\boldsymbol{G},\boldsymbol{I})$9 BLEU-2 44.9, average prefix count 10, average FLOPs 14.05 B, train time 5.53 h (Tang et al., 9 Aug 2025). Compared with direct concatenation, arch3 reduces prefix length by 96.5%, additional FLOPs by 96.0%, and training time by 22.5%, while also being the best-performing variant.

The paper’s qualitative analyses further suggest that the image view helps particularly on more complex molecules, and that the graph-plus-image version better describes substituent positions, substituent types, connectivity of branched structures, and spatial interconnectivity in complex molecules (Tang et al., 9 Aug 2025).

7. Interpretation, scope, and acronym disambiguation

Within molecular LLM research, CROP’s novelty is not simply the use of graph and image alongside SMILES. The paper identifies four more specific contributions: joint resampling of multiple structural views into fixed-length prefixes, SMILES-guided resampling using LLM hidden states, layer-wise gated prefix injection into the upper LLM, and an architecture explicitly designed for context-length efficiency and extensibility (Tang et al., 9 Aug 2025). A plausible implication is that CROP treats structural fusion as a bottlenecked retrieval problem: the model first asks, through SMILES-conditioned latent queries, which graph and image information is most relevant, then presents only that compact summary to the upper LLM layers.

The reported limitations are also specific. The study only uses 2D molecular images, not true 3D geometry. The graph and image views are derived from SMILES rather than experimentally observed. Some benchmarks, especially ToxCast, do not show state-of-the-art performance against all baselines. The paper also notes limited hyperparameter exploration beyond the reported studies of prefix length and LLM partition (Tang et al., 9 Aug 2025). Future directions explicitly mentioned in the appendix include additional structural modalities such as 3D molecular graph and 3D molecular grid.

Because the acronym CROP is overloaded across recent literature, Cross-view Prefixes should not be conflated with unrelated methods sharing the same abbreviation. In the provided literature, CROP also denotes Contextual Region-Oriented Visual Token Pruning for VLM-based VQA (Guo et al., 27 May 2025), Cost-Regularized Optimization of Prompts for token-efficient reasoning in LLMs (Shah et al., 8 Apr 2026), Conformal Reasoning Output Prefixes for certified clean-prefix retention in reasoning traces (Cheung et al., 28 May 2026), and Cross-lingual Entity Projection for zero-shot cross-lingual NER (Yang et al., 2022). In the molecular LLM setting, however, Cross-view Prefixes refers specifically to fixed-length structural prefixes jointly resampled from graph and image views and injected into upper LLM layers to augment SMILES-based reasoning (Tang et al., 9 Aug 2025).