ProtAlign: Multi-Modal Protein Alignment

Updated 5 July 2026

ProtAlign is a family of alignment-centric frameworks in protein ML that align latent embeddings, property preferences, or sequence-structure representations.
It leverages techniques from large language models, geometric deep models, and contrastive learning to enhance protein representation and design.
Applications include inverse folding with multi-objective alignment and cross-modal retrieval, providing diverse strategies for analyzing protein structure and function.

ProtAlign is an overloaded name in contemporary protein machine learning. It refers to at least three distinct alignment-oriented frameworks: a latent representation alignment method between LLMs and geometric deep models for proteins (Shu et al., 2024), a multi-objective preference alignment framework for inverse folding (Hou et al., 6 Mar 2026), and a contrastive framework for aligning protein sequences with three-dimensional structures in a shared embedding space (Ranganath et al., 6 Mar 2026). Across these usages, the common motif is alignment, but the aligned objects, optimization targets, and downstream objectives differ substantially.

1. Terminological scope and disambiguation

A recurrent source of ambiguity is that "ProtAlign" does not denote a single canonical algorithm. In the literature summarized here, it names three separate frameworks with different technical aims.

arXiv id	Title	Alignment target
(Shu et al., 2024)	"Aligning LLMs and Geometric Deep Models for Protein Representation"	LLM text embeddings and GDM structure embeddings
(Hou et al., 6 Mar 2026)	"Property-driven Protein Inverse Folding With Multi-Objective Preference Alignment"	Pretrained inverse-folding policy and multi-property preferences
(Ranganath et al., 6 Mar 2026)	"ProtAlign: Contrastive learning paradigm for Sequence and structure alignment"	Protein sequence embeddings and structure embeddings

The 2024 framework is a systematic method for aligning latent embeddings produced by LLMs and GDMs on protein data, with an explicit focus on cross-modal protein representation (Shu et al., 2024). The 2026 inverse-folding framework uses the same name for a plug-in fine-tuning method that extends a pretrained inverse-folding model via multi-objective preference alignment (Hou et al., 6 Mar 2026). A second 2026 paper uses ProtAlign for a sequence-structure contrastive learning paradigm built around ESM2 and Protein-MPNN encoders (Ranganath et al., 6 Mar 2026).

This suggests that "ProtAlign" is best treated as a family name for alignment-centric protein-learning methods rather than as a uniquely identified architecture. The shared terminology should therefore be interpreted in context: latent-space alignment, preference alignment, and sequence-structure contrastive alignment are related only at the level of high-level learning principle.

2. Latent alignment between LLMs and geometric deep models

In "Aligning LLMs and Geometric Deep Models for Protein Representation" (Shu et al., 2024), ProtAlign is defined as a systematic framework for aligning latent embeddings produced by LLMs and GDMs on protein data. The method takes a raw embedding $\mathbf{t}_i \in \mathbb{R}^{d_t}$ from an LLM, derived from a processed FASTA description, and a raw embedding $\mathbf{g}_i \in \mathbb{R}^{d_g}$ from a GDM, derived from the protein's 3D structure. Two small projection networks, one per modality, map these vectors into a shared space of dimension $d_{\rm out}$ , which in practice is set equal to the LLM embedding size.

The projection mechanism is a two-layer projection head per modality. Layer 1 is a linear map $d_{\rm in}\to h$ followed by ReLU and BatchNorm; Layer 2 is a linear map $h\to d_{\rm out}$ followed by $\ell_2$ normalization. The hidden dimension is chosen as $h\approx(d_{\rm in}+d_{\rm out})/2$ . After projection, ProtAlign computes cosine similarity between the normalized text and structure embeddings, rescales it to $[0,1]$ by $\bigl(\text{sim}+1\bigr)/2$ , and optimizes an InfoNCE loss with temperature $\tau$ . The objective is to pull same-protein pairs together while pushing apart different proteins in the same batch.

At test time, the framework reports three alignment measures: the positive score $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 0, the negative score $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 1, and the overall similarity score $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 2. Large $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 3 indicates that matched protein pairs receive high similarity while mismatched pairs remain near zero. The same study also uses Pearson correlations between alignment scores of different LLM-GDM pairs to quantify concordance.

The framework evaluates three LLMs—Gemma2-2B with $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 4, LLaMa3.1-8B with $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 5, and LLaMa3.1-70B with $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 6—together with four protein-specialized GDMs. GearNet has $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 7 and is described as a relational graph network with both sequential, radius and $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 8-NN edges and an edge-message-passing module that explicitly encodes 3D geometric angles. ScanNet has $\mathbf{g}_i \in \mathbb{R}^{d_g}$ 9 and uses local coordinate frames, spatio-chemical Gaussian filters on atomic neighborhoods, and amino-acid level pooling. GVP has $d_{\rm out}$ 0 and processes scalar and vector residue features with rotation-equivariant layers, then average-pools across residues. GAT has $d_{\rm out}$ 1 and is a pure graph attention network over atoms with 8 heads and 8 features per head, average-pooled to fixed size.

The central conceptual claim of this ProtAlign is that cross-modal protein alignment should not be treated heuristically. Instead, alignment quality depends on specific architectural and data factors on both the language and geometric sides of the model pair.

3. Empirical protocol, findings, and recommendations in the LLM-GDM framework

The experimental protocol in (Shu et al., 2024) uses 20,000 protein structures randomly sampled from RCSB PDB. Each protein is represented by a human-readable FASTA-derived description containing no raw sequence, only length, chains, organism, and name, together with a structural graph from the PDB. The data split is 80% train, 10% validation, and 10% test. GDMs and LLMs are loaded from official repositories; the alignment module is trained for 40 epochs with batch size $d_{\rm out}$ 2, learning rate $d_{\rm out}$ 3, Adam optimizer, temperature $d_{\rm out}$ 4, random seed 42, and best-validation checkpointing.

Several empirical findings are explicit. First, GDMs that incorporate both graph and 3D structural information align better with LLMs than weaker structural encoders. Second, larger LLMs demonstrate improved alignment capabilities. Third, protein rarity significantly impacts alignment performance: proteins categorized as "popular" align with significantly higher $d_{\rm out}$ 5 than "rare" proteins studied only once. The paper attributes this to an uneven training distribution in PDB and the challenge of under-documented proteins.

A focused ablation on GDM embedding dimension retrains GearNet on a multi-task enzyme classification set with hidden sizes $d_{\rm out}$ 6 and reports a monotonic rise in alignment, for example with LLaMa3.1-70B, as $d_{\rm out}$ 7 increases (Shu et al., 2024). The stated interpretation is that larger structural embeddings capture richer shape cues that map more cleanly into LLM space.

The same work reports that fine-tuning each LLM via LoRA on the FASTA-description QA task for 10 epochs with learning rate $d_{\rm out}$ 8, rank $d_{\rm out}$ 9, $d_{\rm in}\to h$ 0, and dropout $d_{\rm in}\to h$ 1 raises alignment by 5–15% uniformly. The paper presents this as evidence that domain-specific language adaptation narrows the modality gap. It also recommends using LLMs of at least 8B parameters with $d_{\rm in}\to h$ 2, GDM embeddings $d_{\rm in}\to h$ 3 and preferably greater than $d_{\rm in}\to h$ 4, and two-layer projection heads rather than deeper heads, since gains plateau beyond two layers.

The limitations are also explicit. Negative pairs can be structurally homologous, so enforcing zero similarity may hurt alignment on true novel negatives. The authors suggest smarter negative sampling or hierarchical contrastive losses that respect protein families. They also note dataset imbalance in PDB, possible augmentation or up-weighting of rare proteins, and the need to validate alignment utility through downstream multimodal fusion tasks such as property prediction and text-structure generation. These caveats are important because they distinguish alignment score optimization from downstream biological usefulness.

4. Multi-objective preference alignment for inverse folding

In "Property-driven Protein Inverse Folding With Multi-Objective Preference Alignment" (Hou et al., 6 Mar 2026), ProtAlign denotes a different framework. Here the task is protein inverse folding: generating sequences $d_{\rm in}\to h$ 5 that fold to a target backbone structure $d_{\rm in}\to h$ 6. The paper defines two goals that are often in tension. Designability is the ability of the designed sequence to recover or refold to the target backbone, measured by RMSD, TM-score, pLDDT, amino-acid recovery, and related metrics. Developability comprises biologically and industrially important properties such as solubility, thermostability, and expression level.

This ProtAlign is a plug-in fine-tuning framework that extends any pretrained inverse-folding model, with ProteinMPNN used as the reference policy $d_{\rm in}\to h$ 7. Each iteration has four stages. In rollout, the current policy samples backbones and generates diverse sequences at elevated temperature. In annotation, each sequence is scored with $d_{\rm in}\to h$ 8 in silico property predictors covering both designability and developability. In pair construction, the method forms pairwise preferences $d_{\rm in}\to h$ 9 for each property if the score gap exceeds a property-specific threshold $h\to d_{\rm out}$ 0. In update, the model is fine-tuned using a multi-objective Direct Preference Optimization loss that aligns to all $h\to d_{\rm out}$ 1 preference datasets while regularizing toward $h\to d_{\rm out}$ 2.

The framework introduces a flexible preference margin,

$h\to d_{\rm out}$ 3

to mitigate conflicts among competing objectives (Hou et al., 6 Mar 2026). The total loss is the sum of per-property multi-objective DPO losses, weighted by objective weights $h\to d_{\rm out}$ 4 and scaled by an alignment strength $h\to d_{\rm out}$ 5. Under the Bradley-Terry assumption, the paper states that the implicit reward is proportional to the log-ratio $h\to d_{\rm out}$ 6, which justifies using policy log-ratios in place of explicit reward models.

Preference data are constructed from predictors that include TM-score from ESMFold versus target, pTM from AlphaFold initial guess, pLDDT, solubility from Protein-Sol, thermostability from TemBERTure, and evolutionary perplexity from ESM-2 pseudo-likelihood. For each backbone and rollout set, sequences are sorted by a property score, and the top half is paired against the bottom half when the gap exceeds $h\to d_{\rm out}$ 7, thereby filtering ambiguous comparisons.

Training uses a semi-online procedure with $h\to d_{\rm out}$ 8 rounds. In each round the method samples $h\to d_{\rm out}$ 9 backbones, generates $\ell_2$ 0 sequences per backbone at rollout temperature $\ell_2$ 1, precomputes preference datasets $\ell_2$ 2, and updates $\ell_2$ 3 for 600 steps with Adam using learning rate $\ell_2$ 4, $\ell_2$ 5, $\ell_2$ 6, $\ell_2$ 7, batch size 64, and 8 GPUs. The regularization strength is $\ell_2$ 8, with $\ell_2$ 9 and other objectives weighted at $h\approx(d_{\rm in}+d_{\rm out})/2$ 0.

On CATH 4.3 crystal structures, the baseline ProteinMPNN achieves RMSD 4.30 Å, TM 0.740, pLDDT 79.1, EP 6.70, Sol 0.719, and Thermo 0.769. The MoMPNN variant trained with Sol+TM+EP yields RMSD 4.38 Å, TM 0.739, pLDDT 79.5, EP 6.18, Sol 0.852, and Thermo 0.790. The Thermo+TM variant yields RMSD 4.30 Å, TM 0.739, pLDDT 78.4, EP 6.24, Sol 0.704, and Thermo 0.947. The paper further states that the best solubility gain is $h\approx(d_{\rm in}+d_{\rm out})/2$ 1 versus $h\approx(d_{\rm in}+d_{\rm out})/2$ 2 and the best thermostability gain is $h\approx(d_{\rm in}+d_{\rm out})/2$ 3 versus $h\approx(d_{\rm in}+d_{\rm out})/2$ 4, while TM-score and RMSD remain on-par.

On de novo backbones, MoMPNN[Sol+IG+EP] yields RMSD 6.17 Å versus 6.86 Å, TM 0.751 versus 0.718, EP 7.34 versus 8.32, and Sol 0.843 versus 0.731 for ProteinMPNN. In binder design, the same model slightly raises sequence-success rate and backbone-success rate over ProteinMPNN while halving EP and doubling relative Sol improvement. Comparative analysis in the paper states that this ProtAlign consistently outperforms post hoc mutation methods such as ProteinDPO, inference-time biasing methods such as Guidance[Sol] and Guidance[Thermo], and retraining on curated subsets such as SolubleMPNN and HyperMPNN (Hou et al., 6 Mar 2026).

The limitations are narrower but consequential. All developability signals rely on in silico predictors, so wet-lab validation remains outstanding. The current focus is on monomeric properties; binder tests are presented as encouraging, but interface complexity and affinity metrics are not yet integrated. The authors identify dynamic margin scheduling, Pareto-front tracing, and extension to larger protein LLMs or diffusion-based inverse fold approaches as future directions.

"ProtAlign: Contrastive learning paradigm for Sequence and structure alignment" (Ranganath et al., 6 Mar 2026) introduces yet another ProtAlign, this time for explicit alignment of protein sequences and their three-dimensional structures in a single shared embedding space. The framework is built on two off-the-shelf encoders: ESM2 for sequence and Protein-MPNN for structure. Given a protein of length $h\approx(d_{\rm in}+d_{\rm out})/2$ 5, ESM2 produces a token-wise embedding $h\approx(d_{\rm in}+d_{\rm out})/2$ 6. Given experimentally resolved or predicted 3D coordinates with $h\approx(d_{\rm in}+d_{\rm out})/2$ 7 structure tokens, Protein-MPNN produces $h\approx(d_{\rm in}+d_{\rm out})/2$ 8.

To project both modalities into a common space, the method uses a small projection head with a single trainable query token for each modality, $h\approx(d_{\rm in}+d_{\rm out})/2$ 9, where in practice $[0,1]$ 0. A one-layer, multi-head self-attention block uses the query token as a pooled representation and the encoder outputs as keys and values. The resulting sequence embedding $[0,1]$ 1 and structure embedding $[0,1]$ 2 are $[0,1]$ 3-normalized before dot products are computed. This design differs from the two-layer MLP heads of (Shu et al., 2024) by using attention-based pooling rather than direct projection of a precomputed global embedding.

The contrastive objective has two variants. The CLIP-style softmax loss symmetrically maximizes the similarity of matched sequence-structure pairs relative to all other pairs in the minibatch. The SigLIP-style loss instead uses pairwise sigmoid terms with labels $[0,1]$ 4 for matched pairs and $[0,1]$ 5 otherwise, together with a learnable bias $[0,1]$ 6 to offset the number of negatives. The training data come from PDBBind 2020: 14,127 protein-ligand complexes in the general set, 5,316 in the refined set, and 285 in the core CASF-2016 set. Ligand SMILES are discarded, and sequences are deduplicated by exact string match, producing approximately 10,071 training pairs, 3,387 validation pairs, and 215 test pairs.

Training uses batch size $[0,1]$ 7, embedding dimension $[0,1]$ 8, Adam with learning rate $[0,1]$ 9, four attention heads, and a temperature of $\bigl(\text{sim}+1\bigr)/2$ 0 for CLIP, with the same value reported as best for SigLIP; the SigLIP bias is set to $\bigl(\text{sim}+1\bigr)/2$ 1. Models converge in 20–30 epochs. The main results freeze the ESM2 and Protein-MPNN encoders and train only the projection heads, though frozen and fine-tuned variants are both discussed (Ranganath et al., 6 Mar 2026).

Evaluation is framed as sequence-to-structure retrieval. For each test sequence embedding, the method ranks all 215 candidate structures by descending cosine similarity and reports Recall@1 and Recall@5. Table 1 reports Recall@1 of 40.0% and Recall@5 of 97.6 for SigLIP, and Recall@1 of 42.7% and Recall@5 of 99.1 for CLIP. An ablation on temperature reports that $\bigl(\text{sim}+1\bigr)/2$ 2 provides the best trade-off, while smaller temperatures produce overly sharp distributions and unstable training.

The paper also emphasizes interpretability. t-SNE visualizations are said to show that before contrastive training, sequence and structure points intermix randomly, whereas afterward they form tight clusters around matched pairs and broader clusters corresponding to protein families. A highlighted cluster containing PDB IDs 3ao4, 3zso, 3zsx, and 4cig is used to illustrate that the model embeds related proteins in the same neighborhood despite insertions, deletions, and point mutations. A pairwise cosine-similarity heatmap on the test set shows clear diagonal dominance after training. For downstream tasks, the authors state that function annotation and stability estimation can reuse the unified embeddings by attaching simple classification or regression heads, but the manuscript provides explicit quantitative detail only for retrieval, with downstream numbers described as ongoing work.

6. Shared motifs, divergences, and open questions

Across the three usages, ProtAlign consistently denotes an alignment strategy, but the level of alignment differs. In (Shu et al., 2024), the aligned objects are latent embeddings from LLMs and GDMs. In (Ranganath et al., 6 Mar 2026), the aligned objects are sequence and structure embeddings from ESM2 and Protein-MPNN. In (Hou et al., 6 Mar 2026), the alignment target is not a pair of embeddings at all but a pretrained inverse-folding policy aligned to multiple property preferences. These are therefore not interchangeable instantiations of one method.

Two of the three frameworks are explicitly contrastive. The LLM-GDM ProtAlign uses a cosine-similarity InfoNCE objective on paired proteins (Shu et al., 2024), while the sequence-structure ProtAlign explores CLIP and SigLIP losses over matched and mismatched pairs (Ranganath et al., 6 Mar 2026). Both depend on the quality of negatives, and both identify the shared-space geometry as central to downstream utility. The inverse-folding ProtAlign instead replaces pairwise embedding contrast with multi-objective DPO over winner-loser sequence pairs, while still retaining a notion of alignment through regularization toward a reference policy and through adaptive handling of objective conflicts (Hou et al., 6 Mar 2026).

The limitations also differ by formulation. In the LLM-GDM setting, negative pairs may be homologous and rare proteins are underrepresented. In the inverse-folding setting, developability objectives are mediated entirely by in silico predictors and current tests emphasize monomeric properties. In the sequence-structure retrieval setting, the paper does not report detailed downstream task numbers beyond retrieval and does not use additional sequence or structure augmentation beyond what is implicit in encoder pretraining. A plausible implication is that "alignment" is not a sufficient description of biological validity: the quality of sampling, supervision, and evaluation protocol remains decisive.

Taken together, these works position ProtAlign as a label for three complementary research programs in protein machine learning: multimodal latent-space matching, multi-objective policy alignment for design, and contrastive sequence-structure representation learning. Their overlap lies in the attempt to make heterogeneous protein information commensurate within a trainable objective. Their divergence lies in what is being aligned, why it is being aligned, and how success is measured.