PLAME: Protein Language Model Enhanced MSA

• PLAME is a computational tool that uses pretrained protein language models and a transformer-based encoder-decoder to generate high-quality multiple sequence alignments.
• It introduces novel conservation and diversity loss functions with the HiFiAD algorithm to balance evolutionary signal and sequence variability.
• Experimental evaluations demonstrate that PLAME significantly improves metrics like pLDDT and GDT, enhancing protein folding predictions even for orphan proteins.

PLAME refers to two distinct concepts in contemporary research: (1) the anomalous microwave emission detected in the Pleiades reflection nebula known as "PLAME" in astrophysics (Genova-Santos et al., 2011), and (2) a protein informatics framework, PLAME, for generating enhanced multiple sequence alignments using pretrained LLMs (Cao et al., 17 Jun 2025). Each represents a domain-specific innovation. The following entry focuses on the protein informatics context, which is prominent in computational biology and structural bioinformatics.

PLAME (“Pretrained LLM Enhanced MSA”) is a de novo multiple sequence alignment (MSA) generator that integrates evolutionary information from pretrained protein LLMs (PLMs) with a transformer-based encoder-decoder architecture. It is designed to overcome the limitations of traditional MSA-based protein structure prediction, particularly for low-homology and orphan proteins where deep MSAs are inaccessible or sparse. PLAME introduces novel loss functions for balancing residue conservation with sequence diversity, a principled selection mechanism for high-quality generated MSAs, and comprehensive sequence quality metrics for downstream assessment and ablation analyses (Cao et al., 17 Jun 2025).

1. Model Architecture and Input Modalities

PLAME employs an encoder-decoder transformer, inspired by T5, augmented with axial (row and column) attentions to jointly capture inter-sequence and per-position dependencies across the MSA tensor. The architecture leverages embeddings from a large-scale pretrained protein LLM (ESM-2), providing the initial evolutionary latent representation.

• Inputs: A protein query sequence $s$ of length $L$; optionally, an original MSA $M$ for few-shot scenarios; and ESM-2’s residue-level embeddings $$H_{\text{input}} = g_\phi(s) \in \mathbb{R}^{L \times h}$$.
• Encoder: 12-layer T5-style encoder, augmented with Row-Attention to average features along the MSA depth, modeling inter-sequence variation:

$$H_r = \tfrac{1}{D} \sum_{d=1}^D H_{\rm enc}^{(l),d}$$

• Decoder: 12-layer autoregressive decoder, with causal self-attention, cross-attention to encoder states, and Column-Attention applied across positional axes to promote alignment-based conservation.
• Output: Generation of $N'$ virtual MSA sequences, sampled autoregressively over the amino acid vocabulary.

Zero-shot mode uses only PLM-derived embeddings, while few-shot mode integrates external MSAs.

2. Conservation–Diversity Loss Functions

PLAME’s training objective combines two components:

• PSSM-Weighted Cross-Entropy (PCE): Cross-entropy loss weighted by position-specific conservation, amplifying contributions from structurally or functionally conserved columns.

$$\mathcal{L}_{\rm PCE} = -\frac{1}{N} \sum_{j=1}^{N} \sum_{l=1}^{L_j} w_l^{(j)} \log p_\theta(y_l^{(j)} \mid y_{<l}^{(j)})$$

with $$w_l = 1 + \delta \left[ \frac{\text{freq}_l - \min \text{freq}}{\max \text{freq} - \min \text{freq}} \right]$$, $\delta=0.5$.

• Diversity Regularization (DIRE): Maximizes entropy at each alignment position to prevent over-conservatism and mode collapse.

$$\mathcal{L}_{\rm DIRE} = -\frac{1}{N} \sum_{j=1}^{N} \frac{1}{L_j} \sum_{l=1}^{L_j} H_l^{(j)}, \;\; H_l^{(j)} = -\sum_{a \in \mathcal{A}} p_\theta(a \mid y_{<l}^{(j)}) \log p_\theta(a \mid y_{<l}^{(j)})$$

• Combined Objective:

$L$0

Focusing on conservation ensures robust modeling of critical residues, while diversity incentivizes chemically and evolutionarily plausible variability.

3. MSA Selection: The HiFiAD Algorithm

MSA selection is critical to prevent negative impacts from spurious or overly conserved sequences. PLAME introduces the HiFiAD (High-Fidelity Appropriate Diversity) algorithm, which systematically selects virtual sequences for downstream folding:

• Scoring: For each candidate, compute BLOSUM-based similarity to the query and per-position recovery rates.
• Zero-shot (no original MSA): Select the top-k BLOSUM sequences, and k/2 with high and k/2 with low recovery. Concatenate to yield a maximally diverse, yet high-fidelity set.
• Few-shot (original MSA present): Expand up to max(16, 2·N_orig) sequences, prioritizing BLOSUM or recovery, then merge with the original.
• Capping: Ensures the depth of the final augmented MSA does not exceed a predefined threshold, controlling computational cost and redundancy.

HiFiAD thus enforces a balance between informativeness and diversity within the enriched alignment.

4. Sequence Quality Assessment Metrics

PLAME develops four orthogonal, interpretable metrics to characterize MSA composition and its suitability for structure prediction:

Metric	Definition	Interpretation
Conservation Score ($L$1)	$L$2	Fraction of most frequent residue in column $L$3
Gap Proportion ($L$4)	$L$5	Fraction of alignments with gap at position $L$6
Substitution Compatibility	$L$7	BLOSUM sum between query and aligned residues
Alignment Entropy ($L$8)	$L$9, $M$0	Diversity of amino acids at position $M$1

These metrics jointly capture evolutionary signal (conservation), coverage (gaps), chemical plausibility (BLOSUM), and diversity—predictive of downstream AlphaFold confidence and accuracy.

5. Experimental Evaluation and Ablations

PLAME was trained on ~294k non-redundant proteins (UniClust30+PDB), with ≥64 MSA sequences per sample and evaluated on 200 test proteins from CASP14–15, CAMEO, and PDB (<90% similarity to train), including both zero-shot (orphan) and few-shot (low-homology) set-ups.

Folding performance summary (Zero-shot, AlphaFold2 Mode1):

Method	pLDDT	GDT	TM-score	RMSD	LDDT	pTM
AF2–MSA	60.07	0.50	0.50	12.34	0.54	0.44
EvoDiff	58.68	0.46	0.46	13.81	0.50	0.40
MSAGPT	59.81	0.48	0.48	12.62	0.53	0.43
PLAME	66.54	0.53	0.53	11.48	0.57	0.49

PLAME achieves statistically significant improvements (p ≤ 1e-3) on pLDDT and GDT in zero-shot; similar gains are observed in AlphaFold2/3 and ESMFold modes. Combining PLAME with ESMFold achieves AF2-level accuracy with reduced inference runtimes (~6 min/protein versus 30–60 min with AF2 full MSA search).

• Ablation studies confirm that HiFiAD selection improves pLDDT by up to 4 points over random or single-feature filters. Removing the conservation or diversity component leads to respective drops of ≈6 and ≈2 points, and using only MSA features without pretrained embeddings reduces pLDDT by ≈4 points.

6. Case Studies and Design Limitations

• Success cases: On orphan proteins (e.g., 8ehb_F, pLDDT improved from 52.3 to 62.1), PLAME-generated MSAs enhance conservation at catalytic residues. On low-homology targets (8okw_B), higher diversity among virtual sequences recovers functional loops, yielding RMSD gains.
• Failure mode: When the generator collapses onto highly similar outputs (>90% identity), local co-variation signals are lost, reducing local pLDDT even if global metrics are stable.

Appendix visualizations demonstrate that optimal tradeoff between conservation and diversity correlates with structure prediction confidence and accuracy.

7. Implications and Integration in Protein Folding Pipelines

PLAME provides a mechanism for enhancing evolutionary information in cases where MSA search is infeasible or computationally prohibitive, notably for MSA-free predictors such as ESMFold. When used as a preprocessor, PLAME infuses sufficient evolutionary signal to close the accuracy gap with MSA-based pipelines at a fraction of the computational cost, enabling broader applicability of state-of-the-art folding algorithms to challenging targets in genomics, proteomics, and drug discovery (Cao et al., 17 Jun 2025).