Equi-mRNA: Equivariant mRNA Model

• The paper introduces Equi-mRNA, which explicitly models synonymous codon symmetries as cyclic groups embedded in SO(2) to preserve protein identity under synonymous substitutions.
• It employs a novel embedding and equivariance regularization strategy that leverages codon geometry to encode translation efficiency, gene expression, and mRNA stability.
• Empirical results across multiple benchmarks show that Equi-mRNA outperforms baselines while offering interpretable insights into codon usage bias and translational optimization.

Searching arXiv for Equi-mRNA and closely related mRNA language/design papers to ground the article in current literature. Searching for "Equi-mRNA protein translation equivariant encoding mRNA LLMs" on arXiv. Equi-mRNA is a codon-level equivariant mRNA LLM for protein-coding messenger RNA that treats synonymous codon structure as an explicit symmetry of the genetic code rather than as a nuisance variable to be absorbed by generic token embeddings (Yazdani-Jahromi et al., 20 Aug 2025). Its central premise is that the genetic code is a many-to-one map from codons to amino acids, so protein identity must be preserved under synonymous substitution while codon-level variation still carries information about translation efficiency, gene expression, mRNA stability, and protein folding dynamics (Yazdani-Jahromi et al., 20 Aug 2025). To encode this, Equi-mRNA represents each synonymous codon set as a cyclic group and embeds it into planar rotations in $SO(2)$, then regularizes the model so that synonymous substitutions induce structured latent transformations rather than arbitrary embedding displacements (Yazdani-Jahromi et al., 20 Aug 2025).

1. Biological setting and conceptual motivation

Equi-mRNA is defined on protein-coding mRNA sequences and is motivated by the claim that synonymous codons are “synonymous but not silent” (Yazdani-Jahromi et al., 20 Aug 2025). The paper formalizes translation as a surjective map

$\pi: \mathcal{C} \to \mathcal{A},$

where $\mathcal{C}$ is the set of 64 codons and $\mathcal{A}$ is the set of 20 amino acids plus stop, with synonymous codon class

$$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$

Within this formulation, amino-acid-level identity is invariant under synonymous substitution, but codon usage bias remains biologically consequential for translation efficiency, elongation speed, gene expression, mRNA stability, and protein folding dynamics (Yazdani-Jahromi et al., 20 Aug 2025).

A central misconception addressed by the paper is that codon degeneracy implies functional interchangeability. Equi-mRNA rejects that view. It argues that amino-acid-level models discard codon-level regulatory information, whereas nucleotide-level models generally do not encode coding-frame structure or synonymous relationships explicitly (Yazdani-Jahromi et al., 20 Aug 2025). It also distinguishes itself from earlier codon-aware approaches that impose auxiliary objectives without explicitly modeling the internal symmetry structure of synonymous sets. In the paper’s framing, those methods may encourage synonymous codons to be treated similarly, but they do not define a mathematically explicit transformation law for moving within a synonym class (Yazdani-Jahromi et al., 20 Aug 2025).

2. Group-theoretic formulation of synonymous codon symmetry

The model’s defining construction is to assign a cyclic group to each synonymous codon set. For each amino acid $a$, Equi-mRNA sets

$G_a \cong \mathbb{Z}_{n_a},$

and introduces a bijection

$\varphi_a: \mathcal{C}_a \to \{0,1,\dots,n_a-1\}.$

This induces a cyclic group structure on $\mathcal{C}_a$ via addition modulo $n_a$ (Yazdani-Jahromi et al., 20 Aug 2025).

To obtain a differentiable representation, the paper embeds each synonymous group into $\pi: \mathcal{C} \to \mathcal{A},$0. It defines

$\pi: \mathcal{C} \to \mathcal{A},$1

and maps each codon $\pi: \mathcal{C} \to \mathcal{A},$2 to the rotation

$\pi: \mathcal{C} \to \mathcal{A},$3

The image is the cyclic subgroup

$\pi: \mathcal{C} \to \mathcal{A},$4

which is isomorphic to $\pi: \mathcal{C} \to \mathcal{A},$5 (Yazdani-Jahromi et al., 20 Aug 2025).

The appendix states the homomorphism property explicitly: $\pi: \mathcal{C} \to \mathcal{A},$6 with $\pi: \mathcal{C} \to \mathcal{A},$7 and $\pi: \mathcal{C} \to \mathcal{A},$8, provided $\pi: \mathcal{C} \to \mathcal{A},$9 (Yazdani-Jahromi et al., 20 Aug 2025). In this construction, synonymous substitution is not represented as a categorical swap but as motion along a discrete rotational orbit.

The paper then generalizes the fixed-angle construction in two ways. In the learned-generator variant, $\mathcal{C}$0 is trainable subject to the cyclicity constraint $\mathcal{C}$1. In the fuzzy-generator variant, each codon is assigned a distribution over angle prototypes, yielding an effective angle

$\mathcal{C}$2

where $\mathcal{C}$3 is a softmax probability and $\mathcal{C}$4 is the number of synonymous codons for amino acid $\mathcal{C}$5 (Yazdani-Jahromi et al., 20 Aug 2025). This replaces a rigid cyclic action with a task-adaptive relaxation.

3. Architecture, embeddings, and equivariance objective

Equi-mRNA operates at codon resolution: sequences are codonized into triplets, producing a vocabulary of 64 codons (Yazdani-Jahromi et al., 20 Aug 2025). In the ablation study, all 12 variants share an identical GPT-2 backbone so that differences can be attributed to embedding geometry and equivariance regularization rather than to backbone scale (Yazdani-Jahromi et al., 20 Aug 2025). The paper also reports a 5M-parameter hybrid Mamba–Transformer variant and a 15M-parameter GPT-2 variant retrained at full scale (Yazdani-Jahromi et al., 20 Aug 2025).

Its embedding construction is not a standard lookup table. For amino acid $\mathcal{C}$6, the model defines a base vector $\mathcal{C}$7, and in the general Stiefel formulation it also learns a two-dimensional orthonormal subspace $\mathcal{C}$8, where

$\mathcal{C}$9

A codon $\mathcal{A}$0 with angle $\mathcal{A}$1 is embedded as

$\mathcal{A}$2

or equivalently

$\mathcal{A}$3

All synonymous codons for an amino acid therefore lie on a circle in a learned two-dimensional semantic plane inside $\mathcal{A}$4 (Yazdani-Jahromi et al., 20 Aug 2025).

To preserve that structure beyond the input layer, Equi-mRNA adds an auxiliary equivariance loss. The intended equivariance condition is

$\mathcal{A}$5

with penalty

$\mathcal{A}$6

and total objective

$\mathcal{A}$7

The paper’s interpretation is that synonym substitutions should produce coherent latent rotations rather than destroy protein-level structure (Yazdani-Jahromi et al., 20 Aug 2025).

Equi-mRNA also introduces symmetry-aware pooling, including polar pooling, DFT-based rotation-aware pooling, and an $\mathcal{A}$8 mean defined through pooled angles

$\mathcal{A}$9

followed by reconstruction into cosine–sine coordinates (Yazdani-Jahromi et al., 20 Aug 2025). This indicates that the symmetry prior is meant to shape both token embeddings and sequence-level readout.

4. Training regime and empirical performance

The pretraining corpus is RefSeq CDS data filtered to coding sequences of 20 to 512 codons. From an initial pool of about 56 million protein-coding entries, the paper samples 25 million coding sequences; the reported taxonomic composition is 37.6% other vertebrates, 24.4% mammals, 22.8% invertebrates, 13.7% fungi, and 1.4% viruses (Yazdani-Jahromi et al., 20 Aug 2025). A 1M-sequence stratified subset is used for the 12-way ablation, and the best-performing variant—fuzzy $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$0 with Stiefel basis and equivariance—is retrained on the full 25M corpus (Yazdani-Jahromi et al., 20 Aug 2025).

The ablation stage uses batch size 1024, gradient accumulation 2, 50 epochs, warmup 5 epochs, cosine scheduling, max length 512 codons, and bf16-mixed precision on eight NVIDIA H200 GPUs (Yazdani-Jahromi et al., 20 Aug 2025). Full-scale training uses batch size 1024, gradient accumulation 2, 20 epochs, warmup 2 epochs, cosine scheduling, max length 512 codons, bf16-mixed precision, and thirty-two NVIDIA H100 GPUs (Yazdani-Jahromi et al., 20 Aug 2025).

Downstream evaluation covers six datasets: MLOS, mRFP, E. coli expression, Tc-riboswitch, iCodon, and SARS-CoV-2 degradation (Yazdani-Jahromi et al., 20 Aug 2025). On the full 25M benchmark, the paper reports the following for Equi-mRNA (15M): E. coli $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$1, MLOS $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$2, iCodon $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$3, Tc-Ribo $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$4, mRFP $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$5, and COV Deg $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$6. The 5M model reports $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$7, $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$8, $$\mathcal{C}_a = \pi^{-1}(a), \qquad n_a = |\mathcal{C}_a|.$$9, $a$0, $a$1, and $a$2, respectively (Yazdani-Jahromi et al., 20 Aug 2025). The paper states that the 15M model achieves the highest score in 5 of 6 tasks while using about 30% of HELM’s parameters (Yazdani-Jahromi et al., 20 Aug 2025).

Generation results are reported on thermostability-style suffix completion. The abstract summarizes these as up to approximately $a$3 more realistic under Fréchet BioDistance and approximately 28% better functional preservation than a vanilla baseline (Yazdani-Jahromi et al., 20 Aug 2025). The detailed full-scale evaluation gives FBD $a$4 for Equi-mRNA (5M) at $a$5, compared with $a$6 for vanilla; the 15M model reports $a$7 at the same temperature (Yazdani-Jahromi et al., 20 Aug 2025). A notable empirical claim is that the equivariant models improve as temperature increases, whereas the vanilla model degrades, which the authors interpret as evidence of structured exploration of synonymous alternatives (Yazdani-Jahromi et al., 20 Aug 2025).

5. Interpretability and recovered biological structure

A distinctive feature of Equi-mRNA is that its codon geometry is intended to remain biologically interpretable. After fine-tuning the 5M model on the human coding transcriptome, the paper measures the Shannon entropy of transcript-level codon-angle distributions,

$a$8

where $a$9 is the empirical bin frequency of learned codon angles in sequence $G_a \cong \mathbb{Z}_{n_a},$0 (Yazdani-Jahromi et al., 20 Aug 2025). The reported relationship between mean entropy and GC content yields Pearson $G_a \cong \mathbb{Z}_{n_a},$1, $G_a \cong \mathbb{Z}_{n_a},$2, and $G_a \cong \mathbb{Z}_{n_a},$3 (Yazdani-Jahromi et al., 20 Aug 2025). The paper interprets this as evidence that learned rotational structure recapitulates GC-driven codon bias.

It also compares learned codon angles to normalized human tRNA abundance using Spearman rank correlation,

$G_a \cong \mathbb{Z}_{n_a},$4

and reports

$G_a \cong \mathbb{Z}_{n_a},$5

Codons with more abundant tRNAs are assigned systematically smaller angles (Yazdani-Jahromi et al., 20 Aug 2025). The authors interpret this as a latent axis related to translational optimality.

These analyses address a second possible misconception: that a symmetry prior would collapse biologically meaningful codon differences. Equi-mRNA does not treat synonymous codons as identical. Rather, it arranges them on structured orbits around a shared amino-acid center. The reported GC and tRNA analyses suggest that this geometry preserves codon-level distinctions while organizing them into an interpretable manifold (Yazdani-Jahromi et al., 20 Aug 2025).

6. Relation to adjacent mRNA modeling directions, scope, and limitations

Equi-mRNA occupies a specific position within mRNA modeling. It is not a full-transcript design system: pretraining keeps CDS only and discards UTRs and other flanking regions (Yazdani-Jahromi et al., 20 Aug 2025). This differentiates it from methods such as mRNAutilus, which generates complete transcripts with joint CDS and UTR optimization through masked discrete diffusion and Pareto-guided search (Patel et al., 29 May 2026), and LinearDesign2, which jointly optimizes 5′ UTR and CDS for translation initiation efficiency, codon adaptation index, and minimum free energy (Liu et al., 2024). It also differs from UTR-LM, which is specific to 5′ UTR function prediction and design (Chu et al., 2023).

At the representation-learning level, Equi-mRNA is closer to geometry-aware sequence modeling than to systems-oriented compression or full-transcript generation. HyperHELM, for example, uses hyperbolic geometry to encode codon–amino-acid hierarchy, but it is explicitly not an equivariant model and instead targets hierarchical structure in a Poincaré ball (Spengler et al., 29 Sep 2025). HelixNano-mRNA, by contrast, is an efficient distilled mRNA encoder derived from Evo2-1B via embedding matching and is primarily a compression strategy rather than a symmetry-aware architecture (Haidari et al., 27 Mar 2026). Equi-mRNA’s distinctive claim is therefore narrower and more formal: codon degeneracy should be represented as symmetry.

The paper also identifies clear limitations. It is restricted to coding regions, assumes fixed triplet tokenization, may obscure some species- or gene-specific codon usage detail, and incurs optimization overhead because fuzzy angle distributions and Stiefel-manifold rotations require Riemannian optimization (Yazdani-Jahromi et al., 20 Aug 2025). Benchmark scope is also limited relative to full therapeutic design. The authors propose future directions including meta-learning codon priors, richer group structures such as non-abelian or product groups, modeling local codon interactions and positional dependencies, and extension to non-coding regions (Yazdani-Jahromi et al., 20 Aug 2025).

A plausible implication is that Equi-mRNA is best read as a codon-geometry foundation for protein-coding sequence modeling rather than as a complete therapeutic mRNA platform. Its contribution is to formalize synonymous codon structure as an equivariance problem and to show that this prior can improve both prediction and generation while remaining interpretable in terms of GC-content bias and tRNA abundance (Yazdani-Jahromi et al., 20 Aug 2025).