InterPLM: Interpreting Protein Language Models

• InterPLM is a mechanistic interpretability framework that applies sparse autoencoders to ESM-2 embeddings, decomposing dense representations into up to 2,548 human-interpretable latent features per layer.
• It aligns these features with up to 143 known biological concepts, exposing superposition in protein language models by contrasting SAE outputs with raw neuron activations.
• The framework enables practical applications such as missing annotation detection, targeted sequence generation via feature steering, and automated LLM-based latent feature interpretation.

InterPLM is a mechanistic interpretability framework and interactive resource for analyzing what protein LLMs represent internally. It applies sparse autoencoders (SAEs) to hidden embeddings from ESM-2 in order to decompose dense, polysemantic representations into a much larger set of sparse latent features that are more human-interpretable. In the reported study, SAEs trained on ESM-2 recover up to 2,548 human-interpretable latent features per layer, align with up to 143 known biological concepts, expose evidence for superposition in protein LLM representations, and support downstream uses including missing-annotation detection and targeted steering of protein sequence generation (Simon et al., 2024).

1. Conceptual basis and motivation

InterPLM was developed to address a central interpretability problem in protein LLMs (PLMs): although models such as ESM-2 are highly effective for protein structure and function prediction, their internal representations are difficult to interpret. The motivating question is what exactly is encoded in these embeddings, and how biological concepts are represented.

The framework is grounded in the mechanistic interpretability notion of superposition. Prior interpretability work is described as showing that single neurons are often polysemantic, with one neuron responding to multiple unrelated concepts. The explanation adopted here is that more concepts are encoded than there are neurons, so the model stores them in overlapping directions in activation space rather than in a one-concept-per-neuron format. InterPLM responds to this by using SAEs to “unmix” the representation space.

The SAE is framed as a dictionary-learning method. Given a PLM embedding vector $\mathbf{x}$, the model learns a sparse latent representation $f(\mathbf{x})$ and reconstructs the original embedding from a larger dictionary of features:

$$\mathbf{x} \approx \mathbf{b} + \sum_i f_i(\mathbf{x}) \mathbf{d}_i$$

where $\mathbf{b}$ is a bias term, $f_i(\mathbf{x})$ is the activation of latent feature $i$, and $\mathbf{d}_i$ is the dictionary vector for feature $i$. The intended effect is to represent a dense embedding as a sparse combination of feature directions. In this formulation, neurons correspond to compressed, entangled, superposed representations, whereas SAE features are sparse directions that partially disentangle the mixture.

This suggests that InterPLM is not primarily an accuracy-oriented framework; its explicit purpose is to identify interpretable latent features, compare them against known biological annotations, detect missing annotations, discover previously unannotated concepts, and manipulate sequence generation through feature steering.

2. Model substrate, data pipeline, and SAE training

The study applies InterPLM to ESM-2-8M-UR50D, described as the smallest ESM-2 variant used in the work. This model has 6 transformer layers, each with hidden size 320. For SAE training, the expansion factor is 32x, yielding a latent dictionary size of about 10,240 features per layer. Separate SAEs are trained for ESM-2 layers 1 through 6, excluding the <cls> and <eos> tokens (Simon et al., 2024).

The training corpus consists of 5 million random protein sequences from UniRef50, which is part of ESM-2’s training data. For each protein, hidden representations are extracted after each transformer block for layers 1–6; <cls> and <eos> tokens are removed; proteins are sharded into groups of 1,000; and tokens are shuffled within those groups to randomize sampling during training. For evaluation of concept annotation alignment, the analysis uses 50,000 Swiss-Prot proteins with length under 1,024 amino acids.

For each ESM-2 layer, the authors train 20 SAEs for 500,000 steps with batch size 2,048, expansion factor 32x, feature dictionary size 10,240, learning rates sampled from $10^{-4}$ to $10^{-8}$ in 10x increments, and L1 penalties from 0.07 to 0.2. Both the L1 penalty and learning rate are linearly warmed up from 0 during early training, with the learning rate reaching its maximum within the first 5% of steps.

The objective combines reconstruction fidelity and sparsity. The metrics given are:

$f(\mathbf{x})$0

$f(\mathbf{x})$1

$f(\mathbf{x})$2

$f(\mathbf{x})$3

Here, $f(\mathbf{x})$4 encourages sparsity, $f(\mathbf{x})$5 measures embedding reconstruction fidelity, $f(\mathbf{x})$6 counts the number of active features, and % Loss Recovered measures how much of the downstream language-model loss is preserved when original embeddings are replaced by SAE reconstructions. The selected SAEs are reported to achieve % Loss Recovered values around 98–100% across layers.

To make features comparable across models and layers, the framework normalizes each feature by scanning 50,000 Swiss-Prot proteins and finding the maximum activation for each feature. Encoder and decoder weights are then rescaled reciprocally so that each feature lies on a normalized scale from 0 to 1 without changing reconstruction behavior.

3. Feature characterization and biological evaluation

InterPLM uses several complementary procedures to identify and interpret latent features. One is direct analysis of activation patterns. Features are examined for structural patterns, where activation occurs on spatially proximal residues in 3D; sequential patterns, where activation occurs on nearby residues in sequence; protein-wide patterns, where activation is broad across an entire protein or family; and functional patterns, where activation falls on annotated motifs, domains, binding sites, and related concepts. The framework visualizes residue-level activation intensities on both sequence and AlphaFold structures.

A second procedure analyzes the geometry of the learned dictionary. Decoder vectors are projected with UMAP and clustered with HDBSCAN to reveal groups of features with shared biological roles. The reported examples include a kinase-related cluster with subtle differences in catalytic-loop preference and a beta-barrel cluster in which some features are specific to TonB-dependent receptors while others generalize to broader beta-barrel proteins. This organization is presented as evidence that the feature space captures biologically meaningful families and subfamilies (Simon et al., 2024).

Quantitative assessment is performed against 433 Swiss-Prot concepts. The concept set spans active sites, binding sites, domains, motifs, disulfide bonds, signal peptides, transit peptides, zinc fingers, modified residues, transmembrane and intramembrane regions, compositional bias, structural elements, and other annotations. Because features and annotations differ in granularity, the evaluation uses a custom scheme: precision is computed per amino acid, recall is computed per domain for domain-level annotations, and F1 scores summarize alignment. For each feature-concept pair, the analysis scans the activation thresholds

$f(\mathbf{x})$7

and selects the threshold with the highest F1.

The use of Swiss-Prot annotations is explicitly treated as a proxy for biological meaning rather than a complete ontology of protein concepts. A practical implication is that coarse annotations may under-describe finer-grained latent features. The domain-adjusted recall is therefore important because it avoids penalizing a feature that detects only the functionally critical residues inside a larger annotated domain.

4. Quantitative results and evidence for superposition

A central quantitative result is the contrast between SAE-derived latent features and raw ESM-2 neurons. SAE features recover up to 2,548 human-interpretable latent features per layer and 143 distinct biological concepts across all layers. The reported repertoire includes catalytic active sites, zinc finger domains, mitochondrial targeting sequences, phosphorylated residues, disulfide bonds, disordered regions, kinase binding sites, peptidase domains, glycosyltransferase regions, TonB-dependent beta barrels, and Nudix motifs, among many others.

By contrast, individual ESM-2 neurons show at most 46 clear concept associations per layer and recover only 15 distinct concepts across all layers. The paper presents this gap as its strongest evidence that biologically meaningful information in PLMs is largely stored in superposition rather than in isolated, concept-aligned neurons (Simon et al., 2024).

A randomized control further sharpens this interpretation. SAEs trained on ESM-2 models with shuffled weights still recover some amino-acid-specific features, but they recover zero complex biological Swiss-Prot concepts. This control indicates that meaningful biological interpretability depends on learned PLM structure rather than merely reflecting the background statistics of protein sequences.

The resulting picture is that ESM-2 hidden states contain many more concept-bearing directions than are visible at the neuron level. InterPLM therefore functions as a decomposition method that partially disentangles a compressed representational basis into sparse directions with clearer biological specificity.

5. Novel latent structure and automated interpretation

The study reports that many features do not correspond neatly to existing Swiss-Prot annotations, yet still display coherent and repeatable activation patterns. These are interpreted as potentially novel protein motifs or structural regularities not captured in curated databases. Reported examples include glycosyltransferase-related patterns that are not fully labeled in Swiss-Prot, a hexapeptide beta helix-like motif appearing across functionally diverse proteins, and feature clusters suggesting more specific subtypes within known families.

The work does not treat such cases as definitive discoveries. Rather, they are framed as hypotheses generated by the model. A common misconception would be to read every coherent feature as a validated new biological entity; the paper is explicit that these cases are promising but provisional, especially when they lack direct correspondence to curated annotations.

To scale interpretation beyond manual inspection, the framework introduces an LLM autointerpretability pipeline using Claude-3.5 Sonnet. For each feature, representative Swiss-Prot proteins are sampled across activation bins; Claude receives protein metadata, activation values, activated residue positions, and amino acid identities at those positions; it then generates both a detailed description and a one-sentence summary beginning with “The feature activates on…”. Validation is performed by asking Claude to predict activation levels on held-out proteins using only the description and protein metadata. The reported result is a median Pearson $f(\mathbf{x})$8 between predicted and true activations (Simon et al., 2024).

An important interpretive point is that the quality of the LLM description is described as only weakly related to Swiss-Prot F1 score. A plausible implication is that a feature may be semantically coherent and describable even when no curated Swiss-Prot annotation provides a strong quantitative match.

6. Applications, causal interventions, and released resources

InterPLM is presented not only as an analysis framework but also as a practical tool for biological curation and model intervention. One application is the identification of likely missing database annotations. Reported examples include f/939, which detects the Nudix box in a protein lacking the Swiss-Prot motif label even though InterPro confirms the motif; f/436, which identifies Peptidase S1 domains missing in Swiss-Prot; and f/9046, which suggests missing UDP-N-acetylglucosamine and Mg$f(\mathbf{x})$9 binding site annotations in bacterial glycosyltransferases. These examples are used to argue that some apparent false positives are more plausibly annotation gaps than feature errors.

A second application is feature steering, which provides a causal test of whether latent features influence model behavior rather than merely correlating with annotations. Using NNsight, the procedure extracts ESM-2 embeddings at a chosen layer, reconstructs them through the SAE, clamps selected feature activations to specified values, adds the SAE reconstruction error back, continues the forward pass, and observes the resulting change in output probabilities. In the reported example, the intervention targets features that fire on periodic glycine patterns such as collagen-like GXXGXX repeats. Activating one such feature at a single glycine position increases the model’s predicted probability of glycine not only at the edited site but also at nearby periodic positions, including masked positions downstream. The effect is described as strongest for the periodic glycine features and weaker or absent for random features and some purely glycine-specific features (Simon et al., 2024).

The associated community resource is InterPLM, an interactive platform at interPLM.ai. It provides views for sequential versus structural activation patterns, protein coverage analysis, UMAP-based feature similarity, and alignment with Swiss-Prot concepts. The work also releases code for training and analysis at github.com/ElanaPearl/interPLM, together with visualization and interpretation tools. In that sense, InterPLM is both a specific empirical study on ESM-2 and a reusable interpretability framework for the broader PLM community.