---
title: 'LAMP-PRo: Multi-label Protein Binding Predictor'
url: https://www.emergentmind.com/topics/lamp-pro
type: topic
---

# LAMP-PRo: Multi-label Protein Binding Predictor

LAMP-PRo denotes “Label-aware Attention for Multi-label Prediction of DNA- and RNA-binding Proteins using Protein Language Models,” a sequence-based framework for identifying DNA-binding proteins (DBPs), RNA-binding proteins (RBPs), dual DNA/RNA-binding proteins (DRBPs), and non-nucleic-acid-binding proteins (non-NABPs) from protein sequence alone [2509.24262]. The method is motivated by two coupled difficulties: DBPs and RBPs are structurally and functionally similar, which leads to substantial cross-prediction errors, and DRBPs are poorly handled by formulations that treat DNA-binding and RNA-binding as unrelated tasks. LAMP-PRo addresses this by combining pretrained protein language model embeddings from ESM-2, a 1D CNN, multi-head self-attention, label-aware attention, cross-label attention, and a multi-label output layer in which DRBP is inferred through simultaneous activation of the DBP and RBP labels rather than through a separate fourth class [2509.24262].

## 1. Task formulation and label semantics

LAMP-PRo is explicitly formulated as a **multi-label** prediction problem rather than a standard multi-class classification problem [2509.24262]. The framework uses three explicit labels: DBP, RBP, and non-NABP. Their encodings are given as DBP \(=[1,0,0]\), RBP \(=[0,1,0]\), and non-NABP \(=[0,0,1]\). DRBP is not introduced as an explicit fourth label; instead it is represented implicitly as co-activation of the DBP and RBP labels, namely \( [1,1,0] \) [2509.24262]. This design is central to the method’s treatment of dual-binding proteins.

A frequent misconception is to read LAMP-PRo as a four-class classifier. The paper does not do that. It treats DRBP as a biologically meaningful multi-label state that emerges when both binding labels are active, while non-NABP remains an explicit negative label [2509.24262]. This matters because the method is designed not only to separate DBPs from RBPs, but also to model the dependence structure between them.

The biological motivation is equally explicit. DBPs and RBPs are described as structurally and functionally similar and evolutionarily correlated, which makes cross-prediction a persistent problem in prior approaches. A DBP may be misclassified as an RBP, or vice versa. DRBPs are more difficult still because methods that treat DNA-binding and RNA-binding as fully separate tasks generally lack a mechanism for representing joint binding capacity [2509.24262]. LAMP-PRo therefore targets three related objectives simultaneously: reducing DBP/RBP confusion, preserving non-NABP discrimination, and improving DRBP recognition.

## 2. Architectural pipeline

The pipeline begins with a raw protein sequence \(\mathcal{P}=\{p_1,p_2,\dots,p_L\}\), which is embedded with ESM-2 [2509.24262]. For a sequence of length \(L\), the embedding stage is written as
\[
X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},
\]
with \(d_e = 640\) [2509.24262]. The paper reports comparing ESM-2-8M, ESM-2-35M, and ESM-2-150M, and selecting the 150M-parameter ESM-2 model because it performed best under the reported computational constraints [2509.24262].

A single 1D convolutional block then extracts local patterns and compresses the sequence representation:
\[
\begin{aligned}
X^{(1)} &= Dropout(GELU(BN(Conv1D(X^{(0)})))) \\
&\in \mathbb{R}^{L' \times d_c}.
\end{aligned}
\]
Here \(L'\) is the post-convolution sequence length and \(d_c\) is the number of CNN output channels [2509.24262]. The paper reports testing 64, 128, 256, and 512 filters and selecting \(d_c=256\) [2509.24262]. It states that sequence length is reduced “via stride,” but does not provide the stride, kernel size, or padding [2509.24262].

The CNN output is passed to multi-head self-attention (MHSA) in order to model long-range dependencies across residues:
\[
head_i = Attention(Q_i,K_i,V_i) = softmax\left(\frac{Q_iK_i^T}{\sqrt{d_i}}\right)V_i,
\]
where
\[
Q_i = X^{(1)}W_i^Q,\qquad K_i = X^{(1)}W_i^K,\qquad V_i = X^{(1)}W_i^V.
\]
The multi-head output is
\[
\begin{aligned}
X^{(2)} &= \text{MHSA}(X^{(1)}) \\
&= \text{Concat}(\text{head}_1, \dots, \text{head}_h) W^O \in \mathbb{R}^{L' \times d_c}.
\end{aligned}
\]
The final model uses \(h=4\) heads, chosen from \(\{2,4,6\}\) [2509.24262].

The CNN and MHSA streams are fused through a gated residual connection:
\[
X^{(3)} = X^{(1)} \oplus g \odot X^{(2)} \in \mathbb{R}^{L' \times d_c},
\]
with \(g=\phi(\mathcal{G})\in(0,1)\) under a sigmoid activation [2509.24262]. This fusion is meant to preserve local motif information while injecting global context in a controlled way.

## 3. Label-aware and cross-label attention

The central representational innovation of LAMP-PRo is the use of **label-aware attention (LAA)** [2509.24262]. Instead of compressing the sequence into a single shared vector, the model learns label embeddings
\[
E_{label} \in \mathbb{R}^{C \times d_c},
\]
with \(C=3\) corresponding to DBP, RBP, and non-NABP. These embeddings act as queries over the residue sequence:
\[
\mathcal{Q} = E_{label}, \qquad \mathcal{K} = X^{(3)}, \qquad \mathcal{V} = X^{(3)},
\]
and the output is
\[
X^{(4)} = softmax\left(\frac{\mathcal{Q}\mathcal{K}^T}{\sqrt{d}}\right)\mathcal{V} \in \mathbb{R}^{C \times d_c}.
\]
This yields three distinct label-specific representations, one per explicit label [2509.24262].

The intended effect is to separate evidence channels. A single pooled representation can blur the distinction between DBP-like and RBP-like cues, especially given the structural and functional similarity of the two binding classes. LAA instead asks, in effect, which residues support DBP, which support RBP, and which support non-NABP. The ablation evidence is unusually strong: when LAA is removed, the paper reports AUC and 1-AURC values collapsing to \(0.50\) across TEST474, PDB255, EZL, and DRBP206, indicating chance-level behavior in that setting [2509.24262]. This suggests that label-specific sequence summarization is not an auxiliary refinement but a load-bearing part of the model.

LAA is followed by **cross-label attention (CLA)**, which operates among the label-specific representations themselves rather than among residues [2509.24262]. The purpose is to model dependencies between labels, particularly DBP–RBP interactions relevant for DRBP recognition. A label mask
\[
\mathcal{M} \in \{0,1\}^{C \times C}
\]
defines which label-to-label interactions are allowed, and the paper gives
\[
M_{i,j} =
\begin{cases}
0, & \text{if label } i \text{ can attend to } j \\
-\infty, & \text{otherwise}.
\end{cases}
\]
For each cross-attention head \(t\),
\[
\mathcal{Q}_{t} = X^{(4)}W_{t}^{\mathcal{Q}}, \qquad
\mathcal{K}_{t} = X^{(4)}W_{t}^{\mathcal{K}}, \qquad
\mathcal{V}_{t} = X^{(4)}W_{t}^{\mathcal{V}},
\]
and the paper writes
\[
X^{(5)} = softmax\left(\frac{\mathcal{Q}_{t}\mathcal{K}_{t}^T}{\sqrt{d} + M}\right)\mathcal{V}_{t} \in \mathbb{R}^{C \times d_c}.
\]
The notation is somewhat unusual as printed, but the stated intent is masked attention over label representations [2509.24262].

CLA is integrated through a second gated residual connection:
\[
X^{(6)} = X^{(4)} \oplus g' \odot X^{(5)} \in \mathbb{R}^{C \times d_c},
\]
with \(g'=\phi(\mathcal{G}')\in(0,1)\) [2509.24262]. The chosen number of CLA heads is 2, selected from \(\{2,4,6\}\) [2509.24262]. The DRBP-specific effect is substantial: in the ablation without CLA, DRBP206 AUC drops to \(0.70\), whereas the full model reaches \(0.96\) [2509.24262]. This is consistent with the design claim that DRBP recognition depends on explicit interaction between the DBP and RBP label channels.

## 4. Prediction layer, loss design, and training protocol

The final prediction layer consists of a linear projection followed by a sigmoid:
\[
z = Linear(X^{(6)}) \in \mathbb{R}^{C}, \qquad
\hat{y} = \sigma(z) \in [0,1]^C,
\]
with \(C=3\) explicit outputs for DBP, RBP, and non-NABP [2509.24262]. Since DRBP is inferred through co-activation, there is no separate DRBP output neuron.

The main objective is binary cross-entropy:
\[
\mathcal{L}_{main} = BCE(\sigma(z), y),
\]
and the paper adds an invalid-label penalty,
\[
\mathcal{L}_{penalty} = InvalidLabelPenalty(\hat{y}),
\]
to discourage biologically inconsistent combinations such as DBP + non-NABP. The total loss is
\[
\mathcal{L}_{total} = \mathcal{L}_{main} + \lambda \cdot \mathcal{L}_{penalty},
\]
with \(\lambda=0.1\) selected from \(\{0.1,0.2,0.3\}\) [2509.24262]. The comparison between BCE and focal loss favored BCE in the reported setup [2509.24262].

The training configuration reported in the paper is partial but specific where given. The selected hyperparameters are learning rate \(10^{-4}\), batch size 32, 256 CNN filters, 4 MHSA heads, and 2 CLA heads [2509.24262]. Training is capped at 15 epochs with early stopping if AUC-ROC does not improve for two consecutive epochs [2509.24262]. All experiments are reported on 2 A40 GPUs [2509.24262].

Several implementation details remain unspecified in the paper. It does not report the optimizer, weight decay, scheduler, dropout probability, CNN kernel size, stride, padding, maximum sequence length, truncation or padding strategy, or whether ESM-2 is frozen or fine-tuned [2509.24262]. It also states that DRBP is inferred from co-activation of DBP and RBP, but does not provide the exact sigmoid threshold used for binarization at inference time [2509.24262]. These omissions are consequential for exact reproduction.

## 5. Datasets, baselines, and empirical results

LAMP-PRo is trained on 10,966 proteins comprising 3846 DBPs, 2616 RBPs, 329 DRBPs, and 4175 non-NABPs [2509.24262]. It is evaluated on four independent datasets: EZL with 2226 DBPs and 1777 RBPs; TEST474 with 175 DBPs, 68 RBPs, 8 DRBPs, and 223 non-NABPs; PDB255 with 93 DBPs, 70 RBPs, and 92 non-NABPs; and DRBP206 with 103 DRBPs and 103 non-NABPs [2509.24262]. The comparisons include DNAbinder, StackDPPred or StackDPP, RNA Pred, RBPPred, Deep-RBPPred, iDRBP_MMC, iDRBP-EL, DMJL, and SERCNN [2509.24262].

For DBP/RBP discrimination, the paper emphasizes AUC and 1-AURC, with the latter interpreted as a measure of cross-prediction mitigation [2509.24262]. On TEST474, LAMP-PRo reports DNA-binding AUC \(0.98\) and 1-AURC \(0.97\), and RNA-binding AUC \(0.90\) and 1-AURC \(0.95\) [2509.24262]. On PDB255, it reports DNA-binding AUC \(0.80\) and 1-AURC \(0.78\), and RNA-binding AUC \(0.81\) and 1-AURC \(0.83\) [2509.24262]. On EZL, the paper reports \(0.96\) for identifying both DNA-binding and RNA-binding proteins in 1-AURC-style analysis [2509.24262]. The results are not uniformly best on every metric—on PDB255 DNA-binding AUC and 1-AURC, some baselines remain stronger—but the paper explicitly interprets this as a trade-off associated with the model’s emphasis on multi-label structure and DRBP prediction [2509.24262].

The most pronounced gains appear in DRBP evaluation. On TEST474, the paper compares predicted DRBP counts and their correctness: iDRBP_MMC predicts 20 proteins as DRBP with only 2 true DRBPs, DMJL predicts 8 with only 2 true DRBPs, while LAMP-PRo predicts 7 with 6 true DRBPs [2509.24262]. Its reported DRBP metrics on TEST474 are recall \(0.75\), precision \(0.85\), and F1-score \(0.80\), compared with SERCNN’s recall \(0.500\), precision \(0.160\), and F1-score \(0.242\) [2509.24262]. On DRBP206, LAMP-PRo reaches AUC \(0.96\), accuracy \(0.88\), and MCC \(0.79\), substantially above iDRBP_MMC at \(0.71/0.64/0.32\), DMJL at \(0.69/0.66/0.34\), and SERCNN at \(0.74/0.69/0.38\) [2509.24262].

The ablation study clarifies the source of these gains. Removing MHSA causes moderate degradation, but removing LAA produces chance-like scores, while removing CLA preserves much of DBP/RBP discrimination yet sharply degrades DRBP performance [2509.24262]. The gated residuals have dataset-specific effects: on more diverse datasets they appear to stabilize prediction and reduce overprediction, but on DRBP206 one gate-removal variant reaches AUC \(0.99\), slightly above the full model’s \(0.96\) [2509.24262]. This suggests that the gating mechanism is useful but not uniformly dominant.

## 6. Interpretability, biological relevance, and limitations

The paper supplements predictive results with attention-based interpretation [2509.24262]. For correctly predicted DBP and RBP sequences, it visualizes the top 30 amino-acid tokens by attention weight. In the reported DBP example, the DBP attention peak is around \(0.3\), whereas the corresponding RBP attention on the same sequence is around \(0.03\); the converse pattern is reported for an RBP sequence [2509.24262]. The authors discuss highlighted residues including K (lysine) and R (arginine), which they note are frequently involved in DNA-binding domains, and also comment on P (proline) in relation to both DNA and RNA interaction contexts [2509.24262]. The paper notes that attention weights are normalized within each sequence, aggregated across heads, and need not sum to 1 in absolute magnitude [2509.24262].

A broader biological relevance analysis combines the test datasets, removes duplicates, and yields 2443 DBPs, 1915 RBPs, and 111 DRBPs [2509.24262]. LAMP-PRo correctly identifies 1957 DBPs, 1633 RBPs, and 85 DRBPs, whereas DMJL identifies 1921 DBPs, 1327 RBPs, and 14 DRBPs [2509.24262]. The paper then discusses recognized disease-related proteins including FUS (P35637), RAR-alpha or RARA (P10276), NONO (Q15233), EP300 (Q09472), DDX3X (O00571), AGO2 (Q9UKV8), DHX9 (Q08211), AGO1 (Q9UL18), HNRPK (P61978), RBMX (P38159), and SON (P18583) [2509.24262]. In a separate new-protein identification analysis using reviewed proteins from the iDRBP-EL site, LAMP-PRo correctly identifies 34 of 36 new DBPs and 4 of 4 new RBPs, with examples including ATF7_CAEEL (Q86MD3) and FB11A_DANRE (A0A2R8QFQ6) [2509.24262].

Several limitations are explicit. The architecture depends on large pretrained PLM embeddings, but the paper does not report whether ESM-2 is frozen, does not specify core optimization details, and does not document sequence-length handling or thresholding for final label decisions [2509.24262]. The full model is not uniformly best on every dataset-metric pair, and the gated residuals can mildly suppress useful overlap on DRBP206 [2509.24262]. A plausible implication is that LAMP-PRo’s strongest contribution is not generic sequence classification accuracy in isolation, but the combination of label-specific evidence extraction and explicit cross-label dependency modeling that is required for distinguishing DBPs from RBPs while still recovering DRBPs [2509.24262].

Source: https://www.emergentmind.com/topics/lamp-pro