LAMP-PRo: Multi-label Protein Binding Predictor

• The paper introduces LAMP-PRo, a framework that leverages protein language models with label-aware and cross-label attention to predict DBP, RBP, and infer DRBP via co-activation.
• It integrates 1D CNN, multi-head self-attention, and gated residual connections to capture local motifs and long-range dependencies in protein sequences.
• Empirical results demonstrate significant improvements in DRBP detection and reduction in cross-prediction errors compared to existing methods.

LAMP-PRo denotes “Label-aware Attention for Multi-label Prediction of DNA- and RNA-binding Proteins using Protein LLMs,” a sequence-based framework for identifying DNA-binding proteins (DBPs), RNA-binding proteins (RBPs), dual DNA/RNA-binding proteins (DRBPs), and non-nucleic-acid-binding proteins (non-NABPs) from protein sequence alone (Ghosh et al., 29 Sep 2025). The method is motivated by two coupled difficulties: DBPs and RBPs are structurally and functionally similar, which leads to substantial cross-prediction errors, and DRBPs are poorly handled by formulations that treat DNA-binding and RNA-binding as unrelated tasks. LAMP-PRo addresses this by combining pretrained protein LLM embeddings from ESM-2, a 1D CNN, multi-head self-attention, label-aware attention, cross-label attention, and a multi-label output layer in which DRBP is inferred through simultaneous activation of the DBP and RBP labels rather than through a separate fourth class (Ghosh et al., 29 Sep 2025).

1. Task formulation and label semantics

LAMP-PRo is explicitly formulated as a multi-label prediction problem rather than a standard multi-class classification problem (Ghosh et al., 29 Sep 2025). The framework uses three explicit labels: DBP, RBP, and non-NABP. Their encodings are given as DBP $=[1,0,0]$, RBP $=[0,1,0]$, and non-NABP $=[0,0,1]$. DRBP is not introduced as an explicit fourth label; instead it is represented implicitly as co-activation of the DBP and RBP labels, namely $[1,1,0]$ (Ghosh et al., 29 Sep 2025). This design is central to the method’s treatment of dual-binding proteins.

A frequent misconception is to read LAMP-PRo as a four-class classifier. The paper does not do that. It treats DRBP as a biologically meaningful multi-label state that emerges when both binding labels are active, while non-NABP remains an explicit negative label (Ghosh et al., 29 Sep 2025). This matters because the method is designed not only to separate DBPs from RBPs, but also to model the dependence structure between them.

The biological motivation is equally explicit. DBPs and RBPs are described as structurally and functionally similar and evolutionarily correlated, which makes cross-prediction a persistent problem in prior approaches. A DBP may be misclassified as an RBP, or vice versa. DRBPs are more difficult still because methods that treat DNA-binding and RNA-binding as fully separate tasks generally lack a mechanism for representing joint binding capacity (Ghosh et al., 29 Sep 2025). LAMP-PRo therefore targets three related objectives simultaneously: reducing DBP/RBP confusion, preserving non-NABP discrimination, and improving DRBP recognition.

2. Architectural pipeline

The pipeline begins with a raw protein sequence $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$, which is embedded with ESM-2 (Ghosh et al., 29 Sep 2025). For a sequence of length $L$, the embedding stage is written as

$$X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},$$

with $d_e = 640$ (Ghosh et al., 29 Sep 2025). The paper reports comparing ESM-2-8M, ESM-2-35M, and ESM-2-150M, and selecting the 150M-parameter ESM-2 model because it performed best under the reported computational constraints (Ghosh et al., 29 Sep 2025).

A single 1D convolutional block then extracts local patterns and compresses the sequence representation: $$\begin{aligned} X^{(1)} &= Dropout(GELU(BN(Conv1D(X^{(0)})))) \ &\in \mathbb{R}^{L' \times d_c}. \end{aligned}$$ Here $L'$ is the post-convolution sequence length and $=[0,1,0]$0 is the number of CNN output channels (Ghosh et al., 29 Sep 2025). The paper reports testing 64, 128, 256, and 512 filters and selecting $=[0,1,0]$1 (Ghosh et al., 29 Sep 2025). It states that sequence length is reduced “via stride,” but does not provide the stride, kernel size, or padding (Ghosh et al., 29 Sep 2025).

The CNN output is passed to multi-head self-attention (MHSA) in order to model long-range dependencies across residues: $=[0,1,0]$2 where

$=[0,1,0]$3

The multi-head output is

$=[0,1,0]$4

The final model uses $=[0,1,0]$5 heads, chosen from $=[0,1,0]$6 (Ghosh et al., 29 Sep 2025).

The CNN and MHSA streams are fused through a gated residual connection: $=[0,1,0]$7 with $=[0,1,0]$8 under a sigmoid activation (Ghosh et al., 29 Sep 2025). This fusion is meant to preserve local motif information while injecting global context in a controlled way.

3. Label-aware and cross-label attention

The central representational innovation of LAMP-PRo is the use of label-aware attention (LAA) (Ghosh et al., 29 Sep 2025). Instead of compressing the sequence into a single shared vector, the model learns label embeddings

$=[0,1,0]$9

with $=[0,0,1]$0 corresponding to DBP, RBP, and non-NABP. These embeddings act as queries over the residue sequence: $=[0,0,1]$1 and the output is

$=[0,0,1]$2

This yields three distinct label-specific representations, one per explicit label (Ghosh et al., 29 Sep 2025).

The intended effect is to separate evidence channels. A single pooled representation can blur the distinction between DBP-like and RBP-like cues, especially given the structural and functional similarity of the two binding classes. LAA instead asks, in effect, which residues support DBP, which support RBP, and which support non-NABP. The ablation evidence is unusually strong: when LAA is removed, the paper reports AUC and 1-AURC values collapsing to $=[0,0,1]$3 across TEST474, PDB255, EZL, and DRBP206, indicating chance-level behavior in that setting (Ghosh et al., 29 Sep 2025). This suggests that label-specific sequence summarization is not an auxiliary refinement but a load-bearing part of the model.

LAA is followed by cross-label attention (CLA), which operates among the label-specific representations themselves rather than among residues (Ghosh et al., 29 Sep 2025). The purpose is to model dependencies between labels, particularly DBP–RBP interactions relevant for DRBP recognition. A label mask

$=[0,0,1]$4

defines which label-to-label interactions are allowed, and the paper gives

$=[0,0,1]$5

For each cross-attention head $=[0,0,1]$6,

$=[0,0,1]$7

and the paper writes

$=[0,0,1]$8

The notation is somewhat unusual as printed, but the stated intent is masked attention over label representations (Ghosh et al., 29 Sep 2025).

CLA is integrated through a second gated residual connection: $=[0,0,1]$9 with $[1,1,0]$0 (Ghosh et al., 29 Sep 2025). The chosen number of CLA heads is 2, selected from $[1,1,0]$1 (Ghosh et al., 29 Sep 2025). The DRBP-specific effect is substantial: in the ablation without CLA, DRBP206 AUC drops to $[1,1,0]$2, whereas the full model reaches $[1,1,0]$3 (Ghosh et al., 29 Sep 2025). This is consistent with the design claim that DRBP recognition depends on explicit interaction between the DBP and RBP label channels.

4. Prediction layer, loss design, and training protocol

The final prediction layer consists of a linear projection followed by a sigmoid: $[1,1,0]$4 with $[1,1,0]$5 explicit outputs for DBP, RBP, and non-NABP (Ghosh et al., 29 Sep 2025). Since DRBP is inferred through co-activation, there is no separate DRBP output neuron.

The main objective is binary cross-entropy: $[1,1,0]$6 and the paper adds an invalid-label penalty,

$[1,1,0]$7

to discourage biologically inconsistent combinations such as DBP + non-NABP. The total loss is

$[1,1,0]$8

with $[1,1,0]$9 selected from $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$0 (Ghosh et al., 29 Sep 2025). The comparison between BCE and focal loss favored BCE in the reported setup (Ghosh et al., 29 Sep 2025).

The training configuration reported in the paper is partial but specific where given. The selected hyperparameters are learning rate $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$1, batch size 32, 256 CNN filters, 4 MHSA heads, and 2 CLA heads (Ghosh et al., 29 Sep 2025). Training is capped at 15 epochs with early stopping if AUC-ROC does not improve for two consecutive epochs (Ghosh et al., 29 Sep 2025). All experiments are reported on 2 A40 GPUs (Ghosh et al., 29 Sep 2025).

Several implementation details remain unspecified in the paper. It does not report the optimizer, weight decay, scheduler, dropout probability, CNN kernel size, stride, padding, maximum sequence length, truncation or padding strategy, or whether ESM-2 is frozen or fine-tuned (Ghosh et al., 29 Sep 2025). It also states that DRBP is inferred from co-activation of DBP and RBP, but does not provide the exact sigmoid threshold used for binarization at inference time (Ghosh et al., 29 Sep 2025). These omissions are consequential for exact reproduction.

5. Datasets, baselines, and empirical results

LAMP-PRo is trained on 10,966 proteins comprising 3846 DBPs, 2616 RBPs, 329 DRBPs, and 4175 non-NABPs (Ghosh et al., 29 Sep 2025). It is evaluated on four independent datasets: EZL with 2226 DBPs and 1777 RBPs; TEST474 with 175 DBPs, 68 RBPs, 8 DRBPs, and 223 non-NABPs; PDB255 with 93 DBPs, 70 RBPs, and 92 non-NABPs; and DRBP206 with 103 DRBPs and 103 non-NABPs (Ghosh et al., 29 Sep 2025). The comparisons include DNAbinder, StackDPPred or StackDPP, RNA Pred, RBPPred, Deep-RBPPred, iDRBP_MMC, iDRBP-EL, DMJL, and SERCNN (Ghosh et al., 29 Sep 2025).

For DBP/RBP discrimination, the paper emphasizes AUC and 1-AURC, with the latter interpreted as a measure of cross-prediction mitigation (Ghosh et al., 29 Sep 2025). On TEST474, LAMP-PRo reports DNA-binding AUC $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$2 and 1-AURC $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$3, and RNA-binding AUC $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$4 and 1-AURC $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$5 (Ghosh et al., 29 Sep 2025). On PDB255, it reports DNA-binding AUC $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$6 and 1-AURC $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$7, and RNA-binding AUC $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$8 and 1-AURC $\mathcal{P}=\{p_1,p_2,\dots,p_L\}$9 (Ghosh et al., 29 Sep 2025). On EZL, the paper reports $L$0 for identifying both DNA-binding and RNA-binding proteins in 1-AURC-style analysis (Ghosh et al., 29 Sep 2025). The results are not uniformly best on every metric—on PDB255 DNA-binding AUC and 1-AURC, some baselines remain stronger—but the paper explicitly interprets this as a trade-off associated with the model’s emphasis on multi-label structure and DRBP prediction (Ghosh et al., 29 Sep 2025).

The most pronounced gains appear in DRBP evaluation. On TEST474, the paper compares predicted DRBP counts and their correctness: iDRBP_MMC predicts 20 proteins as DRBP with only 2 true DRBPs, DMJL predicts 8 with only 2 true DRBPs, while LAMP-PRo predicts 7 with 6 true DRBPs (Ghosh et al., 29 Sep 2025). Its reported DRBP metrics on TEST474 are recall $L$1, precision $L$2, and F1-score $L$3, compared with SERCNN’s recall $L$4, precision $L$5, and F1-score $L$6 (Ghosh et al., 29 Sep 2025). On DRBP206, LAMP-PRo reaches AUC $L$7, accuracy $L$8, and MCC $L$9, substantially above iDRBP_MMC at $$X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},$$0, DMJL at $$X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},$$1, and SERCNN at $$X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},$$2 (Ghosh et al., 29 Sep 2025).

The ablation study clarifies the source of these gains. Removing MHSA causes moderate degradation, but removing LAA produces chance-like scores, while removing CLA preserves much of DBP/RBP discrimination yet sharply degrades DRBP performance (Ghosh et al., 29 Sep 2025). The gated residuals have dataset-specific effects: on more diverse datasets they appear to stabilize prediction and reduce overprediction, but on DRBP206 one gate-removal variant reaches AUC $$X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},$$3, slightly above the full model’s $$X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},$$4 (Ghosh et al., 29 Sep 2025). This suggests that the gating mechanism is useful but not uniformly dominant.

6. Interpretability, biological relevance, and limitations

The paper supplements predictive results with attention-based interpretation (Ghosh et al., 29 Sep 2025). For correctly predicted DBP and RBP sequences, it visualizes the top 30 amino-acid tokens by attention weight. In the reported DBP example, the DBP attention peak is around $$X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},$$5, whereas the corresponding RBP attention on the same sequence is around $$X^{(0)} = ESM\text{-}2(\mathcal{P}) \in \mathbb{R}^{L \times d_e},$$6; the converse pattern is reported for an RBP sequence (Ghosh et al., 29 Sep 2025). The authors discuss highlighted residues including K (lysine) and R (arginine), which they note are frequently involved in DNA-binding domains, and also comment on P (proline) in relation to both DNA and RNA interaction contexts (Ghosh et al., 29 Sep 2025). The paper notes that attention weights are normalized within each sequence, aggregated across heads, and need not sum to 1 in absolute magnitude (Ghosh et al., 29 Sep 2025).

A broader biological relevance analysis combines the test datasets, removes duplicates, and yields 2443 DBPs, 1915 RBPs, and 111 DRBPs (Ghosh et al., 29 Sep 2025). LAMP-PRo correctly identifies 1957 DBPs, 1633 RBPs, and 85 DRBPs, whereas DMJL identifies 1921 DBPs, 1327 RBPs, and 14 DRBPs (Ghosh et al., 29 Sep 2025). The paper then discusses recognized disease-related proteins including FUS (P35637), RAR-alpha or RARA (P10276), NONO (Q15233), EP300 (Q09472), DDX3X (O00571), AGO2 (Q9UKV8), DHX9 (Q08211), AGO1 (Q9UL18), HNRPK (P61978), RBMX (P38159), and SON (P18583) (Ghosh et al., 29 Sep 2025). In a separate new-protein identification analysis using reviewed proteins from the iDRBP-EL site, LAMP-PRo correctly identifies 34 of 36 new DBPs and 4 of 4 new RBPs, with examples including ATF7_CAEEL (Q86MD3) and FB11A_DANRE (A0A2R8QFQ6) (Ghosh et al., 29 Sep 2025).

Several limitations are explicit. The architecture depends on large pretrained PLM embeddings, but the paper does not report whether ESM-2 is frozen, does not specify core optimization details, and does not document sequence-length handling or thresholding for final label decisions (Ghosh et al., 29 Sep 2025). The full model is not uniformly best on every dataset-metric pair, and the gated residuals can mildly suppress useful overlap on DRBP206 (Ghosh et al., 29 Sep 2025). A plausible implication is that LAMP-PRo’s strongest contribution is not generic sequence classification accuracy in isolation, but the combination of label-specific evidence extraction and explicit cross-label dependency modeling that is required for distinguishing DBPs from RBPs while still recovering DRBPs (Ghosh et al., 29 Sep 2025).