Applm: Protein LM Allergen Predictor

• Applm is a computational framework that leverages pretrained protein language model embeddings and a Random Forest classifier to accurately identify allergen proteins from raw sequences.
• It employs a two-stage architecture where sequence embeddings are averaged and fed into either a frozen or LoRA-tuned classifier, ensuring robustness across novel, homologous, and mutational evaluation scenarios.
• Benchmark evaluations demonstrate that Applm outperforms several previous allergen prediction methods, achieving top-tier AUROC and AUPRC scores under stringent, similarity-aware testing protocols.

Applm, short for Allergen Prediction with Protein LLMs, is a computational framework for identifying allergen proteins from raw amino-acid sequences by combining pretrained protein LLM (pLM) embeddings with a downstream classifier (Wong et al., 14 Aug 2025). The framework was introduced for allergen prediction under evaluation settings intended to resemble difficult real-world scenarios, including discovery of novel allergens absent from the training set, discrimination between allergens and non-allergens within homologous families, and assessment of mutations that induce only small sequence changes. In addition to the predictive pipeline itself, Applm includes carefully curated benchmark datasets and an explicitly generalization-focused evaluation methodology.

1. Problem setting and design rationale

Allergens are described as proteins capable of triggering adverse immune responses and as a significant public health challenge. Applm addresses the computational identification of such proteins from sequence alone, with an emphasis on cases where naive similarity-based discrimination is difficult (Wong et al., 14 Aug 2025).

The framework is organized around three evaluation scenarios. The first is novel allergen prediction, implemented through “By Date” splits in which training uses sequences entered no later than 2020 and testing uses sequences entered after 2020. The second is homolog discrimination, where allergens must be separated from non-allergens inside protein families with high inter-class sequence similarity. The third is mutational effects, where positives and negatives can differ by only a few residues and often have sequence identity greater than $95\%$.

This evaluation design places Applm in a setting where success is not reducible to memorizing close training neighbors. A plausible implication is that the framework is intended less as a generic binary classifier on random splits than as a testbed for out-of-distribution and fine-grained sequence discrimination.

2. Two-stage architecture

Applm takes as input a raw protein sequence of length $L$. Its embedding stage supports four pretrained pLM backbones, each producing an $L \times D$ matrix of hidden representations. Applm then applies average pooling across the sequence length to obtain a single vector representation,

$v = \frac{1}{L}\sum_{i=1}^{L} h_i$

where $h_i \in \mathbb{R}^D$ is the hidden vector for residue $i$ (Wong et al., 14 Aug 2025).

The supported backbones are listed below.

Backbone	Parameters	Output embedding size
xTrimoPGLM-100B	100B	10,240
xTrimoPGLM-10B	10B	4,352
ESM-2	650M	1,280
ProtT5	3B	1,024

The classification stage has two variants. The main, frozen-embedding variant feeds the pooled vector $v$ into a Random Forest with 1,000 trees using scikit-learn default, Gini-impurity splitting. The exploratory variant attaches a small multilayer head through LoRA, a low-rank adapter, and fine-tunes the pLM parameters for binary cross-entropy loss.

This separation between representation learning and classification is central to Applm’s design. The paper explicitly attributes the framework’s performance to pLMs, especially xTrimoPGLM, as sequence models trained at massive scale to capture general protein sequence characteristics.

3. Training objectives and optimization variants

For the frozen-embedding pipeline, the downstream classifier uses standard Gini-impurity minimization inside the Random Forest, and no explicit cross-entropy loss is computed (Wong et al., 14 Aug 2025). This is the principal training configuration reported for Applm.

For the fine-tuned variant, the binary target for protein $i$ is $y_i \in \{0,1\}$, the predicted probability is

$p_i = \sigma(w \cdot v_i + b),$

and the optimization objective over $L$0 examples is the binary cross-entropy

$L$1

Optimization uses AdamW, with early stopping on validation loss.

The empirical comparison between frozen embeddings and parameter-efficient adaptation is notable. LoRA tuning of ESM-2, ProtT5, and xTrimo10B did not improve external test performance, and the reported explanation is that the available training sets are small and that the evaluation involves distributional shifts. This directly challenges the common assumption that downstream fine-tuning necessarily dominates frozen-feature pipelines in limited-data biological classification.

4. Dataset curation and evaluation protocol

Applm’s training data are drawn from three independent sources: WHO/IUIS, COMPARE, and AllergenOnline (Wong et al., 14 Aug 2025). Positive examples are peer-reviewed allergens with molecularly characterized, IgE-binding evidence and are filtered to satisfy $L$2, contain canonical amino acids, and exclude substrings/duplicates. Negative examples are reviewed UniProt eukaryotic proteins not annotated as allergens, subject to the same length constraints and with any overlap with positives removed.

The six external test sets are grouped into three scenario families.

Scenario	Construction	Representative sizes/details
By Date	Train on entries $L$3, test on entries $L$4 for WHO/IUIS, COMPARE, AllergenOnline	Allergens: 50, 107, 60; non-allergens: 3,655 each
Homolog families	High inter-class similarity within four families	Arginine Kinases 10 vs. 14; Cysteine Proteases 11 vs. 223; Serine Proteases 32 vs. 417; Tropomyosins 28 vs. 59
Mutations	Literature-derived wild-type vs. mutant discrimination	22 wild-type allergens vs. 43 site-directed or deletion mutants; identity often $L$5

The homolog-family splits include four protein families known for cross-reactivity: Arginine Kinases, Cysteine Proteases, Serine Proteases, and Tropomyosins. The mutation benchmark is designed around subtle perturbations, making the task especially stringent.

For internal ablation, the study also uses similarity-aware cross-validation with two thresholds. $L$6 controls the maximum allowed sequence identity between training and test examples, and $L$7 requires each negative to share at least a specified identity with some positive in its split. The values explored are $L$8 and $L$9.

Performance is reported with AUROC and AUPRC:

$L \times D$0

$L \times D$1

5. Comparative performance

Across the six external test sets, Applm variants occupy the top tier of the comparison, and the framework is reported to outperform seven prior allergen-prediction methods as a family (Wong et al., 14 Aug 2025). The average metrics are as follows.

Model	Avg AUROC	Avg AUPRC
Applm (xTrimoPGLM-100B)	0.872	0.700
Applm (ProtT5)	0.879	0.718
Applm (xTrimoPGLM-10B)	0.870	0.705
Applm (ESM-2)	0.876	0.712
AlgPred2.0	0.807	0.596
Alg-MFDL	0.811	0.616
DeepAlgPro	0.780	0.558
pLM4Alg	0.740	0.554
AllerTOP v2	0.660	0.520
AllergenFP	0.616	0.523
Random baseline	0.500	0.202

The statistical ranking uses Friedman + Nemenyi post-hoc analysis. Under this analysis, Applm (any backbone) ranked #1 by both AUROC and AUPRC, and it significantly outperformed six of seven prior methods at $L \times D$2, with critical difference $L \times D$3 for 10 methods.

Per-scenario highlights further illustrate the intended operating regime. On By Date, Applm reaches AUROC $L \times D$4 versus the next best $L \times D$5. On Homologs, the Tropomyosin split reaches AUROC $L \times D$6 for Applm versus $L \times D$7 for other methods. On Mutations, Applm reaches AUROC $L \times D$8 versus $L \times D$9. These figures are important because they derive from scenarios where positives and negatives can be highly similar in sequence.

6. Ablations, error analysis, and methodological implications

The ablation study compares pLM embeddings to conventional sequence encodings under similarity-aware cross-validation (Wong et al., 14 Aug 2025). pLM embeddings yielded overall AUROC $v = \frac{1}{L}\sum_{i=1}^{L} h_i$0–$v = \frac{1}{L}\sum_{i=1}^{L} h_i$1, whereas One-Hot achieved only $v = \frac{1}{L}\sum_{i=1}^{L} h_i$2 and BLOSUM62 only $v = \frac{1}{L}\sum_{i=1}^{L} h_i$3. The reported heatmap over $v = \frac{1}{L}\sum_{i=1}^{L} h_i$4 shows a smooth performance gradient for pLMs, while conventional encodings collapse in difficult regimes characterized by low $v = \frac{1}{L}\sum_{i=1}^{L} h_i$5 and high $v = \frac{1}{L}\sum_{i=1}^{L} h_i$6.

The classifier comparison also favors the chosen design. A feed-forward neural network with 1,024 hidden units, dropout, and batch-norm, trained on the same embeddings, dropped AUROC by $v = \frac{1}{L}\sum_{i=1}^{L} h_i$7–$v = \frac{1}{L}\sum_{i=1}^{L} h_i$8 points, which the paper interprets as evidence for Random Forest robustness in limited-data settings.

A separate analysis addresses a possible shortcut. Regression of sequence length from pLM embeddings yields low MSE ($v = \frac{1}{L}\sum_{i=1}^{L} h_i$9 on length $h_i \in \mathbb{R}^D$0), indicating that the embeddings encode length information. This led to the “Length Control” training strategy, which matches positive and negative length distributions and improved performance when test length distributions were uniform. This is methodologically significant because it shows that high-capacity sequence embeddings may carry nuisance information alongside task-relevant signal.

Several broader implications follow directly from these analyses. First, the dominant performance gain is not attributable to simple handcrafted encodings. Second, the framework’s strongest results do not depend on end-to-end fine-tuning. Third, evaluation difficulty is explicitly parameterized by sequence-similarity constraints rather than treated as a fixed property of a dataset.

7. Limitations, open questions, and future directions

The paper identifies several limitations and corresponding research directions (Wong et al., 14 Aug 2025). Fine-tuning did not yield gains, so more sophisticated transfer-learning methods or task-specific adapters are proposed as future work. Applm currently uses only sequence-based embeddings; integration of structure predictions such as AlphaFold or ESMFold is suggested as a potential extension.

Model selection for an unlabeled target domain remains unresolved. The reported recommendation is to rely on heuristic estimates of test-set difficulty, such as intra-set identity or known family membership. Interpretability is also presented as a key next step: identifying which amino-acid positions drive allergenicity would support rational design of hypoallergenic variants.

The study additionally discusses possible pretraining-overlap concerns. The impact of pLM pre-training overlap, described as “leakage,” appears task-dependent; Applm reports no inflation on “By Date” tests, but the paper states that this issue deserves case-by-case evaluation in other contexts. This positions Applm within an ongoing methodological debate in foundation-model-based biology: high downstream accuracy is not, by itself, sufficient evidence that evaluation protocols are leakage-resistant.

Taken together, Applm defines allergen prediction as a generalization problem over curated, similarity-aware sequence benchmarks, implements that perspective with frozen pLM embeddings and a Random Forest classifier, and provides open-source software together with benchmark datasets for subsequent work.