epiGPTope: Sequence-Only Epitope Discovery

• epiGPTope is a sequence-only framework that uses a generative protein language model to learn and sample linear epitope sequences de novo.
• It combines generative modeling with downstream classifiers to filter peptides by their bacterial or viral origin, thus reducing the experimental search space.
• The approach validates synthetic epitopes statistically against natural distributions, providing actionable insights for wet-lab vaccine and diagnostic applications.

Searching arXiv for the cited paper and closely related epitope-prediction work. epiGPTope is a sequence-only machine-learning framework for linear epitope discovery that combines a generative LLM with downstream classifiers. It is designed to learn the distribution of known natural linear epitopes, generate new epitope-like peptide sequences de novo, and then filter those candidates with predictive models that enrich for epitopes of a desired origin, specifically bacterial or viral. The method is explicitly restricted to linear epitopes derived from proteogenic antigens, uses only primary amino acid sequences, and is framed as an aid to wet-lab screening for vaccines, diagnostics, immunotherapies, and related biotechnology applications (Manrique et al., 3 Sep 2025).

1. Biological scope and problem setting

Epitopes are short antigenic peptide regions recognized by antibodies or immune-cell receptors. The framework distinguishes linear epitopes, where recognition is associated directly with the contiguous amino acid sequence, from conformational epitopes, where the recognized patch depends on three-dimensional folding. epiGPTope addresses only the former. This restriction is central: the model does not attempt to represent antigen structure, geometric interface shape, or antibody-specific pairing.

The paper’s motivation is combinatorial. For linear peptides of length $n$, there are $20^n$ possible sequences, which makes exhaustive screening impractical even with high-throughput platforms such as droplet microfluidics or phage display. At the same time, the number of experimentally validated epitopes is limited, and the underlying sequence patterns that make a peptide epitope-like are subtle. epiGPTope therefore adopts the analogy between amino acid sequences and natural language, using a protein LLM to estimate a distribution over linear epitope sequences and then sample from that distribution.

The practical framing is two-stage. First, the system creates a broad-purpose synthetic epitope library. Second, predictive models narrow that library toward candidates more relevant for a given application by classifying whether an epitope sequence is of bacterial or viral origin. The paper proposes that the combination of generative and predictive models can assist epitope discovery by reducing the experimental search space and increasing the fraction of useful candidates in downstream screening (Manrique et al., 3 Sep 2025).

2. Sequence-only architecture and probabilistic formulation

The generative backbone is ProtGPT2, a GPT2-like autoregressive protein LLM previously pre-trained on large protein sequence corpora. ProtGPT2 itself was pre-trained on UniRef50 (version 2021_04), a clustering of UniProt at 50% sequence identity. According to the paper, ProtGPT2 “shares the architecture of the original GPT2 model,” has 738 million parameters, and is “arranged in 6 layers with a model dimensionality of 1280.” The same paper later describes the model as “quite large (1.5 billion parameters and about 9GB in memory).” This is an internal inconsistency in the manuscript: both values are reported, and the effective parameter count of the deployed fine-tuned model is not clarified.

epiGPTope adapts this backbone by fine-tuning it on curated linear epitopes from IEDB. The generator then samples sequences autoregressively from the learned epitope distribution. In the paper’s formulation, if $W=(W_1,\dots,W_n)$ is a token sequence, the model defines

$p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$

and training minimizes negative log-likelihood over the dataset: $$L_{\mathrm{CLM}} = - \sum_{k=1}^{R} \log p_\theta(w_k \mid w_{<k}).$$ This is the standard causal-language-model objective in which the model predicts the next amino acid token from the left context.

The paper states that the fine-tuning format emulated that of the pre-training dataset, but it does not specify the exact tokenization scheme in sufficient detail. It reports a contextual embedding size of 1280 per token. For classification embeddings, three pooling strategies were compared to compress token-level embeddings into sequence-level representations: keeping the rightmost final hidden state, summing all final hidden states, and a weighted sum favoring the rightmost state. Summing all final hidden states performed best. This suggests that, within the reported experiments, sequence-level information was better preserved by global aggregation than by right-context summarization alone (Manrique et al., 3 Sep 2025).

3. Data curation and model training

All sequences were retrieved from IEDB. Initial filtering retained only entries from human host assays, restricted to linear epitopes and to assay types T cell, B cell, or MHC. The epitope dataset for generation was then restricted to sequences with at most 11 residues, and duplicates were removed. The final dataset used for fine-tuning the generator contained 504,611 sequences. The paper states that “the whole dataset is used to train epiGPTope,” rather than reporting a conventional train/validation/test split for the generative model.

For classification, the authors used eight subsets filtered by organism type, bacterial or viral, and assay type, T cell, B cell, MHC, or all combined, together with negative data. However, the paper does not provide exact sample counts for these subsets, nor does it describe in detail how the negative examples were assembled, split, balanced, or deduplicated. It also omits explicit train/validation/test proportions and cross-validation protocols for the classifiers. These omissions materially limit reproducibility and interpretability.

Fine-tuning was run on an AWS EC2 G5.2xlarge instance with an NVIDIA A10G Tensor Core GPU with 24 GB memory, 80 RT cores, 320 tensor cores, and 250 TOPS. Stress testing led to the choice of batch size 48. Five hyperparameter combinations involving learning rate, weight decay, epochs, and GPU type were explored. The table extraction is partially corrupted, but the combinations clearly varied learning rates between 0.001 and 0.01, weight decay between 0.001 and 0.01, and used NVIDIA A10G or Tesla T4 GPUs. Evaluation loss curves over 30 epochs showed that Models 1, 3, and 4 reached a minimum early, around epochs 2–5, and then overfit, while Models 2 and 5 showed little improvement. The authors selected experiment 1 for final generation not purely because of loss, but because its generated sequences had the best statistical properties relative to known epitopes. The precise epoch count for the chosen checkpoint is not stated (Manrique et al., 3 Sep 2025).

4. De novo generation and statistical characterization

Sequence generation was performed by autoregressive sampling from the fine-tuned model. The final library contained 192,222 distinct synthetic epitope sequences. Two decoding hyperparameters were tuned: repetition penalty and temperature. Repetition penalty values $[1.2,2,3]$ were tested on about 100k generated sequences, with quality assessed via perplexity: $$\mathrm{PPL}(x) = \exp\left\{-\sum_{i=0}^{t} \log p_\theta(x_i \mid x_{<i})\right\}.$$ The paper notes lower perplexity as indicating that a generated sequence is more compatible with the model’s learned distribution.

For repetition penalty 1.2, the mean perplexity was 5414.087; for 2 it was 5207.757; and for 3 it was 5270.926. Mann–Whitney tests comparing 1.2 vs 2 and 1.2 vs 3 both gave $p = 8.7\times 10^{-52}$, rejecting the null of equal distributions, while comparing 2 vs 3 gave $p = 0.757$. The authors therefore chose repetition penalty 2. Small temperature experiments on 10k sequences with temperatures $[2,1,0.5]$ and repetition penalty 2 showed similar statistical properties; no formal statistical tests were reported, and temperature 1 was chosen.

The generated peptides were assessed primarily by comparing their statistical properties to those of known epitopes and to prior statistical analyses of IEDB epitopes. Generated sequences had a dominant size of 7–9 amino acids, with a sharp peak around 8–9 residues, consistent with the training distribution. Position-specific amino acid statistics were also analyzed. Relative entropy was written as

$20^n$0

comparing the amino acid distribution at a sequence position to a reference background from the human proteome. The paper reports especially strong positional bias at the final residue in 9-mers. Heatmaps of amino acid propensities showed aromatic residues enriched at final positions of short epitopes, which the authors interpret as potentially consistent with $20^n$1-stacking interactions in antibody recognition. They also observed low cysteine frequency across positions, which they suggest may reflect the undesirability of disulfide-forming residues in transient antibody-antigen binding.

Shannon entropy,

$20^n$2

was used to quantify positional diversity. In generated sequences, the final position tended to show relatively strong conservation patterns. The paper further discusses pairwise mutual information between positions and reports that, for lengths with enough generated sequences, mutual information was negligible. When generated sample counts were small for a given length, apparent mutual information increased, which the authors attribute to uncertainty in estimation from finite samples rather than genuine sequence dependency. The stated conclusion is that the generator captures realistic marginal and positional statistics without artificially coupling positions (Manrique et al., 3 Sep 2025).

The evidentiary basis for biological feasibility is therefore statistical resemblance rather than functional demonstration. The paper does not report explicit nearest-neighbor novelty analysis, overlap between generated and training sequences beyond the statement that 192,222 “distinct” synthetic sequences were produced, percent identity distributions to training examples, experimental validation of binding or immunogenicity, examples of individual generated sequences, or embedding-space comparison between generated and real epitopes beyond the dimensionality-reduction discussion for classification data.

5. Downstream classification and library enrichment

The classification stage is intended as a filtering mechanism for candidate libraries. Conceptually, the generator produces many epitope-like peptides, and the classifiers enrich for epitopes of a desired origin. Two classifier families were explored.

The first family was an ensemble model based on XGBoost built on top of sequence embeddings extracted from LLMs. Embeddings from ProtGPT2, ProtBERT, a baseline natural-language BERT, and the fine-tuned ProtGPT2 were compared. Fine-tuning ProtGPT2 for generation did not noticeably improve embedding quality for classification, which the paper characterizes as unsurprising because the generative fine-tuning was not optimized for classification. ProtGPT2 embeddings worked best for bacterial epitopes, whereas bare BERT embeddings unexpectedly outperformed others for viral epitopes. In practice, using only 100 embedding vector elements gave almost the same results as the full vector, and the authors constructed an “ensemble of ensembles” using these reduced inputs with biased majority voting. The bias favored recall: if an individual model predicts positive, its vote is weighted by a bias parameter, whereas a negative prediction gets weight 1.

The second family used fine-tuned LLMs directly, based on ProtBERT and ProtGPT2. These were trained on bacterial and viral IEDB subsets further split by assay type: T-cell, B-cell, MHC, and all assays combined. The paper reports F1, ROC AUC, PR AUC, and positive likelihood ratio $20^n$3, where

$20^n$4

This metric is central to the authors’ practical argument: after retaining only sequences classified as positive, the odds of true epitopes in the library are multiplied by $20^n$5.

The embedding-based ensemble classifiers performed modestly. For bacterial epitopes, the ensemble obtained F1 $20^n$6, accuracy $20^n$7, recall $20^n$8, precision $20^n$9, and $W=(W_1,\dots,W_n)$0. For viral epitopes, it achieved F1 $W=(W_1,\dots,W_n)$1, accuracy $W=(W_1,\dots,W_n)$2, recall $W=(W_1,\dots,W_n)$3, precision $W=(W_1,\dots,W_n)$4, and $W=(W_1,\dots,W_n)$5. These are high-recall but low-precision filters, so they preserve many positives but only slightly enrich the library.

The fine-tuned LLM classifiers produced stronger results, although performance varied substantially by organism and assay type.

Assay subset	Bacterial ProtBERT	Bacterial ProtGPT2
T cell	F1 0.49; ROC AUC 0.686; PR AUC 0.417; $W=(W_1,\dots,W_n)$6 16.247	F1 0.481; ROC AUC 0.734; PR AUC 0.453; $W=(W_1,\dots,W_n)$7 15.716
B cell	F1 0.519; ROC AUC 0.762; PR AUC 0.47; $W=(W_1,\dots,W_n)$8 3.658	F1 0.453; ROC AUC 0.71; PR AUC 0.465; $W=(W_1,\dots,W_n)$9 2.930
MHC	F1 0.869; ROC AUC 0.642; PR AUC 0.818; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$0 1.187	F1 0.865; ROC AUC 0.667; PR AUC 0.83; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$1 1.173
All	F1 0.5; ROC AUC 0.716; PR AUC 0.432; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$2 11.779	F1 0.502; ROC AUC 0.739; PR AUC 0.45; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$3 3.439

Assay subset	Viral ProtBERT	Viral ProtGPT2
T cell	F1 0.565; ROC AUC 0.676; PR AUC 0.537; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$4 1.841	F1 0.583; ROC AUC 0.711; PR AUC 0.595; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$5 1.667
B cell	F1 0.601; ROC AUC 0.918; PR AUC 0.524; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$6 6.697	F1 0.587; ROC AUC 0.916; PR AUC 0.546; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$7 6.949
MHC	F1 0.852; ROC AUC 0.719; PR AUC 0.812; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$8 1.465	F1 0.846; ROC AUC 0.767; PR AUC 0.872; $p(W) = \prod_{i=1}^{n-1} p(W_i \mid W_{<i}),$9 1.391
All	F1 0.607; ROC AUC 0.912; PR AUC 0.61; $$L_{\mathrm{CLM}} = - \sum_{k=1}^{R} \log p_\theta(w_k \mid w_{<k}).$$0 3.253	F1 0.58; ROC AUC 0.899; PR AUC 0.561; $$L_{\mathrm{CLM}} = - \sum_{k=1}^{R} \log p_\theta(w_k \mid w_{<k}).$$1 16.247

The paper states that models trained on MHC binding assay data “show the best performance,” and the highest F1 and PR AUC values are indeed in the MHC-trained models. At the same time, the largest $$L_{\mathrm{CLM}} = - \sum_{k=1}^{R} \log p_\theta(w_k \mid w_{<k}).$$2 values appear in some T-cell or “All” settings. Certain combinations pair high F1 with relatively modest ROC AUC. Since confusion matrices, calibration, confidence intervals, and threshold-selection procedures are not reported, these outcomes are difficult to interpret mechanistically. The central reported conclusion is that the LLM-based classifiers separate positive and negative examples better than the simple embedding-based ensembles and can in principle substantially enrich a candidate library (Manrique et al., 3 Sep 2025).

The authors also projected 1280-dimensional sequence embeddings using PCA and UMAP to check whether bacterial and viral examples form simple separable clusters. PCA and UMAP did not reveal meaningful clustering, which they interpret as evidence that the data are intrinsically complex despite the short sequence lengths.

6. Position within epitope-prediction research, limitations, and future directions

The paper’s main claimed novelty is the introduction of a directly generative linear-epitope model. Prior sequence-based epitope predictors typically score existing candidates; epiGPTope is designed to produce de novo epitope-like sequences and then prioritize them with classifiers. In that sense, it moves epitope machine learning from prediction toward generation.

This positioning is important because related work in epitope prediction often addresses a different problem. “Geometric Epitope and Paratope Prediction” studies joint prediction of antibody–antigen binding sites as a conformational, structure-based, residue-level task and reports that surface-based geometric models dramatically outperform residue-graph models for epitope prediction (Pegoraro et al., 2023). EpiFormer likewise addresses antibody-conditioned, structure-based, residue-level conformational epitope prediction, using interleaved cross-attention within GNN encoding layers and reporting strong gains over prior antibody-aware baselines (Ahmed et al., 2 Jun 2026). Relative to those systems, epiGPTope is deliberately narrower in biological scope and broader in generative intent: it is linear, sequence-only, and antigen-agnostic with respect to antibody partner, rather than a partner-specific structural interface predictor. This suggests that the framework occupies a different niche rather than serving as a direct alternative to geometric conformational epitope predictors.

The limitations are substantial and are discussed candidly in the paper. First, the approach is limited to linear epitopes; many biologically important epitopes are conformational and depend on three-dimensional structure. Second, there is no experimental validation of the generated sequences. The paper shows that they are statistically plausible, but not that they bind antibodies, trigger T-cell responses, or function in diagnostics or vaccines. Third, IEDB-derived positive and negative labels are explicitly treated as noisy. Many negatives may simply be peptides not recognized in a particular assay rather than universally true non-epitopes. Likewise, some learned features may reflect experimental selection bias rather than immunological principle; the discussion gives low cysteine frequency as an example that could arise because experimenters avoid cysteine-containing peptides during candidate selection. Fourth, because IEDB aggregates many independent experiments, common candidate-selection procedures cannot be reconstructed, so biological signal and curation bias cannot be fully disentangled. Fifth, sequence-only models may detect general “epitope-likeness” rather than antibody-specific compatibility. Sixth, many methodological details needed for reproducibility are absent, including exact train/test splits, dataset sizes per subset, exact negative-set construction, classifier head architectures, optimizer settings, and overlap or novelty analyses for generated sequences (Manrique et al., 3 Sep 2025).

Future work proposed in the paper includes compressing epiGPTope using quantum-inspired tensor networks, further fine-tuning ProtGPT2 toward additional tasks such as generating sequences tailored to specific antibody targets, and controlled mutation of known sequences to introduce diversity while preserving epitope-like character. The authors also argue that future models should integrate richer assay-derived and possibly structural data to move from generic epitope potential toward true binding specificity. A plausible implication is that epiGPTope represents an early sequence-only generative step in a broader trajectory toward multimodal epitope design systems, but the paper itself treats wet-lab screening as indispensable and presents the method as an aid rather than a replacement for experiment.