Transformer-Based Active Learning for Data-Efficient Vaccine Epitope Selection in PRRS

Abstract: High-fidelity molecular docking simulations can produce biologically relevant estimates of epitope-receptor binding affinity but are computationally expensive and therefore limit the number of candidates that can be screened for vaccine design. In this work, we evaluate ML approaches where variants of active learning are used to classify instances of high binding affinity between 9-mer epitopes and a well-conserved swine leukocyte antigen (SLA) receptor in the context of Porcine Reproductive and Respiratory Syndrome (PRRS). We use an internally generated dataset of 80 epitope-SLA docking affinities, each requiring more than 48 hours of high-performance computing (HPC). Multiple model families (linear, MLP, CNN, and a small transformer) are trained under strict low-data conditions within a pool-based active learning loop. In each case, optimal model configurations are identified by conducting large-scale hyperparameter optimization over the combined space of model architecture, training configuration, acquisition policy, and ensemble decision rules. To mitigate the effects of data subsample selection, each candidate configuration is evaluated by averaging performance over many randomized and balanced training and validation data subsets. Across experiments, transformer-based sequence models consistently emerged as the best-performing architecture, with active incremental learning yielding significant improvement over a baseline random sample acquisition strategy. Under moderate training data availability (N=30), the optimized ML-model configuration outperforms a standard baseline trained on twice the amount of data. Under higher training data availability (N=60), the same configuration achieves a peak accuracy of 86.8%, consistent with an upper bound of 85% classification accuracy based on two independent estimates of conformational noise.

• The paper demonstrates that transformer-based active learning using EGL and deep ensembles significantly improves data efficiency in classifying PRRS vaccine epitopes.
• It shows that leveraging affinity regression targets and adaptive embedding sizes optimizes classification performance under severe label scarcity.
• The approach achieves near-theoretical accuracy limits for epitope selection while dramatically reducing labeling effort and costly docking simulations.

Transformer-Based Active Learning for Data-Efficient Vaccine Epitope Selection in PRRS

Problem Setting and Motivation

The computational bottleneck in high-fidelity molecular docking for epitope-receptor affinity estimation imposes substantial constraints on large-scale vaccine antigen screening, particularly for Porcine Reproductive and Respiratory Syndrome (PRRS). Given the extreme computational cost (≈48 hours per label) and the scarcity of labeled data, conventional supervised learning approaches are infeasible. This necessitates methodologies tailored for the low-data regime and capable of maximizing information extracted from each exemplar. The authors frame the solution within a pool-based active learning paradigm, targeting binary classification of 9-mer peptide epitopes as strong or weak SLA binders, based on internally generated precise docking simulations.

Methodological Framework

The approach integrates several advanced machine learning strategies:

• Data Processing and Labeling: Each peptide-receptor sample is assigned a binary class using Otsu thresholding on the continuous affinity values, strictly balanced to maintain class equilibrium and minimize information leakage by training-set-based normalization.
• Feature Representation: 9-mer amino acid sequences are one-hot encoded into (9,20) arrays. Transformer models learn position-wise token embeddings; CNNs, MLPs, and linear models consume flattened or convolved features.
• Model Families and Search: Four model families are compared: Linear, MLP, CNN, and Transformer. Each undergoes large-scale, architecture-aware hyperparameter optimization.
• Active Learning Loop: Initial seed samples are iteratively augmented by informative acquisitions from the pool. Policies evaluated include Random, Least Confident, K-Center, and Expected Gradient Length (EGL).
• Training Targets: Both discrete class labels and normalized affinity proxies serve as supervision signals, interrogating whether regression-based targets improve generalization under data scarcity.
• Ensembling: All high-performing configurations incorporate deep ensembles (13-16 members), employing various aggregation strategies (majority vote, mean-round, confidence-weighted).
• Robust Evaluation: Each hyperparameter configuration is evaluated across 40 randomized data splits and initializations, totaling 1000 iterations, with Bayesian optimization via TPE guiding global search.

Key Results

Performance Across Data Regimes

• Transformer Dominance: Across all regimes, transformer-based models consistently surpass CNN and MLP baselines. This is especially pronounced as data availability increases, reflecting the transformer's capacity to capture long-range token dependencies critical for peptide binding representation.
• Active Learning Gain: EGL-based acquisition systematically improves sample efficiency over random, least confident, and k-center policies when moderate to large training budgets (N=60) are available. With sparse data (N≤30), nearly all samples are required up front, reducing the utility of incremental selection.
• Numerical Efficiency: With only 37.5% of the labeled dataset (N=30), the optimized pipeline achieves 80.5% classification accuracy, exceeding a standard transformer baseline trained on double the data (78.0%). With 75% of the data (N=60), optimized accuracy reaches 86.8%.
• Noise Ceiling and Upper Bound: Detailed conformational variability analysis, via replicate docking with AlphaFold2-generated structures, establishes a practical accuracy ceiling of ≈85%. The optimized models reach and slightly exceed this limit, indicating near-maximal extraction of usable signal.

Architecture and Optimization Insights

• Ensembling Robustness: Ensembles consistently emerge as optimal regardless of data regime, highlighting their stabilizing effect on low-data uncertainty.
• Affinity Target Superiority: Training with smooth, normalized affinity proxies (regression) yields systematically higher discrete class accuracy versus direct class label supervision.
• Adaptive Embedding Dimensionality: Counterintuitively, sparser regimes drive the optimizer toward larger embedding sizes (d_model up to 256), leveraging overparameterization to increase sample separability; as data grows, embeddings contract (down to 48).
• Nontrivial Activation Choices: LeakyReLU activation is favored under extreme data scarcity, switching to GELU as more supervision is available. The latter's nonlinearity becomes valuable only when supported by sufficient data.
• Optimization Trajectories: Transformers and CNNs are competitive early on, but transformers universally converge to higher performance with increased optimization trials and data.

Comparative and Baseline Analysis

A rigorous "Standard Benchmark"—a highly tuned, single transformer model without ensembles or active learning—serves as a conservative reference. The full optimization pipeline yields a consistent delta (up to +8.8% at N=60), underlining the compounded advantage from meta-optimization, incremental information gain, and ensemble stabilization.

Theoretical and Practical Implications

The results demonstrate that transformer-based architectures, when coupled with optimized active learning and ensembling, achieve near-theoretical optimality in the practical scenario of limited docking data. The consistent advantage of affinity regression targets over discrete classes indicates the importance of preserving intrinsic data granularity for robust gradient propagation, especially under sample-sparse conditions. Active learning based on expected gradient length provides a principled and effective mechanism for prioritizing highly informative exemplars.

The practical implication is substantial: effective vaccine epitope prioritization can be realized with dramatically reduced labeled data and computational cost, thus enabling rapid response to emergent pathogenic threats with limited biosimulation budgets. The generality of the data pipeline, optimization regime, and acquisition framework makes the approach broadly applicable to other peptide-screening and protein-interaction domains where labeled data acquisition is expensive or slow.

Future Directions

While the transformer-based models in this work are randomly initialized, recent literature suggests further gains could accrue from pretraining on large corpora of protein–protein interactions ("A paired sequence LLM for protein-protein interaction modeling" [Liu et al., 2026]). Exploration of hybrid approaches combining pretraining with task-specific fine-tuning and active acquisition policies is warranted. More sophisticated uncertainty quantification and adaptive curriculum strategies could further refine sample efficiency and generalization.

The demonstrated inverse relationship between embedding dimensionality and data volume also invites a thorough investigation into the interplay between model capacity, sample entropy, and label granularity, with likely implications for transfer learning protocols in other structural bioinformatics applications.

Conclusion

This study establishes that active learning with transformer-based sequence models—augmented by ensemble methods and affinity-proxy supervision—can saturate the informativeness of limited high-fidelity docking datasets for PRRS vaccine epitope selection. The approach outperforms traditionally optimized baselines by wide margins in data efficiency, achieving accuracy at or beyond biologically determined noise ceilings. The methodology has immediate practical relevance in rapid-response vaccine design pipelines and theoretical ramifications for low-data deep learning in high-value experimental regimes.