NanoProFormer: Multimodal Protein Corona Model

• NanoProFormer is a multimodal foundation model integrating protein sequence embeddings with text-encoded descriptors to predict nanomaterial–protein affinity and relative protein abundance.
• It overcomes challenges from small, fragmented datasets and missing experimental features by training on a merged raw and imputed dataset of millions of samples.
• Employing a cross-modal fusion architecture with zero-shot inference and fine-tuning, it supports applications in biosensing, nano-bio interface engineering, and biomarker discovery.

Searching arXiv for the cited NanoProFormer paper and closely related protein-corona modeling work to ground the article in current literature. NanoProFormer is a multimodal foundation model for predicting nanomaterial–protein affinity / relative protein abundance (RPA) in the protein corona problem. Developed together with NanoPro-3M, it is designed to address two bottlenecks that have limited prior machine-learning approaches in this area: small, fragmented datasets and poor generalization to unseen nanomaterials, unseen proteins, and missing experimental features. Its central strategy is to combine protein sequence embeddings with text-encoded structured tabular descriptors in a cross-modal architecture, enabling both affinity classification and RPA regression while supporting zero-shot inference and downstream fine-tuning (Yu et al., 18 Jul 2025).

1. Scope and nomenclature

In the literature considered here, NanoProFormer most specifically denotes the multimodal foundation model introduced with NanoPro-3M for nanomaterial–protein interaction prediction (Yu et al., 18 Jul 2025). The model is framed as a reusable, generalizable system for the protein corona domain rather than a task-specific predictor tied to a single assay or material class.

The label should be distinguished from two separate usages in other contexts. ProFormer is an on-device LSH projection based transformer for text classification that reduces embedding memory from $O(V \cdot d)$ to $O(T)$ and compresses attention through a Local Projection Attention layer; despite the nominal similarity, it addresses mobile NLP rather than nanobio interaction modeling (Sankar et al., 2020). A different paper on SERS sensing of proline and hydroxyproline uses “NanoProFormer” only as a likely conceptual name for a gold nanopillars + NTA-Ni affinity capture + CNN classification framework; that usage is descriptive rather than the formal name of the published method (Zhang et al., 2024).

Within nanomaterial–protein modeling, NanoProFormer is explicitly tied to the need for an “ImageNet-like” benchmark: a large, standardized dataset and a model that can learn broad patterns instead of narrow study-specific correlations (Yu et al., 18 Jul 2025).

2. Problem setting and NanoPro-3M dataset

NanoProFormer targets the prediction of nanomaterial–protein affinity / relative protein abundance in a decision space governed by nanomaterial physicochemical properties, incubation environment, separation / isolation protocol, proteomic profiling depth, and protein identities (Yu et al., 18 Jul 2025). The associated dataset, NanoPro-3M, is described as the largest nanomaterial-protein interaction dataset to date, comprising over 3.2 million samples and 37,000 unique proteins, curated from over 2,500 papers. A second external dataset was built for evaluation and generalization, with 0.46 million samples and 10.4k proteins, including >3.6k proteins unseen in the original dataset (Yu et al., 18 Jul 2025).

As of September 20, 2024, the data collection pipeline searched Web of Science, PubMed, and Scopus, focusing on original research articles. The extraction of structured experimental descriptors used LLMs plus manual verification. The final curation included 29 features distributed across nanomaterial properties, incubation conditions, separation parameters, and proteomic setting (Yu et al., 18 Jul 2025).

Feature group	Count	Examples stated in the paper
Nanomaterial properties	14	core, surface modification, shape, primary size, zeta potential
Incubation conditions	9	protein source, medium, temperature, time, flow condition
Separation parameters	5	separation method, centrifugation speed, centrifugation time
Proteomic setting	1	proteomic depth

The dataset is described as highly heterogeneous, with 100+ categories for core and protein source and 200+ categories for surface modifications (Yu et al., 18 Jul 2025). Because reported RPA values differ across studies, the paper reconstructs them using available quantification methods such as normalization-based methods, molecular-weight normalization, iBAQ, spectral count, intensity, peptide number, molar mass, and emPAI.

Two fill strategies are central to the construction. In local fill, proteins missing from a group within studies with multiple experimental groups are assigned RPA = 0, yielding 1.19 million samples. In global fill, proteins commonly observed across studies but not reported in a specific study are also assigned RPA = 0, adding another 190,000 samples. Because all included samples had at least one missing numerical feature, the authors applied imputation methods including weighted mean, mode, LLM + retrieval-augmented generation (RAG), example-based LLM inference, and reasoning LLMs, producing a Merged dataset of 2.76M samples and a Filled-only dataset of 1.38M samples (Yu et al., 18 Jul 2025).

3. Model architecture and input representations

NanoProFormer is organized into three stages: embedding acquisition, cross-modal fusion, and prediction head (Yu et al., 18 Jul 2025). It uses two modalities.

Modality	Encoder	Output embedding
Protein amino-acid sequence	ESM2 t36 3B_UR50D	2560-dimensional
Text-encoded structured descriptors	Linq-Embed-Mistral	4096-dimensional

The text prompt is not a simple serialization of fields. It contains task description, background context, and sample-specific values. The two modality-specific embeddings are projected into a shared latent space of dimension 1024 using separate projection modules. They are then fused with a multi-head cross-attention mechanism, allowing one modality to query the other rather than treating them as independent feature blocks (Yu et al., 18 Jul 2025).

The fused representation is passed to an MLP prediction head. The paper states that classification and regression foundation models were built separately. The classification model predicts affinity versus non-affinity, whereas the regression model predicts continuous RPA values (Yu et al., 18 Jul 2025).

This architecture reflects a specific modeling claim in the paper: affinity prediction depends jointly on sequence-derived protein properties and experimental / nanomaterial context, and multimodal interaction modeling is therefore preferable to single-modality inference (Yu et al., 18 Jul 2025).

4. Objectives, splits, and robustness to missingness

The training data use an 8:1:1 split ratio. The filled version and its corresponding unfilled counterpart are kept in the same split, and train/validation are stratified by binned RPA. For affinity classification, the threshold is

$\text{RPA} > 0.001\% \Rightarrow \text{positive}$

and otherwise negative. For regression, only affinity samples are used (Yu et al., 18 Jul 2025).

The classification task is imbalanced. To counter this, the paper states that the positive-class weight is twice the ratio of negative to positive instances. For regression, because RPA is highly skewed and long-tailed, the target is transformed using Box-Cox transformation before prediction (Yu et al., 18 Jul 2025). Evaluation uses Accuracy, Precision, Recall, F1 score, and AUC for classification, and $R^2$, MAE, and MSE for regression.

A central claim of NanoProFormer is that robustness requires explicit exposure to missingness patterns. The merged dataset contains both raw and filled samples, enabling the model to learn from incomplete observations rather than overfitting to fully imputed data. The paper argues that models trained only on filled data degrade sharply on raw data, whereas mixed raw-and-filled training improves real-world usability (Yu et al., 18 Jul 2025).

Generalization to unseen inputs is handled by modality design. Unseen categorical values are represented through text prompts and embedded by a pretrained LLM, supporting transfer to unseen core categories, unseen surface modifications, and unseen incubation protein sources. Unseen proteins are addressed through ESM2 sequence embeddings, so the model is not restricted to memorized protein identifiers (Yu et al., 18 Jul 2025).

For interpretability, the paper defines an ablation-based importance score by masking a feature as “Unknown” and measuring the performance drop: F1 drop for classification and $R^2$ drop for regression. Feature-pair interaction is evaluated by masking two features together and comparing the resulting loss with the sum of individual losses, distinguishing synergy from redundancy (Yu et al., 18 Jul 2025).

5. Empirical performance and generalization behavior

The paper reports a marked contrast between models trained on filled-only data and models trained on the merged dataset (Yu et al., 18 Jul 2025). On filled test data, filled-only training performs strongly, with Accuracy: 0.89, F1: 0.84, AUC: 0.96, $R^2$: 0.87, and MAE: 0.52. On raw test data, however, the same regime drops to Accuracy: 0.66, F1: 0.61, AUC: 0.79, $R^2$: 0.45, and MAE: 1.21.

By contrast, models trained on the merged dataset remain strong on both filled and raw test sets: Accuracy: 0.90 on filled, 0.89 on raw; F1: 0.84 on both; AUC: 0.96 on both; and $R^2$: 0.88 on both filled and raw (Yu et al., 18 Jul 2025). This is the paper’s principal empirical evidence that training on a mixture of raw and imputed data improves robustness to incomplete experimental records.

Ablation studies show that protein-only and tabular-only models have comparable performance, with a slight advantage for protein-only, but multimodal fusion performs substantially better than either alone (Yu et al., 18 Jul 2025). The result is used to support the claim that nanomaterial–protein interaction prediction is jointly determined by molecular identity and contextual metadata.

Generalization is tested on an external unseen dataset collected through a new literature search on April 27, 2025, excluding earlier papers. This set contains 15 independent studies, including 13 with previously unseen proteins, 5 with unseen nanomaterial core compositions, 3 with unseen surface modifications, and 7 with unseen incubation protein sources; importantly, no imputation was applied to this dataset (Yu et al., 18 Jul 2025).

On this external unseen dataset, the model achieves average classification performance above 0.7 across accuracy, precision, recall, and F1. For negative samples, defined by $\text{RPA} < 0.001\%$, the reported accuracy is around 0.69. For positive samples, predicted probability and accuracy improve as RPA increases. Regression transfer is substantially weaker, with $R^2 < 0$, which the paper attributes to cross-study heterogeneity, different isolation protocols, different proteomics platforms, and different quantification methods (Yu et al., 18 Jul 2025).

The paper further states that the model still achieves around or above 0.7 across metrics on unseen proteins, unseen core categories, unseen surface modifications, and unseen incubation protein sources, indicating that its generalization is not limited to a single type of novelty (Yu et al., 18 Jul 2025).

6. Interpretability, downstream tasks, and limitations

The interpretability analysis identifies nanomaterial core composition as the most influential feature, which the paper notes is consistent with known chemistry because core material strongly affects protein adsorption (Yu et al., 18 Jul 2025). Among the top classification features, many are tied to surface chemistry, including core composition, zeta potential, and PdI. Additional important features include separation method, proteomic depth, nanomaterial concentration, and incubation protein source.

The paper provides mechanistic interpretations for these rankings. Separation method can alter apparent corona composition; proteomic depth affects detection of low-abundance proteins; nanomaterial concentration changes available surface area; and incubation protein source determines the protein pool available for adsorption (Yu et al., 18 Jul 2025). It also reports a contrast between tasks: classification can rely more on coarse-grained features, whereas regression requires finer detail. Correspondingly, proteomic depth ranks higher in regression than in classification. The pairwise ablation study further shows that classification: redundancy dominates and regression: synergy dominates, suggesting that continuous-abundance prediction depends more strongly on coordinated contextual information.

NanoProFormer is presented as a foundation model rather than only a benchmark predictor. In zero-shot inference, it can score new samples without task-specific retraining and is used for screening, predicting affinity labels, and ranking proteins for novel nanomaterials (Yu et al., 18 Jul 2025). In fine-tuning, only the prediction head is adapted while the projection layers and cross-modal fusion module remain frozen. Four case studies are reported: antibody binding, cell receptor, disease biomarker, and in-depth proteomic. All four show substantial improvement, with AUC improvements from 9.27% to 50.04%, average AUC improvement: 33.18%, and average $O(T)$0 after fine-tuning: 0.71 (Yu et al., 18 Jul 2025). The most difficult case is cell receptor, likely because many proteins were unseen and receptor interactions are more biologically heterogeneous. The in-depth proteomic case benefits strongly, which the authors interpret as evidence that the model should improve further as proteomics becomes more comprehensive.

The principal limitation is that continuous-abundance regression generalizes poorly across heterogeneous external studies, even when binary affinity classification transfers reasonably well (Yu et al., 18 Jul 2025). A plausible implication is that binary affinity prediction is currently the more stable cross-study endpoint for large-scale deployment, whereas precise RPA estimation remains constrained by unresolved protocol variation. The paper’s broader conclusion is that NanoPro-3M functions as an “ImageNet for protein corona research” and that NanoProFormer provides a generalizable basis for reducing experimental reliance, accelerating in vitro nanomaterial studies, and supporting downstream applications in disease biomarker discovery, biosensor development, nano-bio interface engineering, environmental nanoscience, and biomedical nanotechnology (Yu et al., 18 Jul 2025).