SpatialPPIv2: GNN-Based Protein Interaction Predictor
- The paper demonstrates that SpatialPPIv2, a structure-based graph neural network, fails to reliably recover known viral–host interactions, missing about 40% of them.
- SpatialPPIv2 leverages 3D structures from AlphaFold 3 with a binary threshold rule, achieving approximately 60% recall which compromises its effectiveness as a biosecurity filter.
- Despite potential advantages in detecting distant homologs, the model is blind to function-preserving mutations, highlighting critical limitations for use in biosecurity screening.
Searching arXiv for the cited paper and related work on SpatialPPIv2. SpatialPPIv2 is a graph neural network-based protein–protein interaction (PPI) predictor evaluated in the biosecurity study "Resilient Biosecurity in the Era of AI-Enabled Bioweapons" (Feldman et al., 30 Aug 2025). In that study, it was assessed as one of three leading PPI prediction tools, alongside AlphaFold 3 and AF3Complex, on a benchmark of well-characterized viral protein–human receptor interactions and on experimentally characterized SARS-CoV-2 receptor-binding domain (RBD) mutants. The reported results place SpatialPPIv2 within a broader class of inference-time biosafety filters that seek to identify dangerous molecular outputs computationally. The study’s central finding is that such filters, including SpatialPPIv2, fail to reliably recover even known viral–host interactions, with direct implications for the use of PPI prediction as a standalone biosecurity control (Feldman et al., 30 Aug 2025).
1. Position within computational biosecurity screening
The study situates SpatialPPIv2 within existing biosafety measures that "primarily rely on inference-time filters such as sequence alignment and protein-protein interaction (PPI) prediction to detect dangerous outputs" (Feldman et al., 30 Aug 2025). In that framing, SpatialPPIv2 functions as a structure-based screening model intended to estimate whether two proteins interact and thereby support triage of potentially hazardous designs.
Within the comparison presented in the study, SpatialPPIv2 is the benchmarked model "designed to predict whether two protein structures interact." Its method is described as graph neural network-based, and its inputs are "pre-computed 3D structures (from AlphaFold 3)." This distinguishes it from AlphaFold 3, described as "structure-based, but sequence-input-centric," and from AF3Complex, described as "an AlphaFold 3-based method specifically optimized to improve structure-based PPI interface predictions" (Feldman et al., 30 Aug 2025).
This positioning is significant because the study treats PPI prediction not as a generic structural-biology benchmark, but as a safety-critical filter. Under that interpretation, false negatives are more consequential than ordinary benchmark errors: a missed interaction may correspond to failure to flag a pathogenic or otherwise dangerous molecular design. The paper therefore emphasizes recall on known positive viral–host interactions rather than broader balanced classification metrics (Feldman et al., 30 Aug 2025).
2. Model characterization and decision rule
SpatialPPIv2 is characterized in the study as a "structure-based graph neural network" model. The summary table reconstructing the paper identifies its input as "Structure (from AlphaFold 3)" and its method as "Graph neural network" (Feldman et al., 30 Aug 2025). The accompanying references mention Hu and Ohue’s 2025 work, "SpatialPPIv2: Enhancing protein–protein interaction prediction through graph neural networks with protein LLMs," indicating the model’s broader methodological context, although the biosecurity paper itself evaluates the predictor operationally rather than deriving its architecture in detail.
The study defines the prediction task in thresholded form. For each viral–host pair, the model returns a confidence score or probability of interaction, and a positive call is made when the probability is at least 0.5. The explicit binary decision rule is given as
where is the predicted probability of interaction (Feldman et al., 30 Aug 2025).
This operationalization matters because the paper’s conclusions about failure are threshold-relative: the model may assign nonzero scores to true binders, but still fail as a detector if those scores remain below the positive cutoff. The SARS-CoV-2 mutant analysis is especially important in this respect, because all reported SpatialPPIv2 scores remained below 0.5 even when the mutants had experimentally increased ACE2 binding affinity (Feldman et al., 30 Aug 2025).
3. Benchmark design and evaluated interaction set
The study constructed "a benchmarking set of thirteen well-characterized viral protein–human receptor interactions" (Feldman et al., 30 Aug 2025). These interactions were selected from cases with "available high-resolution structures in the Protein Data Bank (PDB), ensuring ground truth for both sequence and structure." The benchmark included HIV, Measles, MERS, Ebola, Adenovirus, Epstein-Barr, HSV with multiple host receptors, Hepatitis B, Nipah, Chikungunya, and SARS-CoV-2. Example PDB identifiers listed in the study include SARS-CoV-2 (6M0J), Hepatitis B (8HRX), HIV (1GC1), and MERS (4L72) (Feldman et al., 30 Aug 2025).
In addition to this positive interaction set, the authors evaluated "four mutated variants of the SARS-CoV-2 spike receptor-binding domain (RBD)," drawn from variants with "experimentally validated changes in ACE2 binding affinity" from Moulana et al. (2022). Two mutants had increased affinity and two had decreased but still functional binding (Feldman et al., 30 Aug 2025).
The benchmark design is notable for focusing on well-characterized viral–host binding events rather than de novo interaction discovery. This suggests that the study is testing whether current models can at least recover canonical, biologically validated interactions before considering extrapolation to novel or engineered threats. A plausible implication is that failure on this benchmark places a strong upper bound on confidence in these systems for prospective biosecurity screening.
4. Comparative performance on known viral–host PPIs
The principal result is reported as binary predictive accuracy on the positive set: whether each model returned a score above its designated threshold for each known interaction. The study states that this quantity is "the percentage of known PPIs correctly marked as positive ('recall' for this positive set)" (Feldman et al., 30 Aug 2025).
The reconstructed Table 1 reports the following interaction-level outcomes:
| Virus | SpatialPPIv2 | AlphaFold3 | AF3Complex |
|---|---|---|---|
| Adenovirus | ✔ | ✔ | ✔ |
| Chikungunya | ✗ | ✗ | ✗ |
| Ebola | ✔ | ✗ | ✔ |
| Epstein-Barr | ✔ | ✔ | ✔ |
| Hepatitis B | ✗ | ✗ | ✗ |
| HIV | ✔ | ✔ | ✔ |
| HSV (HVEM Receptor) | ✔ | ✗ | ✗ |
| HSV (Nectin-1 Receptor) | ✔ | ✔ | ✔ |
| Measles | ✗ | ✗ | ✗ |
| Measles (Edmonston strain) | ✗ | ✔ | ✔ |
| MERS | ✔ | ✔ | ✔ |
| Nipah | ✔ | ✔ | ✔ |
| SARS-CoV-2 | ✗ | ✗ | ✔ |
At the model-wide level, SpatialPPIv2 is reported to have "missed approximately 40%" of established viral–host interactions, corresponding to "recall ≈ 60%." AlphaFold 3 "missed about 50%" and AF3Complex "missed about 30%," corresponding to recalls of approximately 50% and 70%, respectively (Feldman et al., 30 Aug 2025).
The study highlights a particularly consequential result: "both AlphaFold 3 and SpatialPPIv2 failed to detect the SARS-CoV-2/ACE2 interaction," while AF3Complex succeeded in that single case (Feldman et al., 30 Aug 2025). SpatialPPIv2 also failed on Hepatitis B, Chikungunya, Measles, and Measles (Edmonston strain), but outperformed AlphaFold 3 on Ebola and HSV (HVEM receptor) in the reported table.
In context, these results are interpreted not as marginal benchmark variance but as evidence that even state-of-the-art or near-state-of-the-art structural PPI predictors are unreliable on viral–host cases relevant to safety screening. The study’s comparative framing is especially important: SpatialPPIv2 is not singled out as uniquely deficient, but rather as one instance of a broader methodological shortfall affecting all three tested systems (Feldman et al., 30 Aug 2025).
5. SARS-CoV-2 mutant evaluation and score behavior
The mutant analysis is the most stringent failure case reported for SpatialPPIv2. The study evaluated four SARS-CoV-2 spike RBD variants with experimentally validated shifts in ACE2 affinity, including two variants with increased affinity and two with lower but functional binding (Feldman et al., 30 Aug 2025). The reconstructed score table is:
| Affinity Change () | SpatialPPIv2 | AlphaFold3 | AF3Complex |
|---|---|---|---|
| –1.04 (↑ affinity) | 0.27 | 0.16 | 0.11 |
| +2.14 (↓, functional) | 0.19 | 0.24 | 0.06 |
| –1.03 (↑ affinity) | 0.26 | 0.14 | 0.06 |
| +2.11 (↓, functional) | 0.24 | 0.15 | 0.05 |
The study also reports score deltas relative to baseline. For SpatialPPIv2 these were +0.08, +0.00, +0.07, and +0.05, respectively; for AlphaFold 3, +0.03, +0.11, +0.01, and +0.02; and for AF3Complex, –0.47, –0.52, –0.52, and –0.53 (Feldman et al., 30 Aug 2025).
The central interpretation is explicit: "All values for SpatialPPIv2 (threshold = 0.5) are well below the cutoff for a positive prediction—even for mutants that have an experimentally increased binding affinity." More generally, "No model correctly detected binding for any of the functionally tested SARS-CoV-2 RBD/ACE2 mutant pairs" (Feldman et al., 30 Aug 2025).
The study further notes that confidence scores "drifted unpredictably," including cases in which model confidence decreased for higher-affinity variants or increased for lower-affinity but functional variants. This is described as suggesting "a non-principled relationship to true affinity or interaction biology" (Feldman et al., 30 Aug 2025). For SpatialPPIv2, this means the failure is not merely inability to rank exact affinities; it extends to inability to cross even a binary detection threshold for experimentally confirmed binders.
6. Reported strengths, weaknesses, and proposed causes of failure
The paper does attribute a limited strength to SpatialPPIv2. Because it uses "structure-based graph neural networks," it "in principle, could detect PPIs even among distant homologs or engineered proteins that elude sequence similarity detection" (Feldman et al., 30 Aug 2025). The benchmark results also show instances where it outperforms AlphaFold 3, specifically on Ebola and HSV/HVEM.
The weaknesses, however, are emphasized more strongly. The study identifies a "High false negative rate," stating that missing roughly 40% of "well-characterized, often training-set-present viral PPIs is a critical failure for biosecurity." It further describes the model as "Blind to function-preserving or improved mutants," with failure to detect "functionally superior-binding viral mutants" taken as evidence of insensitivity to "subtle, combinatorial or epistatic sequence changes." Additional reported weaknesses include poor generalization, inability to handle similar proteins or small mutational changes, possible limitations beyond training-data coverage, and "Erratic confidence" under mutational perturbation (Feldman et al., 30 Aug 2025).
The paper proposes several reasons for these failures:
- Data Deficiency: "Training data for viral–host PPIs is sparse: the PDB is dominated by non-viral proteins, so the model lacks exposure to features unique to viral-host recognition."
- Biophysical Complexity: "Viral proteins often use unique or disordered regions and can tolerate extensive mutations; interaction physics may not resemble those of canonical eukaryotic PPIs."
- Synthetic/Mutant Case Blindness: "Structure-based neural nets may miss non-canonical, epistatic, or allosteric effects underlying real functional change."
- Threshold issues: "Even for inputs structurally resembling known binders, predicted scores remained sub-threshold for positive detection" (Feldman et al., 30 Aug 2025).
These explanations remain partly interpretive, but they are presented in the paper as the most plausible account of why a structure-based model can fail even when sequence and structural information are available. The study explicitly states that the failure to detect functional SARS-CoV-2 RBD/ACE2 mutants was "not due to lack of sequence or structure information, but because underlying features controlling binding were not 'learned' or correctly represented" (Feldman et al., 30 Aug 2025).
7. Biosecurity significance and limitations as a safeguard
The biosecurity conclusion is unambiguous. The study states that SpatialPPIv2, "despite being among the most advanced structure-based PPI predictors, is not sufficiently reliable to be used as a biosecurity safeguard" (Feldman et al., 30 Aug 2025). This conclusion is grounded in several observations reported directly in the paper: it misses known threats, is "completely blind to viral mutants," exhibits a high risk of false negatives, and may create a "False sense of security" if deployed as a primary computational filter.
The paper emphasizes that "current predictive filters are inadequate for reliably flagging even known biological threats and are even more unlikely to detect novel ones" (Feldman et al., 30 Aug 2025). It further concludes that "current machine learning approaches to PPI prediction lack the accuracy to serve as a standalone biosecurity control" (Feldman et al., 30 Aug 2025). In this framing, the main issue is not simply benchmark imperfection but the asymmetry between the operational requirement of near-zero miss rate for high-consequence threats and the much weaker empirical performance achieved by available models.
The resulting policy implication is a proposed "shift toward response-oriented infrastructure," including "rapid experimental validation, adaptable biomanufacturing, and regulatory frameworks capable of operating at the speed of AI-driven developments" (Feldman et al., 30 Aug 2025). The paper specifically argues for moving "away from preemptive filtering" and toward "response-oriented biosecurity infrastructure," emphasizing "high-throughput experimental validation and rapid countermeasure deployment" rather than reliance on computational PPI prediction (Feldman et al., 30 Aug 2025).
A common misconception addressed implicitly by the study is that increasingly sophisticated structural models can be assumed to generalize to dangerous or engineered viral cases simply because they perform well on some canonical protein-structure tasks. The reported benchmark contradicts that assumption. A plausible implication is that structural sophistication alone does not guarantee safety-screening adequacy when the target domain is sparse, mutation-sensitive, and dominated by viral–host recognition mechanisms that differ from canonical training distributions.