DAVIS-Complete: Modification-Aware Kinase Benchmark

• DAVIS-Complete is a modification-aware extension of the DAVIS dataset that restores and explicitly represents modified kinase entries to capture biologically relevant changes in ligand affinity.
• The benchmark is designed with multiple settings—augmented dataset prediction, wild-type to modification generalization, and few-shot modification generalization—to rigorously assess model performance.
• Empirical results reveal that while docking-based methods generally outperform docking-free models, accurate modeling of modification-induced affinity shifts remains a critical challenge.

Searching arXiv for DAVIS-Complete and related DAVIS resources to ground the article in the relevant papers. Searching arXiv for the DAVIS-Complete protein–ligand benchmark paper. DAVIS-Complete is a curated, modification-aware extension of the widely used DAVIS kinase–inhibitor affinity dataset for protein–ligand binding prediction. It is designed to make benchmarking closer to biologically relevant kinase pharmacology by explicitly representing modified proteins rather than excluding them or collapsing them into wild-type entries. In the paper that introduces it, “complete” refers to restoring and explicitly representing modified kinase entries that were ignored or simplified in common “DAVIS-filtered” usage, while “modification-aware” refers to preserving the distinction between wild type and specific modified variants and to constructing benchmark settings that directly test whether models detect modification-induced affinity changes (Wu et al., 30 Nov 2025).

1. Definition and scientific motivation

DAVIS-Complete is situated in kinase–ligand affinity prediction, where the original DAVIS dataset is attractive because it reports $K_d$ measurements for kinase–ligand pairs generated in a single assay framework, thereby reducing assay heterogeneity. The original DAVIS contains 442 kinase proteins and 72 kinase inhibitors, giving $442 \times 72 = 31{,}824$ affinity measurements. The central critique motivating DAVIS-Complete is that prior deep learning work typically either excluded modified kinases or treated modified proteins as identical to their wild-type forms, which the authors argue wastes biologically valuable information and can induce a misleading kind of overfitting (Wu et al., 30 Nov 2025).

The biological rationale is kinase-specific. Single substitutions, insertions or deletions, domain-specific constructs, and phosphorylation events can materially change inhibitor binding. The paper gives two explicit examples. EGFR T790M reduces Lapatinib binding by roughly 360-fold relative to wild type, and phosphorylation-dependent kinase conformations influence type II inhibitor binding, such as stronger Imatinib binding to inactive ABL1 than to its active state (Wu et al., 30 Nov 2025).

In this sense, DAVIS-Complete is not presented as a larger assay campaign. It is presented as a more faithful benchmark for whether a model can generalize from wild-type proteins to biologically meaningful variants, which is the regime most relevant to mutant-selective drug discovery, mutation-specific resistance analysis, and precision-medicine-style prioritization (Wu et al., 30 Nov 2025).

2. Dataset construction and representation of modified proteins

The construction procedure begins from the DAVIS kinase panel. Entrez Gene Symbols in DAVIS were mapped to UniProt IDs, and the corresponding amino acid sequences were retrieved from UniProt. The authors then manually curated modified amino acid sequences for kinase variants based on the modification annotations available in DAVIS and related references. The curated modifications include substitutions, insertions, deletions, phosphorylation events, and combinations of these; the paper also reports corrections of previously oversimplified protein representations by replacing full-length entries with specific kinase domain constructs where appropriate (Wu et al., 30 Nov 2025).

The curation adds 56 modified amino acid sequences for 11 kinase proteins. Because each curated sequence is paired with the full ligand panel, this produces exactly $56 \times 72 = 4{,}032$ modified protein–ligand pairs. The appendix overlap analysis refers to 56 modified proteins among a total of 444 proteins in DAVIS-Complete. At the same time, the paper also states that DAVIS remains at 31,824 entries because the original assay matrix already has $442 \times 72$ measurements; the contribution of DAVIS-Complete is therefore not new wet-lab measurements, but recovery and proper annotation of the complete protein set within the existing assay matrix (Wu et al., 30 Nov 2025).

Quantity	Value	Note
Original DAVIS proteins	442	Kinase entries
Ligands	72	DAVIS panel
Original assay matrix	31,824	$442 \times 72$
Curated modified sequences	56	For 11 proteins
Modified protein–ligand pairs	4,032	$56 \times 72$
Appendix total in DAVIS-Complete	444	Revised protein representations

The explicit modified proteins listed in the paper include ABL1 variants such as E255K, F317I, F317L, H396P, M351T, Q252H, T315I, and Y253F, in some cases also with Tyr393 phosphorylation; EGFR variants including G719C, G719S, L858R, T790M, L858R/T790M, several exon 19 deletions, and deletion/insertion combinations such as L747E751del,Sins; FLT3 variants including D835H, D835Y, K663Q, N841I, R834Q, and ITD insertion; as well as variants in BRAF, FGFR3, KIT, LRRK2, MET, PIK3CA, RET, and GCN2. The dataset also includes special representation cases such as CDK4-cyclinD1 and CDK4-cyclinD3 complexes and domain-resolved entries for JAK1/2/3, TYK2, RPS6KA4/5, and RSK1/2/3/4 (Wu et al., 30 Nov 2025).

The paper formalizes the protein space by distinguishing wild-type proteins

$$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$

modified variants

$$P^m = \{p^{m_i} \mid i = 1, 2, 3, \dots, \lvert P^m \rvert\},$$

with

$$p^{m_i} = \{p^{m_i}_j \mid j = 1, 2, 3, \dots \lvert p^{m_i} \rvert\},$$

and the combined protein set

$P^* = P^w \cup P^m.$

Ligands are

$442 \times 72 = 31{,}824$0

and affinity is denoted $442 \times 72 = 31{,}824$1 (Wu et al., 30 Nov 2025).

3. Benchmark design

DAVIS-Complete is defined not only as a dataset but as a three-part benchmark suite. Its central concern is whether a model can move from wild-type proteins to modifications without simply reusing wild-type affinity patterns (Wu et al., 30 Nov 2025).

The first setting, Augmented Dataset Prediction, pools wild-type and modified protein–ligand pairs and evaluates conventional affinity prediction under protein, ligand, or joint novelty. Its benchmark summary is:

• training on $442 \times 72 = 31{,}824$2,
• testing on $442 \times 72 = 31{,}824$3 for new-ligand,
• testing on $442 \times 72 = 31{,}824$4 for new-protein,
• testing on $442 \times 72 = 31{,}824$5 for both-new.

The split strategies are further refined into ligand-name and ligand-structure splits, protein-modification, protein-name, and protein-seqid splits, and two both-new combinations. The intended split ratio is approximately $442 \times 72 = 31{,}824$6 for train/validation/test (Wu et al., 30 Nov 2025).

The second setting, Wild-Type to Modification Generalization, isolates zero-shot transfer from wild-type proteins to modified proteins. Training uses

$442 \times 72 = 31{,}824$7

while testing uses

$442 \times 72 = 31{,}824$8

This is further decomposed into:

• a global modification-generalization benchmark,
• a same-ligand, different-modifications task with training on $442 \times 72 = 31{,}824$9 and testing on $56 \times 72 = 4{,}032$0,
• a same-modification, different-ligands task with training on $56 \times 72 = 4{,}032$1 and testing on $56 \times 72 = 4{,}032$2.

This design explicitly asks whether a model trained only on wild-type pairs can distinguish relative affinities among different variants of the same kinase or can rank ligands for a modified target (Wu et al., 30 Nov 2025).

The third setting, Few-Shot Modification Generalization, augments the previous protocol with limited modification-specific fine-tuning. It mirrors the two pairwise tasks above:

• same-ligand, different-modifications: train on $56 \times 72 = 4{,}032$3, fine-tune on $56 \times 72 = 4{,}032$4, test on $56 \times 72 = 4{,}032$5,
• same-modification, different-ligands: train on $56 \times 72 = 4{,}032$6, fine-tune on $56 \times 72 = 4{,}032$7, test on $56 \times 72 = 4{,}032$8.

The paper states that 80% of the available modified protein–ligand pairs are used for fine-tuning and the remaining 20% for evaluation within these task contexts (Wu et al., 30 Nov 2025).

A central derived quantity is the affinity shift between a modified variant and its wild type: $56 \times 72 = 4{,}032$9 This makes the benchmark explicitly comparative rather than merely predictive (Wu et al., 30 Nov 2025).

4. Labels, preprocessing, and evaluation protocol

The prediction target is transformed dissociation constant $442 \times 72$0, not raw $442 \times 72$1. The paper repeatedly refers to $442 \times 72$2 corresponding to $442 \times 72$3, and reports that about 70% of pairs are capped in this way. Among uncapped values, affinities center at $442 \times 72$4, with median 6.24, interquartile range 5.68–7.08, and range $442 \times 72$5. For modified pairs, the paper reports $442 \times 72$6 with mean $442 \times 72$7, median $442 \times 72$8, interquartile range $442 \times 72$9 to $442 \times 72$0, and range $442 \times 72$1; however, because of the $442 \times 72$2 cap, the exact $442 \times 72$3 is unobservable for about 60% of modified pairs (Wu et al., 30 Nov 2025).

The appendix partitions the 4,032 modified pairs into four trackability classes:

• WT-uncapped / modification-uncapped: 1601,
• WT-capped / modification-uncapped: 134,
• WT-uncapped / modification-capped: 157,
• WT-capped / modification-capped: 2068.

This partition is methodologically important because the diagnostic value of a modified-pair benchmark depends strongly on whether the modification effect is actually observable (Wu et al., 30 Nov 2025).

Preprocessing is modification-aware at the protein level. The authors exclusively selected the kinase domain for each kinase protein; if DAVIS lacked a domain annotation, they used UniProt domain information. For corrected entries such as JAK and TYK2, catalytic and pseudokinase domain boundaries were manually selected from literature and UniProt. For docking-free methods, proteins are input as sequences or derived graph/contact-map features, and phosphorylations are not accounted for in docking-free inputs due to intrinsic input constraints. For the docking-based FDA pipeline, wild-type structures came from the AlphaFold Protein Structure Database, while modified and phosphorylated variants were structurally predicted with AlphaFold3. For CDK4-cyclin complexes, docking-free models used concatenated sequences, whereas FDA used AlphaFold3-predicted 3D structures (Wu et al., 30 Nov 2025).

The evaluation protocol varies by benchmark. In Augmented Dataset Prediction, the reported metrics are mean squared error and Pearson correlation coefficient $442 \times 72$4. In Wild-Type to Modification Generalization and Few-Shot Modification Generalization, the reported metrics are MSE, Pearson $442 \times 72$5, and concordance index. The paper also defines two explicit wild-type baselines:

• $442 \times 72$6: the measured wild-type affinity used as prediction for the modified pair,
• $442 \times 72$7: the model prediction on the wild-type pair used as prediction for the modified pair.

These baselines are conceptually central, because failure to beat them implies failure to learn modification-specific effects rather than simple wild-type reuse (Wu et al., 30 Nov 2025).

5. Models and empirical findings

The benchmark compares five docking-free and two docking-based methods. The docking-free set comprises DeepDTA, AttentionDTA, GraphDTA, DGraphDTA, and MGraphDTA. The docking-based set comprises FDA, a three-stage Folding-Docking-Affinity pipeline using structure prediction, DiffDock, and GIGN with an ensemble of five affinity predictors, and Boltz-2, an open-source biomolecular foundation model whose affinity module is retrained on DAVIS-Complete (Wu et al., 30 Nov 2025).

Across the augmented benchmark, docking-based methods generally dominate stricter novelty regimes, and Boltz-2 is the strongest overall model. On the strictest both-new split, ligand-structure plus protein-seqid, the complete-set results reported in the paper are:

• DeepDTA: MSE 0.97, $442 \times 72$8,
• FDA: MSE 0.89, $442 \times 72$9,
• Boltz-2: MSE 0.62, $56 \times 72$0.

On the modification subset for the same split:

• MGraphDTA: MSE 1.61, $56 \times 72$1,
• FDA: MSE 1.37, $56 \times 72$2,
• Boltz-2: MSE 0.95, $56 \times 72$3.

The principal exception is the protein-modification split, where different variants of the same kinase are distributed across train and test. There, the paper reports that DeepDTA slightly surpasses Boltz-2 on both the complete set and the modification subset, with DeepDTA at MSE 0.29 and $56 \times 72$4, versus Boltz-2 at MSE 0.31 and $56 \times 72$5 (Wu et al., 30 Nov 2025).

The most important interpretation arises in Wild-Type to Modification Generalization. In the global benchmark restricted to WT-uncapped / modification-uncapped pairs, several docking-free models achieve raw performance around MSE 0.61–0.66 with $56 \times 72$6–0.80 and C-index $56 \times 72$7–0.80, whereas FDA reports MSE 1.47, $56 \times 72$8, C-index 0.72, and Boltz-2 reports MSE 0.83, $56 \times 72$9, C-index 0.77. However, the same section reports a stronger diagnostic: for the strongest docking-free models,

$$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$0

whereas FDA gives 0.58. This indicates that docking-free predictions for modified proteins are often nearly identical to their corresponding wild-type predictions, which the authors interpret as wild-type memorization rather than genuine modification awareness (Wu et al., 30 Nov 2025).

The distinction becomes clearer in settings where wild-type copying should fail. In WT-capped / modification-uncapped pairs, Boltz-2 achieves MSE 0.24, $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$1, C-index 0.58, compared with DeepDTA at MSE 0.35, $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$2, C-index 0.53 and FDA at MSE 0.30, $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$3, C-index 0.53. In WT-uncapped / modification-capped pairs, the reported MSE values are 1.89 for DeepDTA, 0.65 for FDA, and 0.57 for Boltz-2. This is the empirical basis for the paper’s conclusion that docking-based models generalize better in zero-shot settings where a modification materially changes affinity (Wu et al., 30 Nov 2025).

The most difficult task is same-ligand, different-modifications. Reported mean Pearson values remain low for all methods:

• DeepDTA: $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$4, C-index 0.53,
• MGraphDTA: $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$5, C-index 0.53,
• FDA: $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$6, C-index 0.56,
• Boltz-2: $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$7, C-index 0.60.

The paper explicitly states that even the best result is weak and that the wild-type ground-truth baseline matches or exceeds the models in MSE terms, so zero-shot prediction of relative effects across multiple mutations of the same protein remains unsolved (Wu et al., 30 Nov 2025).

By contrast, same-modification, different-ligands is substantially easier. The paper reports, for example:

• DeepDTA: MSE 0.56, $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$8, C-index 0.83,
• MGraphDTA: MSE 0.54, $$P^w=\{ p^{w_i}\mid i=1,2,3,\dots,\lvert P^w \rvert\},$$9, C-index 0.84,
• FDA: MSE 1.30, $$P^m = \{p^{m_i} \mid i = 1, 2, 3, \dots, \lvert P^m \rvert\},$$0, C-index 0.75,
• Boltz-2: MSE 0.75, $$P^m = \{p^{m_i} \mid i = 1, 2, 3, \dots, \lvert P^m \rvert\},$$1, C-index 0.80.

The paper explains this by noting that in 44 of 55 cases, wild-type and modified ligand-affinity profiles have $$P^m = \{p^{m_i} \mid i = 1, 2, 3, \dots, \lvert P^m \rvert\},$$2, so ligand-ranking structure is often preserved even when mutation-specific absolute values are not (Wu et al., 30 Nov 2025).

Few-shot modification generalization changes the picture. In same-ligand, different-modifications, all docking-free models improve markedly in MSE after fine-tuning; DeepDTA, for example, improves from 0.62 to 0.33, and AttentionDTA from 0.68 to 0.34. Yet the corresponding $$P^m = \{p^{m_i} \mid i = 1, 2, 3, \dots, \lvert P^m \rvert\},$$3 and C-index remain low, so ranking across mutations remains weak. In same-modification, different-ligands, fine-tuning produces modest improvements for most docking-free models, whereas FDA and Boltz-2 do not benefit from the fine-tuning protocols used in the paper. The authors interpret this as evidence that docking-free models can recover some performance with few-shot adaptation, while docking-based models remain stronger in zero-shot robustness but may require different fine-tuning strategies (Wu et al., 30 Nov 2025).

6. Relation to other DAVIS resources, limitations, and availability

A recurring source of ambiguity is the term “DAVIS.” In the literature, DAVIS also denotes the Densely-Annotated VIdeo Segmentation benchmarks, especially DAVIS 2017 and the 2018 challenge extensions, which are video object segmentation resources with multi-object semi-supervised protocols, pixel-level masks, and official train/validation/test-dev/test-challenge splits (Pont-Tuset et al., 2017, Caelles et al., 2018). It also appears in event-camera driving research through the DDD17 “DAVIS Driving Dataset 2017,” which is a DAVIS APS+DVS driving dataset with synchronized vehicle telemetry for end-to-end driving experiments (Binas et al., 2017). DAVIS-Complete, by contrast, is a protein–ligand affinity benchmark and should not be conflated with these vision datasets (Wu et al., 30 Nov 2025).

The protein-affinity benchmark itself is presented with several explicit limitations. It is still small for deep learning, remains kinase-centric, and inherits heavy label censoring because about 70% of $$P^m = \{p^{m_i} \mid i = 1, 2, 3, \dots, \lvert P^m \rvert\},$$4 values are capped at $$P^m = \{p^{m_i} \mid i = 1, 2, 3, \dots, \lvert P^m \rvert\},$$5. The appendix also reports overlap between DAVIS-Complete entities and structural training sources used by FDA components and Boltz-2, including 212 of 444 proteins in AlphaFold-Multimer overlap, 144 of 444 proteins and 36 of 72 ligands in DiffDock overlap, and 246 of 444 proteins plus 47 of 72 ligands in PDB overlap for Boltz-2-related training sources. The authors therefore caution that apparent generalization of docking-based models may be inflated by upstream structural exposure. They also note that AlphaFold3 may fail to reflect subtle modification-specific conformational changes, citing the example that it predicted a phosphorylated-like state for both phosphorylated and non-phosphorylated ABL1 in their inspection (Wu et al., 30 Nov 2025).

Within those limitations, the benchmark’s main methodological contribution is to reveal a specific failure mode of current affinity prediction systems: strong performance can arise from implicit reuse of wild-type affinity patterns rather than true modeling of protein modification effects. DAVIS-Complete is therefore best understood as a stress test for whether a model is genuinely modification-aware. The benchmark is publicly available at the repository identified in the paper: https://github.com/ZhiGroup/DAVIS-complete (Wu et al., 30 Nov 2025).