Explain the prospective–retrospective response-prediction gap

Determine whether the large difference between prospective response prediction from pre-treatment cells and retrospective response classification from post-treatment cells is caused by the scDEFT model, the sample size, unmeasured confounders, or donor-specific dynamics that individualize each treatment transition.

Background

scDEFT predicts responder status prospectively from pre-treatment cells with an AUROC of 0.70, whereas retrospective classification from post-treatment cells reaches an AUROC of 0.93. The authors identify this substantial gap but do not resolve its source, leaving uncertainty about whether it reflects methodological limitations, limited donor numbers, confounding factors, or genuine biological heterogeneity in treatment trajectories.

References

The gap between the prospective and retrospective tasks is large. Whether it reflects the model, the sample size, or unmeasured confounders and donor-specific dynamics that make each transition individualized is an open question; the benchmarking below provides some insights.

— scDEFT: A deep learning framework for drug-effect prediction and counterfactual reasoning  (2609.10831 - Devarakonda, 9 Sep 2026) in Results, page 6

This could be a limitation of the limited number of cohorts, or it is intrinsic to this task and data. It is an open research question.

— scDEFT: A deep learning framework for drug-effect prediction and counterfactual reasoning  (2609.10831 - Devarakonda, 9 Sep 2026) in Supplementary Note 1, page 17