OncoTraj: EGFR NSCLC Resistance Benchmark

Updated 6 July 2026

OncoTraj is a benchmark that defines fixed prediction tasks using baseline tissue-NGS data in EGFR-mutant NSCLC undergoing first-line osimertinib.
It aggregates data from 813 real patients across three clinical-genomic sources with designated train/validation/test splits and leakage auditing.
The resource reveals that single-timepoint genomic inputs yield predictions near chance, emphasizing the need for serial ctDNA sampling for improved forecasting.

OncoTraj is a public benchmark for longitudinal resistance prediction in EGFR-mutant non-small-cell lung cancer treated with first-line osimertinib. It was introduced as a harmonized, leakage-audited resource comprising 813 real patients drawn from three clinical-genomic sources, together with fixed prediction tasks, patient-level train/validation/test splits, an open-source evaluation harness, and six reference baselines. Its central result is deliberately diagnostic rather than triumphant: with v1’s predominantly single-timepoint tissue-NGS inputs, no task exceeds chance on clean within-source evaluation, implying that the limiting factor is the input modality rather than the choice of model class (Sarkar et al., 9 Jun 2026).

1. Clinical setting and conceptual scope

The benchmark is situated in a disease-treatment context that the authors describe as the “canonical example of predictable clonal evolution under therapeutic pressure”: EGFR-mutant NSCLC receiving first-line osimertinib. This setting is attractive for computational trajectory modeling because resistance is common, biologically recurrent, and therapeutically consequential. In FLAURA, median progression-free survival was 18.9 months, yet almost all patients ultimately develop resistance within 2–3 years. The resistance landscape includes recurrent mechanisms such as MET amplification and EGFR C797S, while prior studies cited by the benchmark indicate that serial ctDNA can track resistant clones and that early plasma EGFR mutation clearance has prognostic value (Sarkar et al., 9 Jun 2026).

Within that clinical frame, OncoTraj defines “trajectory” in a specific and somewhat constrained way. The labels are genuinely longitudinal: progression by a 12-month landmark, time to first progression, and dominant resistance mechanism at progression. The inputs, however, are mostly not dense trajectories. The v1 cohort is dominated by baseline or near-baseline molecular snapshots, mainly tissue NGS obtained before or at osimertinib initiation. This asymmetry between dynamic labels and largely static predictors is the benchmark’s key scientific tension.

A consequence is that OncoTraj is both a benchmark and a modality audit. It does not merely ask whether a model can forecast resistance; it asks whether resistance can be forecast from the specific evidence type available in v1. The benchmark’s answer is that mostly baseline tissue genomics can recover weak prognostic associations, but not robust within-source prediction of progression timing or resistance mechanism.

2. Cohort construction, harmonization, and data model

OncoTraj v1 harmonizes three real-world sources into a versioned four-table schema—patients, variants, treatments, and outcomes—released as oncotraj-schema/1.0.0. The contributing sources are MSK-CHORD with 672 patients in the first-line osimertinib cohort, AACR Project GENIE BPC NSCLC v2.0 with 34 such patients, and the FLAURA molecular-resistance supplement with 107 patients, yielding 813 patients in total (Sarkar et al., 9 Jun 2026).

Inclusion required histologically confirmed NSCLC, specifically lung adenocarcinoma or NSCLC-NOS; a documented EGFR sensitizing variant; osimertinib as the first EGFR tyrosine kinase inhibitor; and at least one molecular sample, either tissue NGS or ctDNA, before or at osimertinib initiation. Sensitizing variants listed for eligibility were exon 19 deletion, L858R, G719X, L861Q, S768I, and compound sensitizing mutation. Patients whose first EGFR TKI was erlotinib, gefitinib, afatinib, or dacomitinib were excluded from v1 and retained as a held-out prior-TKI sensitivity slice.

The EGFR composition is dominated by exon19del and L858R. Specifically, exon19del accounts for 424 patients (52%) and L858R for 290 (36%), with smaller counts for compound_with_resistance (34), compound_sensitizing (22), G719X (20), L861Q (18), exon20ins (4), and S768I (1). The benchmark notes that 565 of 813 patients, or 69%, have at least one documented progression event with real timing.

The engineered feature space for the classical baselines is organized into molecular dynamics, co-mutation profile, and clinical features. Molecular dynamics include latest_vaf, vaf_slope, and variant_burden. Co-mutation features are binary indicators for TP53, RB1, CDKN2A, PIK3CA, CTNNB1, MET, ERBB2, KRAS, and BRAF, together with comut_burden. Clinical content includes time_on_therapy_days, plus one-hot encodings of source_dataset and egfr_variant_class. The paper is explicit that this feature design leaves the benchmark dominated by single-snapshot inputs: the schema can represent serial data, but dense serial molecular sampling is sparse in v1.

3. Locked tasks, splits, and evaluation protocol

The benchmark defines three locked tasks whose labeling logic is fixed before modeling. Task A is binary classification of progression by a fixed 12-month landmark. Task B is regression of time-to-first-progression in days. Task C is six-class classification of the dominant resistance mechanism at progression. These tasks are evaluated on fixed patient-level splits of 70/15/15, stratified jointly on source dataset and Task A label, producing 570 training patients, 121 validation patients, and 122 test patients (Sarkar et al., 9 Jun 2026).

Task	Definition	Reported metrics
A	Predict RECIST progression by day 365 using only records on or before osimertinib initiation	ROC-AUC, Brier score, ECE
B	Predict days from osimertinib initiation to first RECIST progression	MAE, RMSE, C-index
C	Predict dominant resistance mechanism at progression among six classes	macro-F1, weighted-F1, per-class F1

Task A is formulated as a landmark problem because an “ever progresses” label is nearly degenerate in this disease setting. A positive label is assigned when progression occurs on or before day 365. A negative label requires confirmed progression-free follow-up beyond day 365. Patients censored before day 365 are excluded. Out of the 813-patient cohort, 760 have determinable Task A labels, 53 are excluded for censoring before the landmark, the cohort positive rate is 45%, and the effective Task A test set size is 110.

Task B is defined only for patients with a documented progression event. Task C is also progression-conditioned and uses six classes: EGFR_C797S, MET_amplification, EGFR_amplification, histologic_transformation, other, and unknown. The paper states that the full hierarchy for assigning dominant mechanisms, including co-occurring mechanisms, low-VAF thresholds, and audit-log requirements, is specified in LABELING_GUIDELINES.md rather than reproduced in the manuscript.

Evaluation is centralized in the oncotraj.eval package. Predictions are submitted through a predictions.csv schema containing patient_id, task_a_pred, task_b_pred, task_c_pred, task_a_proba, task_c_proba, and split, and scored via oncotraj-eval --predictions <csv> --split test. All headline results use bootstrap 95% confidence intervals from 1000 test-set resamples. The benchmark also freezes its split manifest in _splits.json, versions evaluation reports, pins hyperparameters in project files, and tracks runs with MLflow.

4. Reference baselines and empirical performance

OncoTraj releases six reference baselines spanning a majority-class predictor, logistic regression, random forest, XGBoost, an LSTM, and a multi-task transformer. The logistic baseline uses LogisticRegression(class_weight='balanced'). The random forest uses 500 trees with max_depth = 8. XGBoost uses 200 boosting rounds, learning_rate = 0.05, and max_depth = 6. The LSTM is a two-layer model over typed clinical events with hidden sizes searched over $\{32,64,128\}$ and learning rates over $\{10^{-3},10^{-4}\}$ . The multi-task transformer has approximately 10 million parameters, uses rotary position embeddings on timestamp deltas, and is trained in configurations with $d_{\text{model}} = 128, 256, 384$ . For Task B in the deep models, the regression target is transformed as $y'=\log(1+y)$ and inverted at evaluation as $\hat{y}=\exp(\hat{y}')-1$ (Sarkar et al., 9 Jun 2026).

The mixed-source headline table shows superficially nontrivial results. On the held-out test split, the majority floor gives Task A AUC 0.500, Task B MAE 344 days, Task B C-index 0.500, and Task C macro-F1 0.497. Logistic regression reaches Task A AUC 0.680 with 95% CI 0.581–0.781, Task B MAE 261 days, Task B C-index 0.581, and Task C macro-F1 0.517. Random forest reaches Task A AUC 0.678, Task B MAE 252 days, Task B C-index 0.656, and Task C macro-F1 0.488. XGBoost reaches Task A AUC 0.630, Task B MAE 266 days, Task B C-index 0.643, and Task C macro-F1 0.491.

The benchmark’s interpretation, however, is that these pooled figures are upper bounds contaminated by source structure. When Task A is evaluated more cleanly, the signal recedes. With source flags removed, logistic regression yields AUC 0.680; but on the strongest within-source test—MSK-CHORD only with source flags removed—the AUC drops to 0.596 with 95% CI 0.478–0.714, which includes 0.500. The paper therefore concludes that Task A is not distinguishable from chance within a single source at v1 scale.

Task B shows the same pattern. In v1.1 within-source evaluation on MSK-CHORD, the majority floor gives MAE 303 days and C-index 0.50. Logistic gives MAE 301 and C-index 0.49. Random forest gives the best MAE, 293 days, but a C-index of 0.43. XGBoost gives MAE 312 and C-index 0.44. A Cox proportional hazards baseline gives MAE 513 and C-index 0.54. The reported improvement from 303 to 293 days is therefore numerically small, and the ranking performance remains at or below chance for the regressors.

Task C fails more decisively. No baseline beats the majority floor. The benchmark ties this failure to class structure rather than algorithmic underfitting. In particular, EGFR C797S is rare in the extracted v1 data—about 1.6% of MSK-CHORD and about 2% of GENIE BPC—and the held-out test fold contains only one C797S-positive patient.

The deep models do not alter the conclusion. The LSTM reaches Task B MAE 281 days, while the transformer regression head reportedly collapses near a constant prediction of approximately 373 days, worse than the majority floor. The uniformity of this ceiling across classical and deep baselines is the benchmark’s key empirical claim.

5. Leakage auditing, confounds, biological signal, and limits

A defining feature of OncoTraj is that it treats benchmark governance as part of the scientific contribution. Before release, the authors identified a major label-leakage problem in Task A: the engineered features latest_vaf and vaf_slope had initially incorporated variant records after the prediction cut-off, thereby peeking at samples near progression. This produced implausibly high Task A AUCs of 0.92–0.94. The pipeline was rebuilt so that every feature is computed strictly from records dated on or before the cut-off, and an automated check, tests/test_taska_feature_leakage.py, was added to enforce the invariant that cut-off filtering before feature construction must equal deleting all post-cut-off rows before feature construction (Sarkar et al., 9 Jun 2026).

The benchmark also identifies a source-flag confound. Because the FLAURA subset lacks individual progression dates, its harmonized landmark labels are structurally negative by construction. If source_dataset is included as a feature, a model can exploit source identity. The paper quantifies the effect: logistic AUC is 0.716 with all features including source flags, 0.680 when source flags are removed, and 0.596 on MSK-CHORD only. This robustness ladder is central to the argument that mixed-source performance exaggerates true predictive signal.

Although the predictive ceiling is low, OncoTraj recovers a literature-consistent biological association. Across the landmark cohort, TP53 co-mutation raises the 12-month progression rate from 29% in TP53 wild-type patients to 59% in TP53 co-mutant patients, with sample sizes $n=340$ and $n=420$ , respectively. The same direction is preserved within sources: in MSK-CHORD the rate is 60% versus 42%, and in GENIE BPC it is 45% versus 33%. The logistic model’s largest coefficients fall on TP53 and co-mutation burden, suggesting that v1 does contain real osimertinib biology even when the discrimination is insufficient for dependable prediction.

The benchmark’s central limitation is therefore not absence of signal, but mismatch between biological question and measurement regime. Most patients contribute only one tissue-NGS sample before or at therapy start; vaf_slope becomes zero for single-sample patients; and acquired on-treatment events such as C797S are poorly represented. The authors explicitly state that v1 is not a basis for clinical utility claims and formulate concrete requirements for a v2: serial ctDNA acquisition with a median test patient having at least three ctDNA timepoints rather than effectively one; removal of FLAURA from Tasks A and B unless individual progression dates become available; addition of an 18-month landmark task; and mandatory source-stratified reporting. They also suggest that deep models may only become meaningfully competitive once both cohort size and trajectory depth increase, with an empirical regime on the order of 1,000 to 10,000 patients.

6. Position within the broader oncology trajectory literature

OncoTraj occupies a specific niche within a broader and heterogeneous literature on cancer “trajectories.” Some work treats trajectories as evolutionary graphs over genomic alterations. TRONCO, for example, infers cancer progression models from binary genomic and epigenomic alterations, distinguishing population-level accumulation trends from individual clonal evolution, but it is a model-inference framework rather than a disease-specific benchmark for resistance under targeted therapy (Sano et al., 2015). Other work treats trajectories as longitudinal symptom courses. THALIS analyzes 699 head and neck cancer patients through unsupervised clustering, association-rule mining, and visual “filament” plots, emphasizing contextualization of symptom evolution rather than resistance prediction (Floricel et al., 2021).

A second branch of the literature treats trajectories as serial imaging or report-derived lesion change. OncoNet uses weak supervision from report-derived SUVmax to classify change between paired FDG PET/CT exams, reaching AUROC 0.86 internally and 0.84 externally for pairwise response assessment (Joshi et al., 2021). “Tracking Cancer Through Text” builds an open-source LLM pipeline for longitudinal extraction from paired radiology reports, with attribute-level accuracies of 93.7% for target lesions, 94.9% for non-target lesions, and 94.0% for new lesions, thereby addressing lesion linkage in text rather than molecular resistance prediction (Builtjes et al., 10 Mar 2026). ONCOPILOT contributes a promptable CT foundation model for interactive 3D lesion segmentation and radiologist-level RECIST-style measurement, but explicitly leaves longitudinal lesion matching and trajectory analytics for future work (Machado et al., 2024).

A third branch defines trajectories at the level of care pathways or latent patient subtypes. “Care Trajectories Are Linked to Mental Health and Mortality in Cancer Patients” uses Dynamic Time Warping and hierarchical clustering on encounter sequences from more than 8,000 patients to identify nine care-trajectory phenotypes associated with mortality (Lindner et al., 20 Apr 2026). “Trajectory-informed graph-based clustering for longitudinal cancer subtyping” combines multi-state survival modeling, imaging-derived longitudinal features, and graph learning to discover subtypes aligned with relapse and death pathways (Cavinato et al., 10 Mar 2026). By contrast, CoC is a multimodal survival model over pathology, genomics, methylation, and prompt text, but its “chain” is cross-modal rather than longitudinal and does not represent disease evolution across visits (Zhou et al., 28 May 2025).

Against that background, OncoTraj is distinctive because it turns a clinically canonical resistance setting into a public, leakage-audited benchmark with fixed tasks and fixed splits. Its importance lies less in proposing a new trajectory model than in standardizing a hard longitudinal prediction problem and demonstrating, with unusually explicit confound analysis, that single-snapshot tissue genomics are inadequate for solving it. In that sense, OncoTraj functions as both infrastructure and falsification device: it establishes a reproducible floor for osimertinib-resistance forecasting and converts benchmark failure into a concrete design specification for serial-ctDNA-enriched longitudinal modeling (Sarkar et al., 9 Jun 2026).