Adversarial Drift-Aware Predictive Transfer (ADAPT)

• ADAPT is a framework addressing model drift in evolving systems by using adversarial strategies and uncertainty estimation to maintain performance.
• It compensates for prototype and coefficient drift by generating pseudo-exemplars and robust projections without storing raw data.
• Empirical results show ADAPT improves accuracy and slows performance decay in both vision benchmarks and clinical prediction tasks.

Adversarial Drift-Aware Predictive Transfer (ADAPT) is a class of methodologies designed to mitigate the effects of model drift in temporally evolving or continually learned systems, with minimal access to historical or future data samples. The central motivation underlying ADAPT is the intrinsic vulnerability of deployed machine learning models—whether deep continual learners or clinical risk prediction models—to degradation as the underlying data distribution shifts over time, either through covariate or concept drift. ADAPT frameworks, as introduced in recent literature, address this by leveraging adversarial or uncertainty-based mechanisms for drift estimation and robust predictive transfer, without reliance on storing raw exemplars or requiring frequent retraining. Notable instantiations span settings from exemplar-free continual learning in vision to longitudinal predictive modeling in clinical informatics (Goswami et al., 2024, Xiong et al., 17 Jan 2026).

1. Problem Context: Drift in Continual and Temporal Learning

Deployed machine learning models typically experience two forms of drift: covariate drift ($P_t(X)$, changes in input feature distributions) and concept drift ($P_t(Y|X)$, changes in the predictive relationship). In continual learning, particularly in the class-incremental regime without exemplar storage, repeated adaptation of feature extractors leads to prototype drift in the learned embedding space, resulting in catastrophic forgetting. In clinical AI, temporal drift arises from evolving patient populations, diagnostic coding updates, and unanticipated systemic changes. In both domains, frequent full-model retraining is often impractical due to computation, privacy, and data access constraints (Goswami et al., 2024, Xiong et al., 17 Jan 2026).

2. Adversarial Drift-Aware Compensation in Exemplar-Free Continual Learning

In the context of exemplar-free class-incremental learning (EFCIL), ADAPT—termed "Adversarial Drift Compensation" in (Goswami et al., 2024)—addresses the inability to directly update old class prototypes in the absence of prior-task data. A feature extractor $f_t$ and a linear head $W_t$ are trained on the current task's data $\mathcal D_t$, while only the mean embeddings ("prototypes") $P^c_{t-1}$ of past classes $c$ are retained.

To compensate for prototype drift:

• Pseudo-exemplars for each old class are created by adversarially perturbing current data samples such that their embeddings under $f_{t-1}$ are close to $P_{t-1}^c$.
• These adversarial samples, $\mathcal X^c_{adv}$, are selected by targeted iterative gradient steps minimizing $\|f_{t-1}(x)-P_{t-1}^c\|_2^2$, and retaining only those classified as $c$ under $f_{t-1}$'s nearest-mean classifier.
• The drift vector $\Delta^c_{t-1\to t}$ is estimated as the mean embedding shift from $f_{t-1}$ to $f_t$ across $\mathcal X^c_{adv}$.
• Prototypes are updated as $P^c_t = P^c_{t-1} + \Delta^c_{t-1\to t}$.

Inference proceeds via a nearest-class-mean (NCM) classifier, with prototypes for current classes set by averaging real embeddings, and old classes using the compensated $P^c_t$. This approach side-steps the need for storing or replaying previous data, is computationally efficient (few adversarial attack steps per class), and empirically yields superior retention of old knowledge (Goswami et al., 2024).

3. Adversarial Drift-Aware Predictive Transfer for Durable Clinical AI

Within temporal clinical prediction, ADAPT (Xiong et al., 17 Jan 2026) constructs robustness against performance decay by adversarial optimization over a set of plausible future models:

• For outcome $Y$ modeled by a GLM $\mathbb E[Y|X] = g^{-1}(X^\top\beta)$, historical model estimates $\{\widehat\beta^{(1)},\dots,\widehat\beta^{(L)}\}$ (one per past time period) are retained.
• A small, current labeled sample is split for estimation and validation, yielding $\widehat\beta^{(0)}$ and loss-measured mixture estimates.
• The set of future/plausible coefficients is constructed as the convex hull of all $\widehat\beta^{(l)}$, intersected with a loss threshold—the uncertainty set $C(\tau)$.
• The ADAPT estimator $\widehat\beta_{\mathrm{ADAPT}}$ solves the quadratic program:

$$\widehat\beta_{\mathrm{ADAPT}} = \arg\min_{\beta\in C(\tau)}\; (\beta-\beta_0)^\top H (\beta-\beta_0),$$

where $H$ is the Hessian of the negative log-likelihood at $\beta_0$, ensuring minimax optimality over $C(\tau)$ under local quadratic approximation.

Key operational features include use of only summary-level coefficients, ensuring privacy, and a single quadratic projection for robustification, making deployment computationally negligible compared to model retraining (Xiong et al., 17 Jan 2026).

4. Empirical Evaluation and Performance

Experimental validation in (Goswami et al., 2024) spans standard and fine-grained continual learning benchmarks (CIFAR-100, TinyImageNet, ImageNet-Subset, CUB-200, Stanford Cars), emphasizing the challenging "small-start" regime. ADAPT outperforms baselines (LwF, NCM, SDC, PASS, SSRE, FeTrIL, FeCAM) with, for example, a gain of 5.12 pts in last-task accuracy on CIFAR-100 (SDC: 41.36, ADAPT: 46.48) and 6.19 pts on CUB-200 (runner-up: 51.78, ADAPT: 57.97). Prototype-only memory (one per class) suffices to surpass nearest-mean-exemplar baselines using up to 2000 stored images.

In clinical prediction, ADAPT (Xiong et al., 17 Jan 2026) is evaluated on real-world electronic health record datasets (Mass General Brigham, Duke Health), spanning annual periods and systemic shocks (e.g., ICD transitions, COVID-19). Relative to Pooled, TransGLM, and Maximin-DRO, ADAPT slows AUC degradation by roughly half over 2–8 years (e.g., MGB 8-year degradation: ADAPT 7.65%, Pooled 12.1%, TransGLM 18.0%). Cross-site evaluation reaffirms generalizability.

5. Algorithmic Details and Practical Considerations

The ADAPT procedure in continual learning involves:

• Training the new feature extractor and classifier on $\mathcal D_t$ (including distillation loss on old logits).
• For each old class: extracting $m$ nearest samples, adversarial optimization (best at 3 iterations, step-size 25, $m\approx100$), drift estimation, and prototype update.
• New class prototypes are direct averages.
• Runtime is limited to prototype maintenance; no images are stored.

In the clinical ADAPT case, once all $L+1$ penalized GLMs are fit, the robust coefficient is obtained via a single quadratic program in convex-combination space. Empirically, this incurs negligible computational cost compared to full retraining. Only model summaries and the Hessian are required across sites, preserving data privacy (Xiong et al., 17 Jan 2026).

Limitations include potential failure to capture shifts outside the convex hull spanned by past models (e.g., unprecedented events), and the GLM assumption, though extensions exist for deep representations or alternative uncertainty set constructions. Possible extensions address semi-supervised incorporation of unlabeled data and dynamic updating as more current labels accrue.

6. Comparison with Prior and Related Approaches

In continual learning, SDC estimates drift via Gaussian-weighting all current samples, which degrades with few samples near an old prototype—especially under small-start conditions. Other recent methods (PASS, SSRE, FeTrIL, FeCAM) utilize self-supervision or covariance modeling but do not generate pseudo-exemplars targeting every old class. ADAPT's targeted adversarial strategy produces pseudo-exemplars indistinguishable from true old-class embeddings, achieving higher drift-estimation fidelity (cosine similarity to true drift: 0.7–0.9 for ADAPT, 0.1–0.2 for SDC).

The clinical ADAPT contrasts with TransGLM, Maximin-DRO, and standard pooling: it uniquely optimizes worst-case excess risk over an empirically valid mixture set, balancing today’s accuracy against robustness to future drift, with theoretical upper bounds on loss and rapid computation (Goswami et al., 2024, Xiong et al., 17 Jan 2026).

7. Significance and Implications

ADAPT provides a principled and efficient mechanism for drift compensation in exemplar-free continual deep learning and privacy-constrained clinical AI. By solely leveraging class prototypes (in vision) or summary model coefficients (in clinical settings), it is memory-constrained, preserves privacy, and sidesteps the need for raw data retention or retraining. Its empirical stability and robustness to data drift suggest significant potential for extending the operational life of deployed AI models—halving performance decay rates in high-stakes domains such as medical prognosis without compromising privacy or requiring impractical annotation effort. A plausible implication is broader adoption of ADAPT-style robustification for any domain where retraining is difficult and operational reliability must be maintained over extended, non-stationary deployments (Goswami et al., 2024, Xiong et al., 17 Jan 2026).