ADR: Exemplar-Free Continual Graph Learning

• ADR is a continual graph learning framework that achieves privacy-preserving, exemplar-free learning by storing only class-level prototype statistics.
• It employs free task adaptation, hierarchical analytic merging via ridge regression, and analytic classifier reconstruction to balance plasticity and stability.
• ADR demonstrates state-of-the-art performance on various node classification benchmarks, outperforming traditional ACL and rehearsal methods.

Searching arXiv for the primary paper and closely related continual graph learning works. Analytic Drift Resister (ADR) is a continual graph learning framework introduced for Non-Exemplar Continual Graph Learning (NECGL), a setting in which privacy is preserved by retaining only class-level prototype statistics rather than raw graph examples. ADR is designed to address two coupled difficulties in this regime: feature drift induced by encoder updates, and the attenuation of model plasticity associated with frozen Analytic Continual Learning (ACL) encoders. The framework combines free task-wise adaptation by backpropagation, Hierarchical Analytic Merging (HAM) of GNN layers via ridge regression, and Analytic Classifier Reconstruction (ACR), with the stated objective of achieving exemplar-free, privacy-safe, zero-forgetting continual graph learning while remaining competitive with state-of-the-art baselines on node classification benchmarks (Song et al., 3 Apr 2026).

1. Problem setting and motivation

In class-incremental graph learning, a sequence of tasks $T_0,T_1,\ldots,T_{n-1}$ is observed, with each task $T_t=(G_t,Y_t)$, where $G_t=(V_t,E_t)$ is a graph with node features $X_t\in\mathbb{R}^{|V_t|\times d}$ and adjacency $A_t$, and $Y_t$ are its labels. The label spaces are disjoint across tasks. Under Non-Exemplar constraints, once training on $T_t$ is finished, no raw nodes or subgraphs from $T_t$ may be stored; only class-level prototype statistics such as Gaussian means and covariances are cached to represent past tasks (Song et al., 3 Apr 2026).

This privacy-preserving design avoids storing individual nodes or edges, but it introduces a structural instability: as the model encoder updates on new tasks, cached prototypes become misaligned with the current embedding space. The paper identifies this misalignment as feature drift. In parallel, ACL methods avoid feature drift by freezing a pre-trained graph encoder $F_\theta(\cdot)$ and solving class-incremental learning analytically for the linear classifier through recursive least squares. In that formulation, only the accumulated feature autocorrelation $R=\sum H_i^\top H_i$ and cross-correlation $T_t=(G_t,Y_t)$0 are stored, which guarantees no feature drift and perfect privacy. The drawback is that freezing $T_t=(G_t,Y_t)$1 precludes adaptation to newly arriving task distributions, particularly when pre-training and downstream tasks differ markedly in graph structure or node attributes (Song et al., 3 Apr 2026).

ADR is positioned precisely at this junction. It is proposed to break the frozen pre-trained constraint, preserve privacy without rehearsal, and resist the drift that ordinarily follows from parameter updates.

2. Three-stage learning pipeline

ADR processes each arriving task $T_t=(G_t,Y_t)$2 through three stages: (A) Incremental Task Adaptation, (B) Hierarchical Analytic Merging (HAM), and (C) Analytic Classifier Reconstruction (ACR) (Song et al., 3 Apr 2026).

In Incremental Task Adaptation, a GNN encoder $T_t=(G_t,Y_t)$3 with $T_t=(G_t,Y_t)$4 layers and a linear classifier $T_t=(G_t,Y_t)$5 are maintained. For task $T_t=(G_t,Y_t)$6, the forward computation is

$T_t=(G_t,Y_t)$7

$T_t=(G_t,Y_t)$8

$T_t=(G_t,Y_t)$9

The task loss is the ordinary supervised cross-entropy objective

$G_t=(V_t,E_t)$0

Both $G_t=(V_t,E_t)$1 and $G_t=(V_t,E_t)$2 are updated by backpropagation, explicitly with no regularization. This stage is intended to fully exploit plasticity by allowing the encoder to adapt to the current graph distribution rather than remaining frozen (Song et al., 3 Apr 2026).

The algorithmic workflow specifies the per-task inputs as the encoder $G_t=(V_t,E_t)$3, classifier $G_t=(V_t,E_t)$4, memory banks $G_t=(V_t,E_t)$5 and $G_t=(V_t,E_t)$6, learning rate $G_t=(V_t,E_t)$7, and epochs $G_t=(V_t,E_t)$8. After $G_t=(V_t,E_t)$9 epochs of free adaptation, HAM accumulates layer-wise sufficient statistics and reconstructs merged encoder weights, and ACR updates the classifier statistics and solves the corresponding analytic reconstruction. The output is the merged model $X_t\in\mathbb{R}^{|V_t|\times d}$0 together with the memory banks (Song et al., 3 Apr 2026).

3. Hierarchical analytic merging and classifier reconstruction

HAM is the core mechanism used to neutralize feature drift after free adaptation. After training on $X_t\in\mathbb{R}^{|V_t|\times d}$1, encoders from all tasks $X_t\in\mathbb{R}^{|V_t|\times d}$2 are available. The goal is to construct a single merged encoder $X_t\in\mathbb{R}^{|V_t|\times d}$3 that reproduces the layer-wise behavior of each task-specific encoder. For the linear weight matrix $X_t\in\mathbb{R}^{|V_t|\times d}$4 of layer $X_t\in\mathbb{R}^{|V_t|\times d}$5, the paper defines, for each node $X_t\in\mathbb{R}^{|V_t|\times d}$6 in $X_t\in\mathbb{R}^{|V_t|\times d}$7,

$X_t\in\mathbb{R}^{|V_t|\times d}$8

and

$X_t\in\mathbb{R}^{|V_t|\times d}$9

Let $A_t$0 stack the post-activations $A_t$1, and let $A_t$2 stack the pre-activations $A_t$3. HAM solves the joint ridge regression

$A_t$4

Setting the gradient to zero yields the closed form

$A_t$5

Only the layer-wise autocorrelation and cross-correlation terms must be stored:

$A_t$6

The paper states that solving this problem produces $A_t$7 that is provably equivalent to jointly training on all data seen so far, and therefore yields absolute resistance to feature drift (Song et al., 3 Apr 2026).

ACR applies the same analytic principle to the final linear classifier. Given the merged encoder $A_t$8, the classifier $A_t$9 is reconstructed through ridge regression over all tasks’ merged embeddings $Y_t$0:

$Y_t$1

with solution

$Y_t$2

An optional random feature buffer expansion,

$Y_t$3

may be inserted to improve separability, with the paper explicitly invoking Cover’s theorem. In that case, the stored classifier statistics become $Y_t$4 and $Y_t$5, still without retaining any raw data (Song et al., 3 Apr 2026).

4. Theoretical properties

ADR is accompanied by two explicit propositions. Proposition 1 (Absolute Feature-Drift Resistance) states that the merged encoder $Y_t$6 produced by HAM at step $Y_t$7 is exactly the minimizer of

$Y_t$8

and is therefore exactly equivalent to joint training on all tasks up to $Y_t$9. The stated consequence is that prototypes or embeddings extracted from the merged encoder never drift (Song et al., 3 Apr 2026).

Proposition 2 (Zero-Forgetting Class Incremental) states that the classifier $T_t$0 obtained through ACR solves the joint ridge regression over all tasks’ embeddings. Its accuracy on prior tasks is therefore identical to that of a model trained jointly on all data, which the paper characterizes as zero forgetting (Song et al., 3 Apr 2026).

The proof sketches proceed by forming the combined Frobenius-norm objectives associated with HAM and ACR, verifying convexity, setting gradients to zero, and obtaining the closed forms via the Matrix Inverse Lemma. Within the paper’s presentation, the central claim is that free adaptation and analytic reconstruction are not treated as competing objectives; rather, free adaptation supplies task plasticity, while the analytic stages restore global consistency across tasks.

5. Benchmarks, baselines, and observed behavior

The empirical evaluation uses four node classification benchmarks: CS-CL with 18,333 nodes, 163,788 edges, and 15 classes; CoraFull-CL with 19,793 nodes, 126,842 edges, and 70 classes; Arxiv-CL with 169,343 nodes, 1,166,243 edges, and 40 classes; and Reddit-CL with 232,965 nodes, 114,615,892 edges, and 40 classes (Song et al., 3 Apr 2026).

Three evaluation metrics are defined. The average incremental accuracy is

$T_t$1

the final accuracy $T_t$2 is accuracy on all classes after the final task, and the learning accuracy

$T_t$3

is used as a plasticity measure. Baselines span multiple families: regularization methods (LwF, EWC, MAS, TWP), rehearsal methods (CaT, ER-GNN), non-exemplar methods (POLO, EFC), analytic ACL methods (ACIL, DS-AL, DPCR), and two reference conditions (Bare for fine-tune only, and Joint as oracle) (Song et al., 3 Apr 2026).

The reported global testing results show the following $T_t$4 values for ADR: 90.7 / 87.2 on CS-CL, 67.5 / 60.0 on CoraFull-CL, 39.7 / 34.7 on Arxiv-CL, and 82.5 / 75.5 on Reddit-CL. The corresponding Joint $T_t$5 values are 88.9, 61.6, 42.1, and 79.8. The paper states that ADR outperforms all existing NECGL, ACL, rehearsal, and regularization baselines on three of four datasets and achieves state-of-the-art $T_t$6 on all four. On Arxiv-CL, ADR matches the top ACL methods, and the minor drop is attributed to severe intra-task class imbalance (Song et al., 3 Apr 2026).

Ablation results isolate the role of HAM and the importance of plasticity. Removing HAM causes performance to collapse, with $T_t$7, which is attributed to drift. Replacing HAM with naïve model-merge schemes such as weight averaging, Fisher merging, and MAGMAX yields $T_t$8, compared with 90.7% for ADR on CS-CL. The $T_t$9 analysis shows that ACL methods, because they freeze the encoder, attain $T_t$0, whereas ADR’s free adaptation recovers near-upper-bound plasticity comparable to Bare. In feature-drift visualizations on CS-CL and CoraFull-CL, t-SNE plots of base-task embeddings before and after final training show EFC and DPCR drifting significantly, while ADR’s embeddings remain perfectly aligned, consistent with the HAM guarantee (Song et al., 3 Apr 2026).

6. Practical implications, limitations, and acronym disambiguation

The paper presents ADR as a fully privacy-preserving, exemplar-free continual graph learning framework with provable zero-forgetting and high plasticity. Because it stores only analytic summary statistics rather than raw exemplars, it avoids the $T_t$1 memory blow-up associated with exemplar storage and also avoids drift-compensation overhead (Song et al., 3 Apr 2026).

The stated limitation is that extreme class imbalance in some incremental tasks can bias individual task encoders, which then pollutes the merged model; this point is linked in the paper to Hammoud et al. 2024. Suggested extensions include imbalance-robust fusion, as well as applications to streaming recommendations, continual disease classification, anomaly detection on evolving graphs, and multi-modal graph continual learning. This suggests that the method is intended not merely as a graph-specific heuristic, but as an analytic template for privacy-constrained continual learning in settings where encoder adaptation and representation stability must be reconciled (Song et al., 3 Apr 2026).

The acronym ADR is not unique across arXiv literature. In control, it appears in Vector Field Orientation - Active Disturbance Rejection (VFO-ADR) for output-feedback path following of underactuated spatial vehicles (Łakomy et al., 2020). In phase change memory modeling, it appears as Analytic Drift Resistor, a device-level element for stabilized resistance drift in amorphous Ge$T_t$2Sb$T_t$3Te$T_t$4 (Kashem et al., 2022). Within continual graph learning, however, Analytic Drift Resister denotes the specific NECGL framework built from free task adaptation, hierarchical analytic encoder merging, and analytic classifier reconstruction (Song et al., 3 Apr 2026).