---
title: 'ADR: Exemplar-Free Continual Graph Learning'
url: https://www.emergentmind.com/topics/analytic-drift-resister-adr
type: topic
---

# ADR: Exemplar-Free Continual Graph Learning

Searching arXiv for the primary paper and closely related continual graph learning works.
Analytic Drift Resister (ADR) is a continual graph learning framework introduced for Non-Exemplar Continual Graph Learning (NECGL), a setting in which privacy is preserved by retaining only class-level prototype statistics rather than raw graph examples. ADR is designed to address two coupled difficulties in this regime: feature drift induced by encoder updates, and the attenuation of model plasticity associated with frozen Analytic Continual Learning (ACL) encoders. The framework combines free task-wise adaptation by backpropagation, Hierarchical Analytic Merging (HAM) of GNN layers via ridge regression, and Analytic Classifier Reconstruction (ACR), with the stated objective of achieving exemplar-free, privacy-safe, zero-forgetting continual graph learning while remaining competitive with state-of-the-art baselines on node classification benchmarks [2604.02633].

## 1. Problem setting and motivation

In class-incremental graph learning, a sequence of tasks \(T_0,T_1,\ldots,T_{n-1}\) is observed, with each task \(T_t=(G_t,Y_t)\), where \(G_t=(V_t,E_t)\) is a graph with node features \(X_t\in\mathbb{R}^{|V_t|\times d}\) and adjacency \(A_t\), and \(Y_t\) are its labels. The label spaces are disjoint across tasks. Under Non-Exemplar constraints, once training on \(T_t\) is finished, no raw nodes or subgraphs from \(T_t\) may be stored; only class-level prototype statistics such as Gaussian means and covariances are cached to represent past tasks [2604.02633].

This privacy-preserving design avoids storing individual nodes or edges, but it introduces a structural instability: as the model encoder updates on new tasks, cached prototypes become misaligned with the current embedding space. The paper identifies this misalignment as feature drift. In parallel, ACL methods avoid feature drift by freezing a pre-trained graph encoder \(F_\theta(\cdot)\) and solving class-incremental learning analytically for the linear classifier through recursive least squares. In that formulation, only the accumulated feature autocorrelation \(R=\sum H_i^\top H_i\) and cross-correlation \(Q=\sum H_i^\top Y_i\) are stored, which guarantees no feature drift and perfect privacy. The drawback is that freezing \(\theta\) precludes adaptation to newly arriving task distributions, particularly when pre-training and downstream tasks differ markedly in graph structure or node attributes [2604.02633].

ADR is positioned precisely at this junction. It is proposed to break the frozen pre-trained constraint, preserve privacy without rehearsal, and resist the drift that ordinarily follows from parameter updates.

## 2. Three-stage learning pipeline

ADR processes each arriving task \(T_t\) through three stages: **(A) Incremental Task Adaptation**, **(B) Hierarchical Analytic Merging (HAM)**, and **(C) Analytic Classifier Reconstruction (ACR)** [2604.02633].

In Incremental Task Adaptation, a GNN encoder \(F_{\theta_t}\) with \(K\) layers and a linear classifier \(g_{\phi_t}\) are maintained. For task \(T_t\), the forward computation is

$$
h_{t,0}=X_t
$$

$$
h_{t,k}=GNN_{t,k}(h_{t,k-1},A_t),\quad k=1\ldots K
$$

$$
p(y|x)=\mathrm{softmax}(g_{\phi_t}(h_{t,K})).
$$

The task loss is the ordinary supervised cross-entropy objective

$$
L_{ce}(\theta_t,\phi_t)=\mathbb{E}_{(x,y)\sim T_t}[-\log p(y|x)].
$$

Both \(\theta_t\) and \(\phi_t\) are updated by backpropagation, explicitly with no regularization. This stage is intended to fully exploit plasticity by allowing the encoder to adapt to the current graph distribution rather than remaining frozen [2604.02633].

The algorithmic workflow specifies the per-task inputs as the encoder \(F_{\theta_t}\), classifier \(g_{\phi_t}\), memory banks \(\{R_{t-1,k}^\theta,Q_{t-1,k}^\theta\}_k\) and \(\{R_{t-1}^\phi,Q_{t-1}^\phi\}\), learning rate \(\eta_t\), and epochs \(E\). After \(E\) epochs of free adaptation, HAM accumulates layer-wise sufficient statistics and reconstructs merged encoder weights, and ACR updates the classifier statistics and solves the corresponding analytic reconstruction. The output is the merged model \(\{F_{\theta_t^M},g_{\phi_t^M}\}\) together with the memory banks [2604.02633].

## 3. Hierarchical analytic merging and classifier reconstruction

HAM is the core mechanism used to neutralize feature drift after free adaptation. After training on \(T_t\), encoders from all tasks \(\{\theta_0,\ldots,\theta_t\}\) are available. The goal is to construct a single merged encoder \(F_{\theta_t^M}\) that reproduces the layer-wise behavior of each task-specific encoder. For the linear weight matrix \(W_{\theta_i,k}\) of layer \(k\), the paper defines, for each node \(v\) in \(T_i\),

$$
\hat h_{i,k}(v)=\sum_{u\in N(v)} \omega_{vu}\, h_{i,k-1}(u)\in\mathbb{R}^{d_k^{in}}
$$

and

$$
h_{i,k}(v)=\sigma(W_{\theta_i,k}\cdot \hat h_{i,k}(v)).
$$

Let \(H_{i,k}\in\mathbb{R}^{n_i\times d_k^{out}}\) stack the post-activations \(h_{i,k}(v)\), and let \(\hat H_{i,k}\in\mathbb{R}^{n_i\times d_k^{in}}\) stack the pre-activations \(\hat h_{i,k}(v)\). HAM solves the joint ridge regression

$$
W_{\theta_t^M,k}^*=
\arg\min_W
\left\|
[H_{0,k};\ldots;H_{t,k}]-
[\hat H_{0,k};\ldots;\hat H_{t,k}]\,W
\right\|_F^2
+\gamma \|W\|_F^2.
$$

Setting the gradient to zero yields the closed form

$$
W_{\theta_t^M,k}^*=
\left(\sum_{i=0}^t \hat H_{i,k}^\top \hat H_{i,k}+\gamma I\right)^{-1}
\left(\sum_{i=0}^t \hat H_{i,k}^\top H_{i,k}\right).
$$

Only the layer-wise autocorrelation and cross-correlation terms must be stored:

$$
R_{t,k}^\theta=\sum \hat H_i^\top \hat H_i,\qquad
Q_{t,k}^\theta=\sum \hat H_i^\top H_i.
$$

The paper states that solving this problem produces \(\theta_t^M\) that is provably equivalent to jointly training on all data seen so far, and therefore yields absolute resistance to feature drift [2604.02633].

ACR applies the same analytic principle to the final linear classifier. Given the merged encoder \(F_{\theta_t^M}\), the classifier \(g_{\phi_t^M}(h)=hW_\phi\) is reconstructed through ridge regression over all tasks’ merged embeddings \(H_i^M=F_{\theta_t^M}(V_i)\):

$$
W_{\phi_t^M}^*=
\arg\min_W
\left\|
[Y_0;\ldots;Y_t]-[H_0^M;\ldots;H_t^M]\,W
\right\|_F^2
+\gamma\|W\|_F^2,
$$

with solution

$$
W_{\phi_t^M}^*=
\left(\sum H_i^{M\top}H_i^M+\gamma I\right)^{-1}
\left(\sum H_i^{M\top}Y_i\right).
$$

An optional random feature buffer expansion,

$$
H_i^b=\sigma(H_i^M W_\psi),
$$

may be inserted to improve separability, with the paper explicitly invoking Cover’s theorem. In that case, the stored classifier statistics become \(R_t^\phi=\sum H_i^{b\top}H_i^b\) and \(Q_t^\phi=\sum H_i^{b\top}Y_i\), still without retaining any raw data [2604.02633].

## 4. Theoretical properties

ADR is accompanied by two explicit propositions. **Proposition 1 (Absolute Feature-Drift Resistance)** states that the merged encoder \(\theta_t^M\) produced by HAM at step \(t\) is exactly the minimizer of

$$
\sum_{i=0}^t \mathbb{E}_{x\sim T_i}\|F_{\theta_i}(x)-F_{\theta_t^M}(x)\|^2+\gamma\|\theta_t^M\|^2,
$$

and is therefore exactly equivalent to joint training on all tasks up to \(t\). The stated consequence is that prototypes or embeddings extracted from the merged encoder never drift [2604.02633].

**Proposition 2 (Zero-Forgetting Class Incremental)** states that the classifier \(W_{\phi_t^M}\) obtained through ACR solves the joint ridge regression over all tasks’ embeddings. Its accuracy on prior tasks is therefore identical to that of a model trained jointly on all data, which the paper characterizes as zero forgetting [2604.02633].

The proof sketches proceed by forming the combined Frobenius-norm objectives associated with HAM and ACR, verifying convexity, setting gradients to zero, and obtaining the closed forms via the Matrix Inverse Lemma. Within the paper’s presentation, the central claim is that free adaptation and analytic reconstruction are not treated as competing objectives; rather, free adaptation supplies task plasticity, while the analytic stages restore global consistency across tasks.

## 5. Benchmarks, baselines, and observed behavior

The empirical evaluation uses four node classification benchmarks: **CS-CL** with 18,333 nodes, 163,788 edges, and 15 classes; **CoraFull-CL** with 19,793 nodes, 126,842 edges, and 70 classes; **Arxiv-CL** with 169,343 nodes, 1,166,243 edges, and 40 classes; and **Reddit-CL** with 232,965 nodes, 114,615,892 edges, and 40 classes [2604.02633].

Three evaluation metrics are defined. The average incremental accuracy is

$$
A_{avg}=\frac{1}{N}\sum_{t=0}^{N-1} A_t,
$$

the final accuracy \(A_f\) is accuracy on all classes after the final task, and the learning accuracy

$$
A_l=\frac{1}{N}\sum_{t=0}^{N-1} A_{t,t}
$$

is used as a plasticity measure. Baselines span multiple families: regularization methods (**LwF, EWC, MAS, TWP**), rehearsal methods (**CaT, ER-GNN**), non-exemplar methods (**POLO, EFC**), analytic ACL methods (**ACIL, DS-AL, DPCR**), and two reference conditions (**Bare** for fine-tune only, and **Joint** as oracle) [2604.02633].

The reported global testing results show the following \(A_{avg}/A_f\) values for ADR: **90.7 / 87.2** on CS-CL, **67.5 / 60.0** on CoraFull-CL, **39.7 / 34.7** on Arxiv-CL, and **82.5 / 75.5** on Reddit-CL. The corresponding Joint \(A_f\) values are **88.9**, **61.6**, **42.1**, and **79.8**. The paper states that ADR outperforms all existing NECGL, ACL, rehearsal, and regularization baselines on three of four datasets and achieves state-of-the-art \(A_{avg}\) on all four. On Arxiv-CL, ADR matches the top ACL methods, and the minor drop is attributed to severe intra-task class imbalance [2604.02633].

Ablation results isolate the role of HAM and the importance of plasticity. Removing HAM causes performance to collapse, with \(A_{avg}\approx 50\%\), which is attributed to drift. Replacing HAM with naïve model-merge schemes such as weight averaging, Fisher merging, and MAGMAX yields \(A_{avg}\approx 65\text{–}70\%\), compared with **90.7%** for ADR on CS-CL. The \(A_l\) analysis shows that ACL methods, because they freeze the encoder, attain \(A_l\approx 40\text{–}50\%\), whereas ADR’s free adaptation recovers near-upper-bound plasticity comparable to Bare. In feature-drift visualizations on CS-CL and CoraFull-CL, t-SNE plots of base-task embeddings before and after final training show EFC and DPCR drifting significantly, while ADR’s embeddings remain perfectly aligned, consistent with the HAM guarantee [2604.02633].

## 6. Practical implications, limitations, and acronym disambiguation

The paper presents ADR as a fully privacy-preserving, exemplar-free continual graph learning framework with provable zero-forgetting and high plasticity. Because it stores only analytic summary statistics rather than raw exemplars, it avoids the \(O(N)\) memory blow-up associated with exemplar storage and also avoids drift-compensation overhead [2604.02633].

The stated limitation is that extreme class imbalance in some incremental tasks can bias individual task encoders, which then pollutes the merged model; this point is linked in the paper to Hammoud et al. 2024. Suggested extensions include imbalance-robust fusion, as well as applications to streaming recommendations, continual disease classification, anomaly detection on evolving graphs, and multi-modal graph continual learning. This suggests that the method is intended not merely as a graph-specific heuristic, but as an analytic template for privacy-constrained continual learning in settings where encoder adaptation and representation stability must be reconciled [2604.02633].

The acronym **ADR** is not unique across arXiv literature. In control, it appears in **Vector Field Orientation - Active Disturbance Rejection (VFO-ADR)** for output-feedback path following of underactuated spatial vehicles [2001.01963]. In phase change memory modeling, it appears as **Analytic Drift Resistor**, a device-level element for stabilized resistance drift in amorphous Ge\(_2\)Sb\(_2\)Te\(_5\) [2210.14035]. Within continual graph learning, however, **Analytic Drift Resister** denotes the specific NECGL framework built from free task adaptation, hierarchical analytic encoder merging, and analytic classifier reconstruction [2604.02633].

Source: https://www.emergentmind.com/topics/analytic-drift-resister-adr