DG-KD: Domain-Group KAN Detector

• The paper introduces DG-KD, a neural module that uses groupwise RBF activations and data-free replay to address catastrophic forgetting in face forgery detection.
• DG-KD partitions high-dimensional features into groups, applying domain-specific local activations to enable continual adaptation with minimal interference.
• Empirical results demonstrate high accuracy and minimal forgetting, achieving up to 99.48% memory reduction compared to traditional replay-based methods.

The Domain-Group KAN Detector (DG-KD) is a neural module designed for continual face forgery detection problems, enabling robust adaptation to new domains or forgery types while minimizing catastrophic forgetting. It is a component of the KAN-CFD framework, which integrates DG-KD with data-free replay and feature separation mechanisms to achieve high continual-learning performance with minimal storage and computational overhead (Zhang et al., 5 Aug 2025).

1. Rationale and Problem Setting

Continual face forgery detection is formulated as a domain-incremental learning scenario where the binary label space $\mathcal{Y}_t = \{\text{Real}, \text{Fake}\}$ is constant, but the data distribution $p(\mathcal{X}_i)$ varies for each task/domain $\mathcal{D}_t$. The principal challenge is to learn new forgery types without incurring severe performance drops on previously seen domains—a phenomenon termed catastrophic forgetting.

DG-KD builds on Kolmogorov-Arnold Networks (KANs), which feature locally plastic activation functions: parameter updates in one input region only affect function behavior nearby, not globally. However, standard KANs employ B-spline activations, which are inefficient for high-dimensional image features and do not scale well. Conversely, conventional activations suitable for images (e.g., rational or nonlocal bases) lack the local adaptivity needed for continual learning.

DG-KD addresses this by combining locally supported radial basis function (RBF) activations with a group-wise domain partitioning strategy, facilitating high-dimensional feature modeling while retaining the locality essential for catastrophic forgetting mitigation.

2. Mathematical Structure

2.1 KAN Activation and Layer

A standard KAN layer maps an input $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$ to an output via a sum of learnable univariate functions $\phi_{ij}$: $$\mathrm{KAN}(\mathbf{x})= \begin{bmatrix} \sum_{i=1}^{d_{\text{in}}}\phi_{1,i}(x_i) \ \vdots \ \sum_{i=1}^{d_{\text{in}}}\phi_{d_{\text{out}},i}(x_i) \end{bmatrix}$$ Each $\phi(x)$ is parameterized by spline or similar local bases.

2.2 Local RBF Activation

DG-KD replaces spline bases with radial basis functions: $$\phi_{ij}(x_i)=\exp\!\left(-\frac{(x_i-c_{ij})^2}{2\sigma_{ij}^2}\right)$$ where $c_{ij}$ and $\sigma_{ij}$ denote center and width, enabling the activation to be significant only locally in feature space.

Correspondingly, a layer’s output can be written: $p(\mathcal{X}_i)$0

2.3 Domain Grouping Mechanism

The input feature vector is partitioned into $p(\mathcal{X}_i)$1 groups of size $p(\mathcal{X}_i)$2. Group index assignment is $p(\mathcal{X}_i)$3.

Within each group, all feature dimensions share a domain-specific local activation: $p(\mathcal{X}_i)$4 A DG-Layer for domain $p(\mathcal{X}_i)$5 applies groupwise RBFs, accumulated as: $p(\mathcal{X}_i)$6 with $p(\mathcal{X}_i)$7 a learnable weight matrix.

The overall DG-KD is the sum over DG-Layers: $p(\mathcal{X}_i)$8 This setup ensures that each task/domain utilizes a distinct, locally supported region of the activation manifold.

2.4 Locality and Knowledge Retention

Learning for a new domain only updates basis functions within the region associated with that domain: $p(\mathcal{X}_i)$9 This formalizes retention: earlier tasks’ knowledge, represented by nonoverlapping local regions, remains stable throughout subsequent task learning.

3. Integration with Model Pipeline

DG-KD is implemented as the head of a ConvNeXt-B convolutional neural network. The system operates as follows:

1. ConvNeXt-B backbone extracts high-dimensional image features.
2. These features are processed by the DG-KD module—a stack of DG-Layers (one per domain/task) with groupwise RBF activations.
3. Each group shares local activation parameters within a DG-Layer.
4. The classifier produces a binary real/fake output.

A canonical example with $\mathcal{D}_t$0, $\mathcal{D}_t$1, $\mathcal{D}_t$2 demonstrates that each DG-Layer contains two RBF groups, each responsible for two input features, reducing redundancy in feature processing.

The feature memory is limited to 500 feature representatives, achieving data-free replay by storing latent features instead of raw input images, yielding a reported $\mathcal{D}_t$3 memory reduction compared to conventional replay-based approaches (Zhang et al., 5 Aug 2025).

4. Objective Function and Optimization

Training comprises a composite loss: $\mathcal{D}_t$4 where:

• $\mathcal{D}_t$5 is binary cross-entropy,
• $\mathcal{D}_t$6 for supervised contrastive loss $\mathcal{D}_t$7,
• $\mathcal{D}_t$8 for feature-level knowledge distillation loss $\mathcal{D}_t$9,
• temperature for contrastive loss $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$0.

The supervised contrastive loss is: $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$1 where each real/fake per task is a unique class, giving $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$2 classes.

Knowledge distillation constrains backbone drift: $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$3

KAN Drift Compensation Projection (KDCP) is used for data-free feature alignment: $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$4

Optimization is performed using Adam ($\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$5, $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$6), with learning rates $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$7 for the main model and $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$8 for KDCP.

5. Locality, Plasticity, and Overlap Avoidance

Catastrophic forgetting in KANs arises when input features for multiple domains overlap, risking shared activation regions and global interference. DG-KD mitigates this in two ways:

• Local plasticity: The RBF basis ensures that updates for a given domain modify only its neighborhood in activation space:

$\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$9

with $\phi_{ij}$0 for $\phi_{ij}$1 if feature-space separation is achieved.

• Domain separation in feature space: FS-KDCP aligns memory features and supervised contrastive learning enforces separation, further minimizing domain overlap in DG-KD activations. This ensures that the network restructures only the relevant local regions during new-task learning, preserving stored knowledge from prior domains.

6. Empirical Findings and Ablations

DG-KD, within KAN-CFD, demonstrates strong resistance to forgetting and high accuracy on multiple continual learning benchmarks:

Protocol	Metric	DG-KD/KAN-CFD Performance
Dataset-Incremental ($\phi_{ij}$2FF++, DFDC-P, DFD, CDF2$\phi_{ij}$3)	Avg. Accuracy	$\phi_{ij}$4
	Avg. Forgetting	$\phi_{ij}$5
Forgery-Type Incremental ($\phi_{ij}$6Hybrid, FR, FS, EFS$\phi_{ij}$7)	Avg. AUC	$\phi_{ij}$8
	Final Forgetting	$\phi_{ij}$9
Long-Sequence (DF40, 10 tasks)	Highest accuracy, lowest forgetting	Yes

Ablation studies reveal:

• Relying solely on $$\mathrm{KAN}(\mathbf{x})= \begin{bmatrix} \sum_{i=1}^{d_{\text{in}}}\phi_{1,i}(x_i) \ \vdots \ \sum_{i=1}^{d_{\text{in}}}\phi_{d_{\text{out}},i}(x_i) \end{bmatrix}$$0 causes severe forgetting ($$\mathrm{KAN}(\mathbf{x})= \begin{bmatrix} \sum_{i=1}^{d_{\text{in}}}\phi_{1,i}(x_i) \ \vdots \ \sum_{i=1}^{d_{\text{in}}}\phi_{d_{\text{out}},i}(x_i) \end{bmatrix}$$1 by end of training).
• Adding $$\mathrm{KAN}(\mathbf{x})= \begin{bmatrix} \sum_{i=1}^{d_{\text{in}}}\phi_{1,i}(x_i) \ \vdots \ \sum_{i=1}^{d_{\text{in}}}\phi_{d_{\text{out}},i}(x_i) \end{bmatrix}$$2 or $$\mathrm{KAN}(\mathbf{x})= \begin{bmatrix} \sum_{i=1}^{d_{\text{in}}}\phi_{1,i}(x_i) \ \vdots \ \sum_{i=1}^{d_{\text{in}}}\phi_{d_{\text{out}},i}(x_i) \end{bmatrix}$$3 improves retention, with the full loss achieving the best results.
• DG-KD outperforms MLP, standard KAN, and GroupKAN, with its groupwise RBF architecture being critical for high-dimensional images and locality.

7. Technical Significance

DG-KD is a locality-preserving adaptation of Kolmogorov-Arnold Networks tailored for domain-incremental settings in high-dimensional image analysis. Its core technical contributions are:

1. Replacement of B-splines with local RBFs to scale KANs to image features while preserving plasticity.
2. Groupwise parameter sharing—partitioning features into groups each handled by a separate local basis—yielding parameter efficiency and locality.
3. Integration with KDCP for data-free feature replay and enforced domain separation, maintaining high performance with minimal memory.

DG-KD exemplifies an architecture that combines local compositionality, continual-learning robustness, and computational feasibility for modern face forgery detection tasks, achieving strong empirical performance without the need for full data replay (Zhang et al., 5 Aug 2025).