---
title: 'DG-KD: Domain-Group KAN Detector'
url: https://www.emergentmind.com/topics/domain-group-kan-detector-dg-kd
type: topic
---

# DG-KD: Domain-Group KAN Detector

The Domain-Group KAN Detector (DG-KD) is a neural module designed for continual face forgery detection problems, enabling robust adaptation to new domains or forgery types while minimizing catastrophic forgetting. It is a component of the KAN-CFD framework, which integrates DG-KD with data-free replay and feature separation mechanisms to achieve high continual-learning performance with minimal storage and computational overhead [2508.03189].

## 1. Rationale and Problem Setting

Continual face forgery detection is formulated as a domain-incremental learning scenario where the binary label space $\mathcal{Y}_t = \{\text{Real}, \text{Fake}\}$ is constant, but the data distribution $p(\mathcal{X}_i)$ varies for each task/domain $\mathcal{D}_t$. The principal challenge is to learn new forgery types without incurring severe performance drops on previously seen domains—a phenomenon termed catastrophic forgetting.

DG-KD builds on Kolmogorov-Arnold Networks (KANs), which feature locally plastic activation functions: parameter updates in one input region only affect function behavior nearby, not globally. However, standard KANs employ B-spline activations, which are inefficient for high-dimensional image features and do not scale well. Conversely, conventional activations suitable for images (e.g., rational or nonlocal bases) lack the local adaptivity needed for continual learning.

DG-KD addresses this by combining locally supported radial basis function (RBF) activations with a group-wise domain partitioning strategy, facilitating high-dimensional feature modeling while retaining the locality essential for catastrophic forgetting mitigation.

## 2. Mathematical Structure

### 2.1 KAN Activation and Layer

A standard KAN layer maps an input $\mathbf{x} \in \mathbb{R}^{d_{\text{in}}}$ to an output via a sum of learnable univariate functions $\phi_{ij}$:
\[
\mathrm{KAN}(\mathbf{x})= 
\begin{bmatrix}
\sum_{i=1}^{d_{\text{in}}}\phi_{1,i}(x_i) \\
\vdots \\
\sum_{i=1}^{d_{\text{in}}}\phi_{d_{\text{out}},i}(x_i)
\end{bmatrix}
\]
Each $\phi(x)$ is parameterized by spline or similar local bases.

### 2.2 Local RBF Activation

DG-KD replaces spline bases with radial basis functions:
\[
\phi_{ij}(x_i)=\exp\!\left(-\frac{(x_i-c_{ij})^2}{2\sigma_{ij}^2}\right)
\]
where $c_{ij}$ and $\sigma_{ij}$ denote center and width, enabling the activation to be significant only locally in feature space.

Correspondingly, a layer’s output can be written:
\[
f(\mathbf{x}) = 
\begin{bmatrix}
\sum_{i=1}^{d_{\text{in}}} \phi_{1i}(x_i) & \dots & \sum_{i=1}^{d_{\text{in}}} \phi_{d_{\text{out}}i}(x_i)
\end{bmatrix}
\]

### 2.3 Domain Grouping Mechanism

The input feature vector is partitioned into $g$ groups of size $d_g = \left\lfloor \frac{\dim(\mathbf{x})}{g} \right\rfloor$. Group index assignment is $g(i) = \left\lfloor \frac{i}{d_g}\right\rfloor$.

Within each group, all feature dimensions share a domain-specific local activation:
\[
\phi_{\lfloor i/d_g\rfloor,t}(x_i)
\]
A DG-Layer for domain $t$ applies groupwise RBFs, accumulated as:
\[
\mathrm{DG\text{-}Layer}_t(\mathbf{x}) = W\, \begin{bmatrix} 
\phi_{\lfloor 1/d_g\rfloor,t}(x_1) & \dots & \phi_{\lfloor d_{\text{in}}/d_g\rfloor,t}(x_{d_{\text{in}}})
\end{bmatrix}^{\top}
\]
with $W$ a learnable weight matrix.

The overall DG-KD is the sum over DG-Layers:
\[
\mathrm{DG\text{-}KD}(\mathbf{x}) = \sum_{k=1}^{t}\mathrm{DG\text{-}Layer}_k(\mathbf{x})
\]
This setup ensures that each task/domain utilizes a distinct, locally supported region of the activation manifold.

### 2.4 Locality and Knowledge Retention

Learning for a new domain only updates basis functions within the region associated with that domain:
\[
\Delta \phi_{ij}(x) = 0 \quad \text{for } x \notin \Omega_t
\]
This formalizes retention: earlier tasks’ knowledge, represented by nonoverlapping local regions, remains stable throughout subsequent task learning.

## 3. Integration with Model Pipeline

DG-KD is implemented as the head of a ConvNeXt-B convolutional neural network. The system operates as follows:

1. ConvNeXt-B backbone extracts high-dimensional image features.
2. These features are processed by the DG-KD module—a stack of DG-Layers (one per domain/task) with groupwise RBF activations.
3. Each group shares local activation parameters within a DG-Layer.
4. The classifier produces a binary real/fake output.

A canonical example with $d_{\text{in}}=4$, $d_{\text{out}}=4$, $g=2$ demonstrates that each DG-Layer contains two RBF groups, each responsible for two input features, reducing redundancy in feature processing.

The feature memory is limited to 500 feature representatives, achieving data-free replay by storing latent features instead of raw input images, yielding a reported $99.48\%$ memory reduction compared to conventional replay-based approaches [2508.03189].

## 4. Objective Function and Optimization

Training comprises a composite loss:
\[
\mathcal{L}_{\text{Overall}} = \mathcal{L}_{CLS} + \lambda_1 \mathcal{L}_{SC} + \lambda_2 \mathcal{L}_{KD}
\]
where:
- $\mathcal{L}_{CLS}$ is binary cross-entropy,
- $\lambda_1 = 2$ for supervised contrastive loss $\mathcal{L}_{SC}$,
- $\lambda_2 = 1$ for feature-level knowledge distillation loss $\mathcal{L}_{KD}$,
- temperature for contrastive loss $\tau = 0.1$.

The supervised contrastive loss is:
\[
\mathcal{L}_{SC} = -\frac{1}{N} \sum_{i=1}^{N} \log \left( \frac{ \exp\left(\mathbf{f}_i\cdot \mathbf{f}_j/\tau\right) }{ \sum_{k=1}^{N}\mathbb{I}_{[d_i\neq d_k]}\exp\left(\mathbf{f}_i\cdot \mathbf{f}_k/\tau\right) } \right)
\]
where each real/fake per task is a unique class, giving $2T$ classes.

Knowledge distillation constrains backbone drift:
\[
\mathcal{L}_{KD} = \frac{1}{N}\sum_{i=1}^{N} \mathrm{MSE}\!\left(f_{\theta}^{t-1}(x_i),f_{\theta}^{t}(x_i)\right)
\]

**KAN Drift Compensation Projection (KDCP)** is used for data-free feature alignment:
\[
\mathcal{L}_{Align} = \frac{1}{N}\sum_{i=1}^{N} \mathrm{MSE}\!\left( p_{KAN}^{t}\!\left(f_{\theta}^{t-1}(x_i)\right), f_{\theta}^{t}(x_i) \right)
\]

Optimization is performed using Adam ($\beta_1 = 0.9$, $\beta_2 = 0.999$), with learning rates $2 \times 10^{-4}$ for the main model and $5 \times 10^{-4}$ for KDCP.

## 5. Locality, Plasticity, and Overlap Avoidance

Catastrophic forgetting in KANs arises when input features for multiple domains overlap, risking shared activation regions and global interference. DG-KD mitigates this in two ways:

- **Local plasticity:** The RBF basis ensures that updates for a given domain modify only its neighborhood in activation space:
  \[
  \Delta \phi^{(t)}_{ij}(x)\neq 0 \text{ only on } \mathcal{Z}_t
  \]
  with $\mathcal{Z}_i \cap \mathcal{Z}_j \approx \varnothing$ for $i \neq j$ if feature-space separation is achieved.

- **Domain separation in feature space:** FS-KDCP aligns memory features and supervised contrastive learning enforces separation, further minimizing domain overlap in DG-KD activations. This ensures that the network restructures only the relevant local regions during new-task learning, preserving stored knowledge from prior domains.

## 6. Empirical Findings and Ablations

DG-KD, within KAN-CFD, demonstrates strong resistance to forgetting and high accuracy on multiple continual learning benchmarks:

| Protocol                                      | Metric               | DG-KD/KAN-CFD Performance |
|------------------------------------------------|----------------------|---------------------------|
| Dataset-Incremental ($[$FF++, DFDC-P, DFD, CDF2$]$) | Avg. Accuracy        | $91.64\%$                 |
|                                                | Avg. Forgetting      | $4.08\%$                  |
| Forgery-Type Incremental ($[$Hybrid, FR, FS, EFS$]$) | Avg. AUC             | $94.40\%$                 |
|                                                | Final Forgetting     | $2.60\%$                  |
| Long-Sequence (DF40, 10 tasks)                 | Highest accuracy, lowest forgetting | Yes  |

Ablation studies reveal:
- Relying solely on $\mathcal{L}_{CLS}$ causes severe forgetting ($AF = 31.35\%$ by end of training).
- Adding $\mathcal{L}_{KD}$ or $\mathcal{L}_{SC}$ improves retention, with the full loss achieving the best results.
- DG-KD outperforms MLP, standard KAN, and GroupKAN, with its groupwise RBF architecture being critical for high-dimensional images and locality.

## 7. Technical Significance

DG-KD is a locality-preserving adaptation of Kolmogorov-Arnold Networks tailored for domain-incremental settings in high-dimensional image analysis. Its core technical contributions are:
1. Replacement of B-splines with local RBFs to scale KANs to image features while preserving plasticity.
2. Groupwise parameter sharing—partitioning features into groups each handled by a separate local basis—yielding parameter efficiency and locality.
3. Integration with KDCP for data-free feature replay and enforced domain separation, maintaining high performance with minimal memory.

DG-KD exemplifies an architecture that combines local compositionality, continual-learning robustness, and computational feasibility for modern face forgery detection tasks, achieving strong empirical performance without the need for full data replay [2508.03189].

Source: https://www.emergentmind.com/topics/domain-group-kan-detector-dg-kd