SPARK-IL: Spectral Retrieval-Augmented RAG for Knowledge-driven Deepfake Detection via Incremental Learning

Abstract: Detecting AI-generated images remains a significant challenge because detectors trained on specific generators often fail to generalize to unseen models; however, while pixel-level artifacts vary across models, frequency-domain signatures exhibit greater consistency, providing a promising foundation for cross-generator detection. To address this, we propose SPARK-IL, a retrieval-augmented framework that combines dual-path spectral analysis with incremental learning by utilizing a partially frozen ViT-L/14 encoder for semantic representations alongside a parallel path for raw RGB pixel embeddings. Both paths undergo multi-band Fourier decomposition into four frequency bands, which are individually processed by Kolmogorov-Arnold Networks (KAN) with mixture-of-experts for band-specific transformations before the resulting spectral embeddings are fused via cross-attention with residual connections. During inference, this fused embedding retrieves the $k$ nearest labeled signatures from a Milvus database using cosine similarity to facilitate predictions via majority voting, while an incremental learning strategy expands the database and employs elastic weight consolidation to preserve previously learned transformations. Evaluated on the UniversalFakeDetect benchmark across 19 generative models -- including GANs, face-swapping, and diffusion methods -- SPARK-IL achieves a 94.6\% mean accuracy, with the code to be publicly released at https://github.com/HessenUPHF/SPARK-IL.

• The paper introduces a dual-path spectral encoding framework that fuses pixel and semantic features to enable generator-agnostic deepfake detection.
• It employs multi-band KAN-FFT transformations and retrieval-augmented inference, achieving a mean accuracy of 94.6% on the UniversalFakeDetect benchmark.
• Experimental ablations demonstrate that integrating spectral analysis with non-parametric retrieval significantly enhances detection performance and supports incremental learning.

SPARK-IL: Spectral Retrieval-Augmented Deepfake Detection with Incremental Learning

Introduction

The proliferation of increasingly sophisticated generative models—ranging from GANs to diffusion models—has continually raised the bar for synthetic image detection in both forensics and security research. SPARK-IL introduces a knowledge-driven framework that combines dual-path frequency domain analysis with retrieval-augmented inference, addressing both the challenge of generator-agnostic deepfake detection and the need for incremental adaptation as new synthesis paradigms emerge. The model leverages robust, multi-band spectral embeddings processed through Kolmogorov-Arnold Networks (KAN), fuses pixel-level and semantic feature-level signals, and employs scalable non-parametric retrieval for open-world generalization.

Methodology

SPARK-IL’s architecture operates via two parallel spectral paths. The first path processes raw RGB pixel representations projected to a feature embedding, while the second path extracts semantic features using a partially frozen CLIP ViT-L/14 encoder. Both representations are independently decomposed in the frequency domain using 1D FFT and log-magnitude transformations. These spectral features are further partitioned into four non-overlapping frequency bands, each processed by KANs to perform band-specific, nonlinear transformations. This partitioning facilitates the capture of both global (low-frequency) and localized or aliasing (high-frequency) artifacts, preserving generator-agnostic signatures.

Figure 1: The SPARK-IL architecture: dual spectral encoding (RGB and ViT), multi-band KAN-FFT feature extraction, and cross-attention fusion for robust spectral embedding.

The resultant pixel-path and feature-path spectral embeddings are then fused via multi-head cross-attention, aligning pixel-level spectral anomalies with semantic inconsistencies. Residual connections inject complementary information from both paths, further regularizing the representation. This dual-path multi-band configuration captures inter-path and inter-frequency complementarity, which is quantitatively validated through an ablation study.

Figure 2: The multi-band KAN-FFT block: sequential FFT, log-magnitude computation, bandwise partitioning, KAN-based transformation, and fusion into a spectral embedding.

During inference, the fused spectral embedding of each input is indexed in a Milvus vector database with its ground-truth label and generator identifier. New queries retrieve their k-nearest neighbors via cosine similarity, and predictions are made by majority vote over the retrieved set. This retrieval-augmented prediction decouples detection from static parametric decision boundaries, enabling adaptation to new generative distributions as the database expands with new generators and samples.

Incremental learning is enabled through (i) database expansion with new sample embeddings, (ii) elastic weight consolidation to avoid catastrophic forgetting of prior frequency transformations, and (iii) distillation-based regularization to preserve embedding space alignment across updates. The architecture involves minimal fine-tuning—only the final transformer blocks—making it computationally efficient for continual learning.

Experimental Results

SPARK-IL is extensively evaluated on the UniversalFakeDetect benchmark, covering 19 generators including GANs, face swaps, low-level manipulations, perceptual-loss based methods, and recent diffusion models. Training is constrained exclusively to ProGAN, rigorously testing cross-generator generalization.

Quantitative Results: SPARK-IL achieves a mean accuracy of 94.6%, outperforming previous leading approaches such as REVEAL (+0.7%) and UniFD (+13.2%), and offering robust results across almost all generator categories. Accuracy consistently exceeds 99% for GAN families and surpasses 98% for DALL·E, with minor drops only on the most challenging low-level manipulation cases (e.g., SAN: 65.2%).

Figure 3: Detection accuracy and model efficiency comparison on the UniversalFakeDetect benchmark: SPARK-IL achieves top accuracy with competitive model size.

Retrieval Augmentation and t-SNE Analysis: Ablation results demonstrate that retrieval-based inference (majority voting over top-$K$ neighbors) yields the largest performance gain over a pure parametric classifier, with mean accuracy saturating near $K=15$ neighbors. This validates that the embedding space meaningfully clusters generator families and out-of-distribution samples.

Figure 4: t-SNE visualization highlights clear separation between real and synthetic clusters for various generator types in the SPARK-IL embedding space.

Computational Efficiency: With 315M parameters and a moderate GFLOPs cost, SPARK-IL delivers a superior accuracy-to-efficiency tradeoff. It uses fewer parameters than competing large ViT-based models (FatFormer, RINE) yet delivers higher mean accuracy.

Ablation and Component Analysis

Component-wise ablation quantifies the incremental value of each architectural feature:

• Adding pixel-level FFT increases mean accuracy by 3.7% over a ViT baseline.
• Feature-domain FFT contributes an additional 3.5%.
• KAN-based frequency transformations offer +1.2% improvement relative to MLPs, indicating their efficacy in modeling band-specific nonlinearities.
• Retrieval-augmented inference imparts the largest single improvement (+4.3%) by locally adapting to embedding neighborhoods.
• Dual-path, multi-band, and retrieval mechanisms collectively achieve the final 94.6% mAcc.

These findings indicate that both spectral multi-band processing and local, database-driven inference are critical for generator-agnostic generalization and robust incremental learning.

Implications and Future Directions

SPARK-IL’s framework demonstrates that frequency domain features, especially via dual-path analysis and adaptive band-specific modeling, offer greater cross-generator consistency than spatial signals or global feature representations alone. The retrieval-augmented paradigm moves beyond the rigidity of static classifiers, naturally supporting lifelong learning and adaptation to continually evolving generative models.

Practically, this makes SPARK-IL suitable for deployment in settings where detection pipelines must be updated frequently and with minimal retraining. The parametric-nonparametric hybrid design allows both high-fidelity recognition and scalable expansion.

Several theoretical and practical implications arise:

• Retrieval-augmented architecture aligns with open-world settings, where novel generator distributions are expected.
• Multi-band KAN transformation motivates research into more adaptive/learnable frequency partitioning schemes and multi-scale mechanisms.
• Limitations remain for spatially localized or patch-wise artifacts, suggesting future extensions to local spectral modeling or patch embeddings.

As generative models continue to minimize spatial and temporal artifacts through adversarial training and architectural innovations, deepfake forensics must further emphasize robust spectral representations, scalable retrieval, and incremental model updating. Extensions to video, multimodal, or context-grounded detection are also promising directions.

Conclusion

SPARK-IL establishes a new state-of-the-art for universal deepfake detection by coupling dual-path, multi-band spectral embedding with retrieval-augmented incremental learning. The empirical results affirm the utility of frequency-based analysis for generator-agnostic detection, the necessity of local evidence aggregation, and the feasibility of scalable knowledge integration via database expansion. The framework provides a strong foundation for ongoing work in robust, adaptive synthetic media forensics.