Scaling Up AI-Generated Image Detection via Generator-Aware Prototypes

Abstract: The pursuit of a universal AI-generated image (AIGI) detector often relies on aggregating data from numerous generators to improve generalization. However, this paper identifies a paradoxical phenomenon we term the Benefit then Conflict dilemma, where detector performance stagnates and eventually degrades as source diversity expands. Our systematic analysis, diagnoses this failure by identifying two core issues: severe data-level heterogeneity, which causes the feature distributions of real and synthetic images to increasingly overlap, and a critical model-level bottleneck from fixed, pretrained encoders that cannot adapt to the rising complexity. To address these challenges, we propose Generator-Aware Prototype Learning (GAPL), a framework that constrain representation with a structured learning paradigm. GAPL learns a compact set of canonical forgery prototypes to create a unified, low-variance feature space, effectively countering data heterogeneity.To resolve the model bottleneck, it employs a two-stage training scheme with Low-Rank Adaptation, enhancing its discriminative power while preserving valuable pretrained knowledge. This approach establishes a more robust and generalizable decision boundary. Through extensive experiments, we demonstrate that GAPL achieves state-of-the-art performance, showing superior detection accuracy across a wide variety of GAN and diffusion-based generators. Code is available at https://github.com/UltraCapture/GAPL

• The paper’s main contribution is revealing that increased generator diversity degrades detection performance and proposing GAPL to counteract this effect.
• It employs a two-stage method, extracting low-variance prototypes with PCA and adapting fixed encoders via LoRA to improve real-fake separation.
• Empirical results show that GAPL achieves state-of-the-art accuracy (90.4%) and robustness to post-processing, setting new benchmarks in the field.

Scaling Up AI-Generated Image Detection via Generator-Aware Prototypes

Problem Formulation and Motivation

The proliferation of high-fidelity generative models has heightened the need for robust detectors capable of reliably distinguishing AI-generated images (AIGI) from real images, independent of the underlying generative architectures. The conventional approach to generalization aggregates samples from numerous generators to increase feature diversity, with the expectation that detectors trained in this setting will generalize better to novel generators. This work systematically demonstrates a critical limitation in this paradigm: the "Benefit then Conflict" effect, where further increasing generator diversity eventually degrades detection performance.

Empirical analysis attributes this performance plateau—and ultimate decline—to two compounding factors: (1) severe data-level heterogeneity, with features from real and generated distributions growing overlap as more generative models are included, and (2) a model-level bottleneck due to reliance on fixed, pretrained encoders (e.g., CLIP) whose embedding space cannot adequately separate highly diverse forgery patterns.

A t-SNE visualization of fixed encoder embeddings (CLIP) clearly demonstrates this phenomenon: with a single generator, the boundary between real and fake images is clear, while aggregation from thousands of generators reveals profound overlap and inseparability.

Figure 1: T-SNE projections illustrating how real-vs-fake separability degenerates when scaling from a single generator to thousands.

Generator-Aware Prototype Learning (GAPL)

Methodological Innovation

To address the inherent limitations of unconstrained data aggregation, the paper introduces Generator-Aware Prototype Learning (GAPL), a structured learning paradigm rooted in two interdependent objectives: (1) compacting the feature representation for generated images by mapping them onto a low-variance, prototype-defined space, and (2) overcoming the restrictions of fixed feature extractors via targeted encoder adaptation using Low-Rank Adaptation (LoRA).

The GAPL framework consists of two stages:

1. Prototype Construction: Using a carefully selected subset of generators (spanning canonical GAN, diffusion, and commercial models), an MLP is trained to project embeddings to a discriminative subspace. Principal Component Analysis (PCA) is applied on real and generated subsets independently to extract top-N components that summarize prototypical real and forgery artifacts.
2. Prototype Mapping and Encoder Adaptation: The pretrained encoder is then adapted end-to-end using LoRA, guided by a cross-attention-based mapping that fuses image embeddings with learned prototypes. Each image is represented as a linear combination of prototypes, dynamically weighted according to feature similarity, thereby bounding intra-class variance and mitigating data-level heterogeneity.

A schematic overview of the two-stage training procedure is provided below.

Figure 3: Schematic outline of the GAPL training flow—Stage 1: prototype extraction, Stage 2: LoRA-based adaptation and prototype mapping for detection.

Empirical Findings and Performance

Scaling Paradox Quantification

Through careful experiments on benchmark datasets sampled to independently vary the number of generators (but not data volume), the following is established:

• Trace analysis of the scatter matrix (proxy for covariance) shows the variance of generated images grows monotonically with generator count, whereas real image variance remains stable.
• The Fisher ratio, measuring real vs. fake class separability, as well as validation accuracy, both sharply decline with increased generator heterogeneity when using a frozen encoder. End-to-end models retain superior separability but are prone to overfitting.

  Figure 2: (a) Trace (total variance) of scatter matrices highlights increased heterogeneity as generators are aggregated. (b) Detection accuracy and Fisher ratio degrade with generator count, especially in fixed-encoder settings.

SOTA Results

GAPL sets a new benchmark across 6 major testbeds, covering 55 distinct test subsets sampling GANs, diffusion models, and advanced commercial generative models:

• GAPL achieves an average accuracy of 90.4% and mean AP of 94.9%, representing a 3.5% absolute accuracy improvement over the previous best scaling-up detector.
• GAPL maintains robust accuracy even under aggressive post-processing (e.g., JPEG compression, Gaussian blur) with much lower performance degradation than prior methods relying on high-frequency or format shortcuts.

  Figure 5: Robustness comparison under post-processing. GAPL's minimal performance degradation validates its invariance to photometric transformations.

Ablation and Interpretation

Comprehensive ablations demonstrate the necessity of each module (PCA-based prototype extraction, cross-attention mapping, LoRA-based adaptation). Increasing the number of prototypes provides diminishing returns; three or four generator classes are sufficient to capture major forgery concepts.

Visualization of average attention scores between images and prototypes shows meaningful clustering: images with high attention to the same prototype share interpretable visual attributes (e.g., distorted objects, oversmooth regions in fakes; complex illumination or natural scenes in reals).

Figure 4: Distribution of attention weights over prototypes reveals that the learned prototype space encodes semantically consistent artifacts and visual themes.

Figure 6: Self-attention map comparison between frozen and GAPL-finetuned CLIP backbone. GAPL encourages broader spatial focus, especially in deeper transformer layers.

Theoretical and Practical Implications

GAPL's core insight—that constraining generated feature variance via a generator-aware prototype space and decoupling from static pretrained representation spaces—establishes a systematic direction for universal AIGI detection under unconstrained generator scaling. Theoretically, as shown in the appendix, mapping to a finite prototype basis explicitly bounds class variance, fundamentally limiting the deleterious effects of uncontrolled generator diversity (see detailed derivation in paper).

Practically, this approach is robust to post-processing and agnostic to the specific generative architecture, making it highly applicable in both forensic and open-environment trustworthiness settings. The two-stage adaptation strategy also preserves the generalization capacity of large pretrained models while effectively injecting and organizing generator-specific cues.

Future Directions

While GAPL substantially narrows the generalization gap under current generator diversity, ultimate detector universality faces open challenges. Notably:

• If generative models further improve such that artifacts are eliminated or manifest only at high cognitive or logical levels, low-level prototype approaches may lose effectiveness.
• Integration with visual reasoning, physical-world consistency checks, and embodied perception will become crucial, demanding compositional and neuro-symbolic frameworks beyond artifact-based detection.

Conclusion

This work provides a rigorous diagnosis of the limitations inherent in naive scaling of AIGI detectors and proposes GAPL—a prototype-driven, LoRA-adapted framework—that robustly overcomes both data-level and model-level barriers. Quantitative results and interpretability analyses confirm the validity and practical applicability of the approach, establishing a new state-of-the-art for AI-generated image detection across a broad range of challenging generators and robustness conditions.