- The paper introduces DeFakerOne, an InternVL2-2B and SAM2 foundation model that jointly detects image tampering and generates pixel-level forgery masks across DeepFake, AIGC, document, and natural-image domains.
- The model achieves 95.8 AUC on DeepFake detection, 87.5% AIGC accuracy, 78.7% document localization F1, and 95.77% accuracy on the 71-sample GPT-Image-2-Bench.
- The paper finds that operation-level artifact similarity, balanced multi-domain data, fine-grained segmentation supervision, and resolution-preserving visual features matter more than simply increasing dataset size or using newer VLM backbones.
Motivation: from fragmented forensics to unified forgery generation
The paper addresses a structural mismatch in Fake Image Detection and Localization (FIDL). Research has historically been organized into four largely isolated subfields—document forgery detection, natural image manipulation detection and localization, DeepFake detection, and full-image AIGC detection—each with its own artifact assumptions, supervision granularity, architectures, and evaluation protocols. The authors argue that foundation generative models for text-to-image generation and image-to-image editing are eroding these boundaries: a single modern generator can produce forged documents, manipulated faces, edited natural scenes, and fully synthetic content through shared generation and editing operations. Detectors that rely on domain-specific artifacts consequently degrade sharply on frontier synthesizers such as GPT-Image-2, which produce high texture fidelity and few low-level synthesis traces.
Two challenges frame the work: (1) the lack of systematic modeling of cross-domain artifact transfer and interference—whether supervision from one forensic domain helps or harms another remains underexplored; and (2) the limited capacity of existing small vision models to jointly support image-level detection, pixel-level localization, and cross-domain generalization within one framework.
Architecture
DeFakerOne couples two cascaded components. An MLLM-based perception-and-detection module built on InternVL2-2B performs coarse-grained authenticity judgment via a dynamic VQA formulation: rather than coupling output to a single binary label, the model is trained on diverse question–answer template pairs whose first tokens encode polarity. At inference, detection uses a constrained-vocabulary scoring scheme over eight response words ({Yes, Yeah, True, Sure, No, Not, Never, None}); softmax-normalized first-token logits are aggregated into tampering and authenticity scores summing to one, yielding a fixed 0.5 decision boundary without per-task threshold tuning.
A SAM2-based segmentation module, following an SA2VA-style design, consumes segmentation tokens emitted by the LLM as spatial prompts. These tokens interact with multi-scale SAM2 encoder features via cross-attention in the decoder, producing pixel-level forgery masks optimized with BCE plus Dice loss to handle foreground–background imbalance.
Training proceeds in three stages: full-parameter SFT on 2M samples for paradigm validation; full-parameter SFT on 12.5M samples with balanced domain sampling; and decoupled refinement, where LoRA (rank 128) adapts the LLM while the SAM2 backbone is fully fine-tuned on 340K document and natural-image mask pairs. The SFT objective combines autoregressive cross-entropy for text with the segmentation loss.
Data engine
The training corpus comprises 12.5M samples across four domains: DeepFake (3.1M), AIGC (3.6M), Document (2.5M), and Nature (3.3M), mixing public benchmarks (FaceForensics++, CelebDF-v2, GenImage, DocTamper, MIML, etc.) with substantial private data—notably 2.22M private DeepFake samples and 2.338M private document samples drawn from a pool of over 4,000 real-world document classes. A closed-loop pipeline drives continuous adaptation: failure cases from the trained model are analyzed by an agent that reverse-engineers missing manipulation patterns and invokes appropriate generation or editing models to synthesize targeted augmentation data, which re-enters the training pool.
The paper also introduces GPT-Image-2-Bench, a 71-sample benchmark spanning documents, faces, natural scenes, general AIGC content, posters, and social-media-style images, all generated by GPT-Image-2 using a VLM-driven description-then-regenerate pipeline. The authors are candid that this benchmark is small; its value lies in targeting a distribution shift that existing detectors handle poorly.
Empirical results
DeFakerOne reports state-of-the-art results on 39 detection benchmarks and 9 localization benchmarks across four domains, with domain averages of 95.8 AUC on DeepFake, 87.5 ACC on AIGC, 87.4 ACC on Document, and 86.7 AUC on Nature. On OpenMMsec it reaches 91.8 average accuracy, versus 80.4 for the strongest prior baseline (SegFormer), with particularly large margins in Document (90.1 vs. 73.4). Notably, several MLLM-based competitors (FakeVLM, Ivy-xDetector) fall below 20% accuracy in the Document domain due to absent document training data—an outcome the authors use to underscore the primacy of data composition over architecture.
Localization results show the largest margins: DeFakerOne attains 78.7% average F1 on document benchmarks (second-best DTD: 67.4%) and 67.4% on nature benchmarks (second-best Mesorch: 52.0%). Robustness evaluations under Gaussian blur, brightness/contrast shifts, JPEG compression, noise, resizing, and saturation show consistent superiority—for example, 79.46% average accuracy under Gaussian blur versus 64.63% for the best baseline—and decoding hyperparameters (seed, temperature) affect accuracy by at most 0.1 percentage points. On GPT-Image-2-Bench, DeFakerOne achieves 95.77% accuracy, while DRCT, DDA, Mirror, and FakeShield remain below 51%, supporting the claim that detecting frontier-generator outputs requires semantic and layout-level reasoning beyond low-level artifact recognition.
Analysis: four design principles
The paper's analytical contributions arguably outweigh its leaderboard results.
Single-domain scaling does not guarantee performance. Adding roughly 14M ScaleDF samples to the DeepFake training set (from 2.336M to ~16.3M) decreased open-source DeepFake benchmark AUC by 4.9%. This directly contradicts naive "more data is better" scaling expectations and motivates the paper's data-centric framing: ratio, quality, and distributional complementarity matter more than raw scale.
Operation-level artifacts govern transfer and interference. Progressive domain supplementation produces mixed effects: adding Nature data improved Nature performance by ~16.5% and helped AIGC, but hurt Doc and DeepFake. Conversely, adding AIGC+Doc data lowered overall Nature average by 1.6% while improving specific Nature subsets involving generative editing (AutoSplice +20.42%, OpenSDI +13.83%, CocoGlide +9.48%). The authors conclude that transfer is determined by manipulation-mechanism similarity (blending inconsistency, semantic completion traces, texture bias) rather than macro-domain labels—a finding that implies FIDL datasets should be organized by operation type, not domain taxonomy.
Balanced recomposition stabilizes unified training. After supplementing weakened domains, Doc recovered by ~6.0% and AIGC by ~5.9%, yielding a total average improvement of ~9.6% across the four domains through a cycle of target enhancement, interference, supplementation, and rebalancing.
Supervision granularity and resolution preservation matter. Joint classification-plus-segmentation supervision improves even image-level AUC by +3.3 points on average over classification-only training on local-manipulation benchmarks (+10.7 on NIST16), which the authors attribute to a "fine-grained attack vs. coarse-grained defense" asymmetry. Separately, swapping InternVL2-2B for newer backbones degrades unified performance—InternVL3.5-2B loses 13.2% relative on Document—because stronger visual token compression in efficiency-oriented VLMs dilutes weak forensic artifacts such as text-edge changes and compression traces. This is a notable and somewhat contrarian claim: newer general-purpose VLMs are not automatically better forensic backbones.
Limitations and open questions
Several caveats bear directly on the reported results. GPT-Image-2-Bench contains only 71 samples, so the 95.77% accuracy figure carries wide uncertainty and should be treated as indicative rather than definitive. A large share of training data is private business data, limiting reproducibility of the exact data mixture despite the released code. The scaling-law analysis is observational—based on progressive supplementation experiments rather than controlled factorial designs—and the proposed explanation in terms of operation-level artifact similarity, while plausible, is not validated with direct artifact-space measurements. The claim that InternVL2's dynamic tiling preserves forensic evidence better than newer backbones is supported only by end-task comparisons, not by ablations isolating tiling strategy from other architectural changes. Finally, localization supervision covers only Document and Nature domains (340K mask pairs); whether fine-grained supervision would similarly benefit DeepFake and AIGC remains untested, and the agentic bad-case mining loop is described qualitatively without quantifying its marginal contribution.
Conclusion
Venus-DeFakerOne demonstrates that a unified InternVL2 + SAM2 foundation model, trained on carefully recomposed multi-domain data, can dominate specialized detectors across detection and localization tasks simultaneously, including on outputs of frontier generators. Its most durable contributions are empirical laws for unified FIDL: single-domain scaling can be counterproductive, cross-domain transfer follows operation-level artifact similarity rather than domain labels, balanced recomposition recovers interfered domains, fine-grained supervision strengthens even image-level detection, and visual backbones must preserve original-resolution local evidence. The main open questions left by the paper are how to quantify artifact similarity directly, how to scale expert-in-the-loop data pipelines efficiently, and whether the observed composition principles hold at larger model and data scales.