UniShield: An Adaptive Multi-Agent Framework for Unified Forgery Image Detection and Localization (2510.03161v1)

Abstract: With the rapid advancements in image generation, synthetic images have become increasingly realistic, posing significant societal risks, such as misinformation and fraud. Forgery Image Detection and Localization (FIDL) thus emerges as essential for maintaining information integrity and societal security. Despite impressive performances by existing domain-specific detection methods, their practical applicability remains limited, primarily due to their narrow specialization, poor cross-domain generalization, and the absence of an integrated adaptive framework. To address these issues, we propose UniShield, the novel multi-agent-based unified system capable of detecting and localizing image forgeries across diverse domains, including image manipulation, document manipulation, DeepFake, and AI-generated images. UniShield innovatively integrates a perception agent with a detection agent. The perception agent intelligently analyzes image features to dynamically select suitable detection models, while the detection agent consolidates various expert detectors into a unified framework and generates interpretable reports. Extensive experiments show that UniShield achieves state-of-the-art results, surpassing both existing unified approaches and domain-specific detectors, highlighting its superior practicality, adaptiveness, and scalability.

• The paper introduces a unified, adaptive framework that integrates multi-agent reasoning, outperforming traditional domain-specific methods.
• It combines LLM-based and non-LLM-based detectors with dynamic scheduling to deliver precise detection and pixel-level localization.
• Experimental evaluations show significant improvements in F1 scores across forgery domains, demonstrating superior cross-domain generalization.

UniShield: An Adaptive Multi-Agent Framework for Unified Forgery Image Detection and Localization

Motivation and Problem Statement

The proliferation of highly realistic synthetic images generated by advanced AIGC, DeepFake, and manipulation techniques has escalated the risk of misinformation, fraud, and privacy violations. Existing Forgery Image Detection and Localization (FIDL) methods are typically domain-specific, exhibiting strong in-domain performance but poor cross-domain generalization. Moreover, the lack of an integrated, adaptive framework forces users to manually select detection tools, which is impractical in real-world scenarios. UniShield addresses these limitations by introducing a unified, multi-agent system capable of automatic, interpretable, and scalable forgery detection and localization across diverse domains.

System Architecture

UniShield is composed of two principal components: the Perception Agent and the Detection Agent. The Perception Agent is responsible for analyzing the input image, determining the forgery domain, and selecting the most appropriate detection tool. The Detection Agent aggregates expert models from all forgery domains and generates a structured, interpretable report.

Figure 1: The UniShield pipeline, illustrating the interaction between the Perception Agent and Detection Agent for adaptive forgery detection and report generation.

Perception Agent

The Perception Agent consists of a Task Router and a Tool Scheduler. The Task Router leverages a fine-tuned multimodal LLM (MLLM), specifically Qwen2.5-VL, optimized via Group Relative Policy Optimization (GRPO), to classify images into one of four forgery domains: Image Manipulation Detection and Localization (IMDL), Document Manipulation Detection and Localization (DMDL), DeepFake Detection (DFD), and AI-Generated Content Detection (AIGCD). The router is trained to perform source tracing and forgery type inference, not just standard image classification.

The Tool Scheduler further analyzes the image to decide between LLM-based and non-LLM-based detection methods. LLM-based detectors are preferred for semantic/logical inconsistencies, while non-LLM-based models are selected for low-level artifact detection. The scheduler uses carefully designed prompts for Qwen2.5-VL to make this binary decision without additional training.

Figure 2: Comparative performance of LLM-based (FakeShield) and non-LLM-based (IML-ViT) detectors on semantic and artifact-based forgeries.

Detection Agent

The Detection Agent maintains a toolbox of expert detectors for each domain, including both LLM-based and non-LLM-based models. For IMDL and DMDL, detectors output both confidence scores and tampering region masks; LLM-based models also provide natural language explanations. For DFD and AIGCD, detectors output detection results and explanations. In domains lacking open-source LLM-based detectors (DFD, DMDL), Qwen2.5-VL is fine-tuned using GRPO to provide interpretable outputs. For localization tasks, textual outputs are fed into GLaMM for mask prediction.

After detection, the Summarizer module (GPT-4o) generates a structured report containing a description of the image, detection conclusion, localization (if applicable), and the reasoning behind the judgment.

Figure 3: Example of a structured forgery report generated by UniShield, detailing content, detection, localization, and judgment basis.

Experimental Evaluation

UniShield is evaluated on authoritative benchmarks across all four domains: CASIA1+ and IMD2020 (IMDL), RTM (DMDL), AIGCDetectionBenchmark (AIGCD), and DF40 (DFD). The framework integrates eight expert detectors and is trained using 4 NVIDIA A800 80GB GPUs, with GRPO-based optimization for agent modules.

Cross-Domain Performance

UniShield demonstrates superior cross-domain generalization, outperforming both unified and domain-specific baselines. For example, in DFD, UniShield achieves an F1 score of 0.911, exceeding FakeShield by 0.201. In DMDL, it achieves an F1 score of 0.736, surpassing ResNet by 0.086. These results highlight the limitations of existing methods, which suffer significant performance degradation when transferred to new domains.

Sub-Domain Expert Comparison

On IMDL, UniShield achieves an image-level F1 of 0.96 and pixel-level F1 of 0.77, outperforming FakeShield and IML-ViT. In DeepFake detection, it attains AUC scores of 0.997 (FS) and 0.913 (FR), exceeding all baselines. For DMDL, it achieves the best image-level F1 (0.737) and pixel-level F1 (0.341). In AIGCD, UniShield achieves the highest mean accuracy (0.942) across 16 generative models. The cooperative reasoning mechanism yields performance gains beyond any single expert, demonstrating effective model integration and synergy.

Ablation Study

Ablation experiments confirm the necessity of dynamic tool scheduling and learning-based integration. Variants that always select LLM-based or non-LLM-based tools, or use simple rule-based aggregation, perform significantly worse than the full UniShield system.

Implementation Considerations

• Model Integration: UniShield’s modular design allows seamless integration of new expert detectors, supporting future scalability as forgery techniques evolve.
• Resource Requirements: The framework requires substantial GPU resources for training agent modules and running large MLLMs, but inference can be optimized via model distillation or quantization.
• Deployment: UniShield’s interpretable reporting and adaptive scheduling make it suitable for forensic, security, and content verification applications. The system can be deployed as a cloud service or on-premises, depending on privacy and latency requirements.
• Limitations: The reliance on large MLLMs for perception and summarization introduces computational overhead. The system’s performance depends on the quality and diversity of expert detectors in the toolbox.

Implications and Future Directions

UniShield establishes a new paradigm for unified, adaptive forgery detection, bridging the gap between domain-specific expertise and practical deployment. Its multi-agent architecture enables robust cross-domain generalization and interpretability, critical for real-world security and forensic applications. The framework’s scalability positions it as a foundation for future research in trustworthy AI, especially as generative and manipulation techniques become more sophisticated.

Potential future developments include:

• Expanding the toolbox with video forgery detectors and multimodal analysis.
• Enhancing the perception agent with continual learning for emerging forgery types.
• Integrating privacy-preserving and federated learning strategies for sensitive applications.
• Extending the reporting module to support regulatory compliance and legal evidence standards.

Conclusion

UniShield presents a comprehensive, adaptive solution for unified forgery image detection and localization, integrating multi-agent reasoning, dynamic tool selection, and interpretable reporting. The system achieves state-of-the-art performance across all major forgery domains, demonstrating robust generalization, scalability, and practical applicability. UniShield is poised to become a critical component in the AI security ecosystem, supporting information integrity and public trust in the face of increasingly sophisticated synthetic media.