LayLens: Explainable Deepfake Forensics

• LayLens is a multi-stage deepfake forensics tool that combines technical detection with user-friendly explanations to address media authenticity challenges.
• It employs a three-stage pipeline: high-accuracy forgery localization, natural language simplification using Qwen-VL, and guided visual reconstruction with Step1X-Edit.
• Empirical evaluations demonstrate its effectiveness in reducing cognitive load and boosting user confidence, making it valuable for both experts and laypersons.

LayLens is a multi-stage tool designed to enhance deepfake forensics by bridging high-accuracy detection with transparent, human- and lay-interpretable explanations. Developed to address issues of media authenticity and the opacity of technical detection outputs, LayLens integrates state-of-the-art forgery localization, natural language simplification, and guided image editing. The system targets users across technical backgrounds, from journalists and moderators to everyday citizens, and is evaluated for its efficacy in reducing cognitive load and increasing user confidence in deepfake identification (Narang et al., 14 Jul 2025).

1. Motivation and Foundational Pipeline

The proliferation of AI-generated or manipulated media ("deepfakes") presents significant challenges to media authenticity and public trust. While detection models consistently achieve high accuracy, their outputs—typically saliency maps, Grad-CAM visualizations, or jargon-laden explanations—are largely opaque to non-specialists. LayLens is designed to address this gap by providing “high-performance detection” combined with “human-understandable explanation.”

The LayLens system follows a three-stage pipeline:

1. Explainable Deepfake Detection: Utilizes a state-of-the-art forgery localization model to produce per-pixel probability masks and technical rationales.
2. Natural-Language Simplification: Converts technical detections into simplified, lay-accessible explanations with region-level details, emoji cues, and edit suggestions.
3. Visual Reconstruction via Guided Image Editing: Generates a plausible original (pre-manipulation) image based on identified forgery regions and correctional cues, enabling direct visual comparison.

This architecture provides both high-accuracy technical results and accessible, actionable interpretation.

2. Explainable Deepfake Detection

Stage 1 employs the FakeShield model (Xu et al. 2025) as a forgery localization backbone. FakeShield integrates a vision encoder (e.g., ResNet-50) to extract deep feature representations, fusing them with embeddings from a frozen large multimodal LLM. The output is a per-pixel mask $M \in [0,1]^{H \times W}$ indicating the probability of each pixel being manipulated.

Formal Definitions:

• Input Image: $I \in \mathbb{R}^{H \times W \times 3}$ (normalized to $[0,1]$).
• Ground-Truth Manipulation Mask (during training): $Y \in \{0,1\}^{H \times W}$.
• Predicted Mask: $M = \mathrm{FakeShield}(I) \in [0,1]^{H \times W}$.

The canonical segmentation loss function applied is:

$$L_{seg}(M, Y) = -\sum_{i=1}^{H \times W}[Y_i \cdot \log M_i + (1 - Y_i) \cdot \log (1 - M_i)]$$

Quantitative metrics, as reported in the original FakeShield publication, include an IoU of approximately 0.75 and a pixel-wise F1 score of about 0.80 on the FaceForensics++ test set. LayLens does not retrain or re-benchmark FakeShield, instead leveraging these established results (Narang et al., 14 Jul 2025).

3. Natural-Language Simplification of Explanations

Stage 2 applies Qwen-VL (Bai et al. 2025), a large vision-LLM, to translate technical detections into lay-accessible explanations. Qwen-VL is prompted with the input image, the FakeShield-generated technical rationale (e.g., descriptions of lighting, resolution, and perspective anomalies), and an instruction to produce structured JSON outputs. Each region detected as suspect is assigned:

• A simplified English explanation.
• An emoji cue for rapid identification.
• A concise edit instruction.

Example:

• Technical Output: "The facial shadow falls in the wrong direction relative to the lighting source; the teddy bear’s resolution is 30 px lower than the surrounding face region."
• Simplified Output:
  • region: “Left cheek”
  • emoji: 🤔
  • explanation: “That side of the face looks too bright compared to the rest.”
  • edit instruction: “Adjust that cheek’s brightness to match the other side.”

The prompt construction and model invocation can be formalized as:

1. Aggregate complex explanation $T_{complex}$ from FakeShield.
2. Construct prompt $P =$ “Simplify this explanation:” $+ T_{complex} +$ “Output JSON [region,name,emoji,why,itfix].”
3. Invoke $$(I, P) \rightarrow \mathrm{Qwen}\text{-}\mathrm{VL} \rightarrow J_{simplified}$$.

4. Guided Visual Reconstruction

Stage 3 uses Step1X-Edit (Liu et al. 2025), a mask-guided diffusion editing pipeline, to reconstruct a plausible original image $I \in \mathbb{R}^{H \times W \times 3}$0. The process takes as input:

• The original (potentially manipulated) image $I \in \mathbb{R}^{H \times W \times 3}$1,
• The region mask $I \in \mathbb{R}^{H \times W \times 3}$2,
• Region-level edit instructions (e.g., "correct brightness", "restore mouth contour").

The editing objective is to minimize a canonical loss:

$I \in \mathbb{R}^{H \times W \times 3}$3

where $I \in \mathbb{R}^{H \times W \times 3}$4 represents the conditional guidance (e.g., text embedding or color for inpainting), and $I \in \mathbb{R}^{H \times W \times 3}$5 is used by default. The system employs 1,000 diffusion steps, classifier-free guidance weight $I \in \mathbb{R}^{H \times W \times 3}$6, and learning rate $I \in \mathbb{R}^{H \times W \times 3}$7. The reconstructed image is presented side-by-side with the input and detection mask, enabling evaluation of the original versus plausibly restored content.

5. User Interface and Interaction Design

The interface presents a visual and interactive comparison:

• Comparison View: Displays the uploaded image $I \in \mathbb{R}^{H \times W \times 3}$8 overlaid by the mask $I \in \mathbb{R}^{H \times W \times 3}$9 (soft red, pulsing), juxtaposed with the reconstructed image $[0,1]$0. A slider allows pixel-wise comparison.
• Explanation Card: Below the images, users access a flip-card interface:
  • Front: Complex explanation (FakeShield output) with technical reasoning.
  • Back: Simplified, emoji-augmented, region-level JSON as bullet points, each with a short rationale and fix.

This dual-layered approach is engineered to facilitate both technical audit and lay comprehension.

6. Empirical Evaluation and User Study

A user study with $[0,1]$1 participants (11 familiar with deepfakes, 3 with prior tool use) was conducted, where participants analyzed 10 AI-manipulated images each using LayLens. After each trial, users completed a 5-point Likert survey on ease, clarity, cognitive load, confidence, and related metrics.

Key Quantitative Results:

Metric	Outcome
Simplified > Complex preference	65.3%
Cognitive load reduction	81.3%
Side-by-side visualization helpful	69.3%
Confidence in deepfake detection	80%
Tool adoption intent	93.3%

Wilcoxon signed-rank tests for the Complex → Simplified conditions found statistically significant improvements:

• Ease: $[0,1]$2,
• Clarity: $[0,1]$3,
• No significant change in perceived accuracy ($[0,1]$4).

Qualitative feedback emphasized the utility of emojis for region highlighting and the value of concise correctional guidance. No participant reported being misled by the image reconstruction, and most found it “convincing.”

7. Limitations and Future Directions

LayLens does not benchmark alternative simplification engines or verify robustness on video deepfakes. It currently relies on proprietary APIs for the VLM component and is restricted to still image analysis. Planned improvements include:

• Open-sourcing a lightweight VLM to enable standalone regional explanation generation.
• Extension to real-time video, audio deepfakes, and multimodal (text + image) deepfake forensics.

A plausible implication is that broadening the modality support and reducing external API reliance could further democratize access and effectiveness of explainable deepfake detection (Narang et al., 14 Jul 2025).