MGFFD-VLM: Face Forgery Detection & Explanation

• The paper introduces MGFFD-VLM, a framework that unifies forgery detection with natural language explanations to answer why a face is forged.
• It leverages an expanded VQA dataset (DD-VQA+) and an attribute-driven hybrid LoRA strategy to adapt using detailed face-quality cues.
• Multi-granularity prompt learning and a forgery-aware training strategy enable precise localization, classification, and interpretation of deepfake artifacts.

MGFFD-VLM is a vision-language framework for face forgery detection that treats deepfake analysis as both a recognition problem and an explanation problem. Rather than limiting inference to a binary real/fake decision, it is designed to answer why a face is forged in human-readable language, using richer supervision over forgery regions, forgery types, and face-quality attributes. The framework combines an expanded VQA-style dataset, DD-VQA+, with an Attribute-Driven Hybrid LoRA Strategy, Multi-Granularity Prompt Learning, and a Forgery-Aware Training Strategy, and is built on MobileVLM with a ViT-L/14 vision encoder, a lightweight projector, and a MobileLLaMA LLM (Chen et al., 16 Jul 2025).

1. Research setting and problem formulation

MGFFD-VLM arises from a specific limitation in prior deepfake detection research: conventional detectors can be effective at binary classification, but they do not provide interpretable rationales, while earlier VLM-based methods improve interpretability yet still underuse face-quality attributes and lack sufficiently effective forgery-aware training strategies (Chen et al., 16 Jul 2025). The paper explicitly situates the task in a VQA-style regime in which the system should answer both “Is this face a forgery?” and “Why is the face a forgery?” using natural-language outputs.

The framework is motivated by four difficulties. First, prior VLM-based methods do not fully leverage face quality-related attributes such as illumination, clarity, visibility, and facial integrity, even though forged faces often exhibit abnormalities in these dimensions. Second, they do not provide sufficiently effective forgery-aware training strategies, which weakens attention to subtle artifacts. Third, available datasets are often small and manually annotated, limiting diversity. Fourth, forgery artifacts are frequently imperceptible, so prompting a large VLM to attend to the relevant evidence is intrinsically difficult (Chen et al., 16 Jul 2025).

Within this formulation, MGFFD-VLM is not merely a language wrapper around a visual classifier. Its design couples visual forgery signals, prompt construction, and language generation so that classification, localization, and explanation become mutually constraining components. This suggests a shift from post hoc explanation toward explanation-conditioned detection.

2. DD-VQA+ and the supervision space

A central component of the framework is DD-VQA+, which extends the earlier DD-VQA dataset with a richer attribute set and a more diverse sample distribution (Chen et al., 16 Jul 2025). DD-VQA already contains forgery-related VQA pairs concerning authenticity, forged facial region, and common explanations, but the extension is introduced because the original dataset is manually annotated, less diverse, and insufficiently expressive about face-quality cues.

DD-VQA+ organizes supervision around four per-sample question-answer groups:

$$Q_i = \{Q_{local}^i, Q_{common}^i, Q_{classify}^i, Q_{quality}^i\}, \qquad A_i = \{A_{local}^i, A_{common}^i, A_{classify}^i, A_{quality}^i\}.$$

These are randomly sampled during training to diversify the linguistic surface form.

For local forgery supervision, the dataset augments samples by blending real and fake faces using binary masks over regions such as the mouth, nose, eyes, or the entire face:

$$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$

The corresponding response is composed as

$A_{local} = T_{label} + T_{region} + T_{type}.$

Here, the answer specifies authenticity, forged region, and forgery type. The paper states that five forgery-type descriptions are used, including blur, structural abnormality, color difference, and blend boundary (Chen et al., 16 Jul 2025).

The most important extension beyond prior VQA-style deepfake datasets is face-quality evaluation. DD-VQA+ asks the model to evaluate overall impression, facial integrity, illumination intensity, illumination uniformity, clarity, and visibility, with responses assembled as

$$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$

Visibility is based on CPBD blur/visibility scoring, illumination intensity is estimated by a CNN trained on WIDER FACE, and GPT-4 is used to generate multiple synonymous descriptions for each score level (Chen et al., 16 Jul 2025).

This dataset design broadens supervision from authenticity and localization to a more structured notion of forensic evidence. A plausible implication is that the LLM is exposed not only to categorical forgery cues but also to degradations and inconsistencies that often co-occur with synthesis artifacts.

3. Base architecture and Attribute-Driven Hybrid LoRA

MGFFD-VLM is built on MobileVLM, which comprises a ViT-L/14 vision encoder, a lightweight projector, and a MobileLLaMA LLM (Chen et al., 16 Jul 2025). Its principal architectural modification is the Attribute-Driven Hybrid LoRA Strategy, introduced to adapt the LLM differently depending on image attributes.

The underlying premise is that forged faces are heterogeneous. Some are high quality, whereas others exhibit blur, compression artifacts, illumination problems, or other quality-dependent distortions. Instead of applying a single low-rank adaptation pathway to all cases, MGFFD-VLM uses a selection mechanism informed by both image features and face-quality indicators.

The selection process is described as follows. A pretrained image-quality model $M_q$ extracts quality indicators $Q_I$. The image feature $f_v$ is mapped by a linear layer $l_v$ to a feature-based selection vector $v_s$, while $Q_I$ is mapped by another layer $$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$0 to $$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$1. These vectors are concatenated and passed through a selection layer $$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$2, producing a 4-dimensional vector. After softmax, the model obtains selection probabilities over four specialized LoRA branches. The highest-probability expert is selected, but its output is mixed with a global expert through

$$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$3

where $$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$4 is the highest expert-selection probability, $$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$5 are the LoRA matrices of the selected specialized expert, $$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$6 are the global expert matrices, and $$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$7 is the original linear transformation with residual connection (Chen et al., 16 Jul 2025).

This mechanism is intended to let the LLM adapt to different image conditions, including high-quality forged faces, blurry or manipulated faces, and heavily compressed faces. In effect, the paper substitutes attribute-conditioned specialization for uniform finetuning. This distinguishes MGFFD-VLM from a naive LoRA configuration in which all samples share the same adaptation subspace.

4. Multi-Granularity Prompt Learning

The framework’s defining interface between visual evidence and language reasoning is Multi-Granularity Prompt Learning (Chen et al., 16 Jul 2025). The paper’s main claim here is that classification results and segmentation results are transformed into prompts and concatenated with the normal question prompt and visual embeddings:

$$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$8

Two prompt types are central. The first is the Probability Prompt, which injects global authenticity evidence. The model extracts image features, passes them through a segmentation decoder based on DeepLabv3, and uses the decoder’s last-layer features to produce a binary classification prediction $$x_i^{blend} = M_i \times x_i^{real} + (1 - M_i) \times x_i.$$9. Learnable text templates are defined for fake and real cases:

$A_{local} = T_{label} + T_{region} + T_{type}.$0

The prompt is then composed as

$A_{local} = T_{label} + T_{region} + T_{type}.$1

When $A_{local} = T_{label} + T_{region} + T_{type}.$2 is high, the prompt leans toward the fake template; when it is low, it leans toward the real template (Chen et al., 16 Jul 2025).

The second component is the Location Prompt, which injects local spatial evidence. The segmentation feature is projected into text space as $A_{local} = T_{label} + T_{region} + T_{type}.$3 with shape $A_{local} = T_{label} + T_{region} + T_{type}.$4, and combined with learnable vectors:

$A_{local} = T_{label} + T_{region} + T_{type}.$5

This prompt provides location-aware forgery clues to the LLM (Chen et al., 16 Jul 2025).

The multi-granularity designation refers to the coexistence of a global authenticity cue and a local spatial cue. A common misconception would be to treat this as generic prompt engineering. The architecture indicates something more specific: the prompts are generated from the model’s own intermediate forensic predictions, so prompt tokens encode explicit decision confidence and segmentation-derived evidence rather than generic task descriptions.

5. Forgery-Aware Training Strategy

MGFFD-VLM uses a three-stage progressive training pipeline designed to make visual forgery discrimination and language generation cohere (Chen et al., 16 Jul 2025).

In Stage 1, termed Multi-Granular Visual Learning, the LLM is frozen while the segmentation decoder and binary classification head are trained. The losses are binary cross-entropy for image-level real/fake prediction, denoted $A_{local} = T_{label} + T_{region} + T_{type}.$6, and pixel-wise cross-entropy for segmentation, denoted $A_{local} = T_{label} + T_{region} + T_{type}.$7. This stage establishes coarse and fine forgery discrimination before language adaptation.

In Stage 2, termed Forgery Adaptation with Prompts, the image projector, prompt embeddings, and segmentation-feature projector are unfrozen, while the LLM remains frozen. Training now uses all DD-VQA+ QA types. The key additional objective is a fine-grained contrastive loss:

$A_{local} = T_{label} + T_{region} + T_{type}.$8

where $A_{local} = T_{label} + T_{region} + T_{type}.$9 is a real image, $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$0 is a fake image, $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$1 is a blended image, and $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$2 is cosine similarity. The intent is to force the blended image $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$3 to be closer to the fake image $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$4 than to the real image $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$5, making $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$6 a hard positive for subtle forgery learning (Chen et al., 16 Jul 2025).

In Stage 3, termed Forgery Instruction Tuning, the full system is finetuned, including the LLM and LoRA modules. Two losses are emphasized here: the autoregressive text loss $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$7 and a text calibration loss $$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$8. The text calibration loss is applied at the positions where the model generates words such as “real” or “fake,” using binary cross-entropy on the logits for those tokens to sharpen authenticity judgments (Chen et al., 16 Jul 2025).

The full objective is

$$A_{quality} = T_{overall} + T_{face} + T_{intensity} + T_{uniformity} + T_{clarity} + T_{visibility}.$$9

This integrates language generation, classification, segmentation, contrastive alignment, and explicit authenticity calibration into a single training criterion (Chen et al., 16 Jul 2025). The training design indicates that the explanatory channel is supervised not only by textual targets but also by auxiliary forensic objectives that constrain the visual substrate.

6. Evaluation, empirical results, and research significance

The framework is trained primarily with FaceForensics++ for image data and DD-VQA plus newly generated DD-VQA+ attributes for text data (Chen et al., 16 Jul 2025). Frames are extracted from videos, faces are aligned, and images are resized to $M_q$0. The optimizer is Adam with betas $M_q$1. Batch size is 64 for Stages 1 and 2 and 48 for Stage 3. The learning rate is $M_q$2 for Stages 1 and 2 and $M_q$3 for Stage 3. LoRA rank is 64, LoRA alpha is 16, and LoRA dropout is 0.05.

Evaluation is conducted in two regimes. On DD-VQA-style tasks, the model is evaluated for deepfake detection and answer generation using Accuracy, Recall, Precision, F1, BLEU-4, CIDEr, ROUGE-L, METEOR, and SPICE. Baselines include BLIP, BLIP-TI, and MobileVLM (Chen et al., 16 Jul 2025). On the key comparison, BLIP on DD-VQA reaches Accuracy 0.8168, BLIP-TI on DD-VQA reaches 0.8749, MobileVLM on DD-VQA reaches 0.7699, MobileVLM on DD-VQA+ reaches 0.8802, and MGFFD-VLM on DD-VQA+ reaches Accuracy 0.9072, Recall 0.9619, Precision 0.9228, and F1 0.9420. For answer generation, MGFFD-VLM reports BLEU-4 0.5349, CIDEr 3.3008, ROUGE-L 0.6963, METEOR 0.3988, and SPICE 0.6629 (Chen et al., 16 Jul 2025).

The paper also evaluates MGFFD-VLM as an enhancement module for conventional deepfake detectors using AUC on Celeb-DF, DFDC, and WildDeepfake. With Xception + MGFFD-VLM, the reported AUCs are 74.78 on Celeb-DF, 77.03 on DFDC, and 77.84 on WDF. With RECCE + MGFFD-VLM, they are 70.77, 72.67, and 74.38. With SBI + MGFFD-VLM, the reported values are 93.53 on Celeb-DF video-level, 87.80 on Celeb-DF frame-level, 81.77 on DFDC frame-level, and 81.05 on WDF frame-level (Chen et al., 16 Jul 2025). The paper interprets these results as evidence that the learned VLM features are useful beyond the immediate VQA setting.

Ablation studies support the importance of the individual components. Adding Forgery Regions and Forgery Types improves performance over the baseline; adding the Location Prompt further boosts accuracy and explanation quality; adding the Probability Prompt adds another accuracy gain; the Attribute-Driven Hybrid LoRA improves detection and most text metrics; and the auxiliary losses further improve results. In the loss ablation, removing both $M_q$4 and $M_q$5 gives Accuracy 88.32, adding $M_q$6 raises it to 89.22, and adding $M_q$7 yields 90.64 (Chen et al., 16 Jul 2025).

The principal limitation stated by the paper is generalization to unseen manipulation methods. Additional constraints are implied by the design: the framework depends on generated or self-supervised prompts, curated quality cues, and prompt templates, and its strongest results are reported on specific benchmark settings rather than a fully open-world deepfake regime (Chen et al., 16 Jul 2025). Accordingly, MGFFD-VLM should not be understood as solving universal face forgery detection. Its contribution is more precise: it demonstrates that structured VLM adaptation, prompt construction from forensic intermediate outputs, and multi-attribute supervision can improve both detection quality and explanation quality within a benchmarked face-forensics setting.

A plausible broader implication is that MGFFD-VLM exemplifies a wider VLM design pattern in which lightweight adaptation, task-specific prompting, and structured intermediate supervision are preferred to undifferentiated end-to-end scaling. Related VLM systems in autonomous driving, federated fairness, and embodied navigation also rely on prompt-based conditioning, modular adaptation, or structured reasoning rather than generic multimodal generation alone (Gopalkrishnan et al., 2024, Chen et al., 3 May 2025, James et al., 18 Mar 2026). Within face forensics, MGFFD-VLM specializes that pattern to the dual objective of evidential detection and natural-language explanation.