Multimodal Models Meet Presentation Attack Detection on ID Documents

Abstract: The integration of multimodal models into Presentation Attack Detection (PAD) for ID Documents represents a significant advancement in biometric security. Traditional PAD systems rely solely on visual features, which often fail to detect sophisticated spoofing attacks. This study explores the combination of visual and textual modalities by utilizing pre-trained multimodal models, such as Paligemma, Llava, and Qwen, to enhance the detection of presentation attacks on ID Documents. This approach merges deep visual embeddings with contextual metadata (e.g., document type, issuer, and date). However, experimental results indicate that these models struggle to accurately detect PAD on ID Documents.

• The paper demonstrates that state-of-the-art multimodal models exhibit physical blindness, misclassifying spoofed ID documents under various attack modalities.
• It employs diverse prompt engineering strategies, including multi-turn and background-enriched prompts, to assess both semantic accuracy and visual discrimination capabilities.
• Evaluation metrics like EER, APCER, and BPCER reveal critical performance limitations, stressing the need for architectural improvements in forensic applications.

Multimodal Model Evaluation for Presentation Attack Detection in ID Documents

Introduction

The paper "Multimodal Models Meet Presentation Attack Detection on ID Documents" (2603.29422) systematically investigates the applicability of state-of-the-art multimodal visual LLMs (VLLMs) for Presentation Attack Detection (PAD) in identity documents, a security-critical domain within biometric authentication. Leveraging PaliGemma2-3b-mix-224, LLaVA1.6-7b-mistral, and Qwen2.5-3b-instruct, the authors benchmark these models in PAD tasks where attack modalities include print, screen, PVC, and tampered document variants. The assessment targets both their semantic grasp of domain-specific terminology and their visual discrimination of bona fide versus attack presentations.

Multimodal Models and Prompt Engineering Strategies

The study first reviews the architectures and strengths of each model. PaLIGemma2-3b-mix-224 is designed for lightweight multimodal inference, optimized for real-time visual-text understanding at low resolution. LLaVA1.6-7b-mistral integrates enhanced instruction-following and visual reasoning capabilities, whereas Qwen2.5-3b-instruct prioritizes contextual fluency and efficiency. The prompt engineering approach systematically explores seven families of prompt types—single-turn, multi-turn, example-based, background-rich, task-oriented, and recipe-style—to probe the extent to which these VLLMs generalize across PAD scenarios.

The multi-turn prompt strategy allows stepwise semantic scaffolding, attempting to decompose the PAD task into sequential definitions and visual inspection, but domain knowledge is often shallow. For example:

Figure 1: Multi-turn prompt strategy sequence, breaking PAD task into definition and classification stages.

Scoring Mechanism and Evaluation Metrics

To enable objective model evaluation, the authors standardize output selection via logits-based scoring. Prompts are designed to constrain the initial token (e.g., "Yes"/"No" or "A"/"B"), and output logits are softmax-normalized across token variants. The PAD performance metrics adhere to ISO/IEC 30107-3, reporting APCER, BPCER, and EER at conventional thresholds. Lexical token variants are aggregated to represent semantic equivalence, mitigating post-processing inconsistencies.

Figure 2: Logit-based scoring process for classifying model response tokens in PAD settings.

Dataset Construction and Attack Modalities

The evaluation dataset comprises 100 bona fide and 100 attack samples, spanning ID cards and passports from seven countries. Attack modalities include paper prints, digital screen displays, PVC reproductions, and physically tampered specimens, reflecting realistic spoof scenarios encountered in remote onboarding pipelines.

Figure 3: Example of a paper attack, illustrating re-capturing strategies used in presentation attacks.

Figure 4: Visual keypoints of a spoof PVC ID card, highlighting texture-based artifacts.

Semantic and Visual Competence Analysis

Experiment 1: Definition Accuracy

Semantic keyword analysis reveals pronounced differences between models. PaliGemma lacks technical vocabulary and produces largely generic or blank responses. LLaVA and Qwen demonstrate richer domain lexicon, but are prone to excessive verbosity (Qwen) or hallucination (LLaVA), especially when prompted for detailed fraud types.

Figure 5: Distribution of domain-specific terminology usage by each multimodal model.

Experiment 2: Visual Description

A scoring scheme (blindness, mention, guided detection, spontaneous detection, hallucination) is applied to model responses on PVC attacks. Models generally fail to detect physical defects unless specifically prompted. LLaVA provides the most stable performance, yet relies on guided attention rather than autonomous forensic insight.

Experiment 3: Prompt Design and PAD Performance

Across binary classification setups, models frequently exhibit systematic decision bias. PaliGemma consistently classifies nearly all samples as bona fide unless exposed to background-enriched prompts, which invert its decision logic without substantive visual analysis. LLaVA demonstrates marginally improved trade-off, achieving its best EER at 25% with Simple_8 prompts, while Qwen maintains moderate consistency across prompt types but remains close to random performance (EER 45–55%).

DET curves for best-performing prompts underscore these trends:

Figure 6: DET curves for PaLIGemma2, LLaVA, and Qwen illustrating PAD performance with optimal prompt configurations.

Implications and Future Research Directions

The numerical results indicate that even with sophisticated prompt engineering and zero/few-shot approaches, current multimodal VLLMs exhibit "physical blindness"—a deficit in detecting texture, reflection, and other visual artifacts critical for identifying spoofed documents. This semantic over-reliance results in systematic misclassification and undermines practical deployment in high-security, real-world onboarding systems.

The study raises salient theoretical questions regarding tokenization and output binarization. The model's handling of tokens (e.g., capitalization, output variants) may bias decision boundaries, suggesting that internal token mapping and prompt syntax directly impact PAD reliability. Further research into VLLMs' token-level behavior and visual-text integration is essential for improving robustness and operational fairness.

Conclusion

This work delivers an authoritative benchmark of multimodal VLLMs for PAD in ID document verification pipelines. Results emphasize the inadequacy of current models for operational deployment, rooted in their lack of visual artifact sensitivity and susceptibility to semantic prompt bias. Advancing PAD with multimodal AI will require both targeted architecture modifications and rigorous token-level analysis to bridge the gap between semantic comprehension and forensic visual discrimination.