Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs

Abstract: Multimodal LLMs (MLLMs) have raised new privacy challenges. On the data side, user-provided inputs often include unpredictable sensitive information; while on the downstream task side, model reasoning depends on rich visual context that may itself be privacy-sensitive. Existing privacy protection methods, however, rely on predefined sensitive categories and fixed obfuscation strategies, struggling to tackle such challenges in MLLMs. To address this dilemma, we propose Anchored Privacy Drifting (APD), a training-free method that drifts privacy-sensitive elements toward semantically equivalent alternatives while anchoring contextual cues to the source image. To systematically evaluate this dual objective of privacy protection and contextual preservation, we introduce AdaptShield, a comprehensive benchmark covering 22 privacy categories, which combines conventional privacy metrics with MLLM-based assessments of contextual utility. Extensive experiments show that our method achieves balanced improvements in both privacy sanitization and content retention, with average gains of 10.4% on textual categories and 8.5% under MLLM-based evaluation across four MLLM series, i.e., Qwen2.5, Qwen3, InternVL3, and InternVL3.5.

• The paper introduces APD, a training-free method that dynamically controls latent drift in sensitive regions to balance privacy preservation with content utility.
• The approach achieves significant gains, including a 10.4% improvement in F1-privacy for textual obfuscation and state-of-the-art performance in facial privacy protection.
• It also presents the AdaptShield benchmark with 32,491 images across 22 privacy categories to holistically evaluate privacy concealment and content fidelity.

Adaptive Privacy Control for MLLMs via Anchored Privacy Drifting

Motivation and Problem Formulation

The widespread adoption of Multimodal LLMs (MLLMs) introduces nuanced privacy risks, as user-provided inputs often carry heterogeneous, unpredictable, and personalized sensitive information embedded in both visual and textual elements. Classical privacy protection mechanisms, largely based on static obfuscation or category-specific attributes (e.g., face blurring), are fundamentally mismatched to the open-set, context-hungry nature of MLLMs. They either destroy the contextual cues required for downstream reasoning, or their rigidity fails to scale to the diversity and subjectivity of real user privacy requirements.

Figure 1: Anchored Privacy Drifting (APD) defines a new privacy protection paradigm for MLLMs. It preserves content coherence and high perceptual quality while explicitly maintaining information beneficial for downstream tasks and personalized user requirements across diverse inputs.

Qualitative examples in MLLM query/response setups show that simple obfuscation of visual regions can induce both catastrophic MLLM failures and severe loss of perceptual information. In contrast, the proposed approach aims to safeguard privacy while maintaining functional utility for downstream AI applications.

Figure 2: High-fidelity visual privacy protection is required in MLLM scenarios; naive obfuscation impairs utility, whereas the proposed method maintains contextual cues and plausible model responses while masking sensitive regions.

Anchored Privacy Drifting: Framework and Control Mechanism

The paper proposes Anchored Privacy Drifting (APD), a training-free approach that frames privacy-preserving image transformation as a controllable trajectory in a multimodal latent space. APD is specifically developed for generation-level privacy protection aligned with the requirements of open-world, personalized MLLM usage.

The method introduces a two-field latent transport protocol:

• Semantic Drift Field: This component induces stochastic divergence in privacy-designated regions, guided either by explicit textual editing instructions or implicit semantic sampling, driving the sensitive content away from its original instantiation.
• Source Anchoring Field: To counteract destructive drift and preserve structural as well as high-level contextual fidelity, APD implements a dynamic anchoring field that penalizes deviation from the source image manifold. Anchoring strength is adaptively adjusted across spatial regions (sensitive/non-sensitive) and time steps.

The generation process thus becomes a spatially and temporally controlled weighted sum of these two vector fields. For non-sensitive areas, source anchoring dominates, enforcing maximal content preservation. In privacy-masked regions, semantic drift is enabled after an early binding phase, allowing context-coherent yet privacy-divergent sampling.

Figure 3: Left: Overview of APD; the source image is inverted into latent noise and regenerated under dual guidance. Right: The adaptive controller (middle) yields privacy-preserving and high-fidelity outputs compared to destructive obfuscation (top) or inflexible preservation (bottom).

This mechanism supports both user-defined attribute substitution (e.g., changing displayed names, obfuscating faces) and generic, stochastic semantic redirection.

AdaptShield Benchmark: Dataset and Metrics

To systematically evaluate privacy mechanisms in MLLMs, the paper introduces AdaptShield, a large-scale benchmark comprising 32,491 images across 22 privacy categories, including facial, textual, and composite types (e.g., signatures, credit cards, tickets, educational credentials). Categories are annotated with spatial privacy masks and semantically grounded labels for downstream evaluation.

Unlike prior benchmarks, AdaptShield provides:

• Holistic joint assessment of privacy concealment and content fidelity.
• Multifactorial evaluation: Identity similarity (for faces), OCR-correctness (for text), MLLM probing (for composites), perceptual metrics (SSIM, PSNR), and model-based semantic scoring.
• Unified metric: F1-Privacy ($\text{F1-privacy}$), defined as the harmonic mean of normalized protection ($P$) and fidelity ($F$) scores.

Experimental Results and Analysis

Textual Privacy Obfuscation: APD achieves a 10.4% average gain in F1-privacy over the best prior general editing baseline (FluxEdit), both preserving the visual layout and strongly reducing sensitive textual content (e.g., credit card digits, addresses, names) even in the presence of complex layouts.

MLLM-based Evaluation: APD delivers an 8.5% improvement across four MLLM evaluators (Qwen2.5/3 and InternVL3/3.5), outperforming InstructPix2Pix, FluxEdit, and region-based obfuscators on both protection and semantic plausibility. Notably, APD maintains plausible model-generated answers that are non-identifiable, avoiding implausible or blank responses typical with rigid obfuscation.

Facial Privacy Protection: On face anonymization, APD achieves state-of-the-art average F1-privacy scores across VGGFace, FaceNet, and ArcFace evaluation axes, outperforming training-based models and other training-free approaches such as NullFace ($0.857$ vs. $0.639$ on average). Importantly, APD supports attribute-level control (age, gender, ethnicity), enabling more adaptive and user-tailored privacy without sacrificing model utility.

Figure 4: APD yields high-quality, privacy-preserving transformations across textual, composite, and facial categories, maintaining structure and style consistency while masking sensitive semantic content.

Ablation studies confirm the necessity of dynamic control: optimal settings of anchoring magnitude ($\eta$) and release timestep ($\tau$) trade off between structural preservation in sensitive regions and anonymization strength, and can be tuned per privacy type for maximal joint efficacy.

Implications and Future Directions

This work formalizes the privacy-utility tradeoff for MLLMs as a spatially and temporally tunable latent transport problem, demonstrating that training-free, controllable generation offers both practical scalability and compliance with real-world privacy demands. APD's generality, combined with the adoption of comprehensive, model-agnostic benchmarks such as AdaptShield, provides a foundation for research into context-adaptive privacy mechanisms across vision-language stacks.

The paper further exposes that static, universal obfuscation approaches fundamentally fail in the presence of context-hungry downstream reasoning, motivating future research into personalized interpretable privacy interfaces, semantic editing protocols, and adaptive constraints in open-world settings. The latent transport principle underlying APD is applicable to broader multimodal generation pipelines, suggesting avenues for plug-in privacy modules in deployment frameworks.

Conclusion

"Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs" (2606.07175) presents a unified, adaptive, and training-free approach to privacy protection in MLLMs, addressing the limitations of both concealment- and fidelity-centric methods. The APD framework demonstrates substantial quantitative and qualitative gains across diverse privacy categories and MLLM backends, with the AdaptShield benchmark setting a new standard for systematic evaluation in privacy-aware multimodal intelligence. This research is a critical step toward robust, user-aligned privacy in model-driven visual reasoning systems.