Laundering AI Authority with Adversarial Examples

Abstract: Vision-LLMs (VLMs) are increasingly deployed as trusted authorities -- fact-checking images on social media, comparing products, and moderating content. Users implicitly trust that these systems perceive the same visual content as they do. We show that adversarial examples break this assumption, enabling \emph{AI authority laundering}: an attacker subtly perturbs an image so that the VLM produces confident and authoritative responses about the \emph{wrong} input. Unlike jailbreaks or prompt injections, our attacks do not compromise model alignment; the attack operates entirely at the perceptual level. We demonstrate that standard attacks against publicly available CLIP models transfer reliably to production VLMs -- including GPT-5.4, Claude Opus~4.6, Gemini~3, and Grok~4.2. Across four attack surfaces, we show that authority laundering can amplify misinformation, disparage individuals, evade content moderation, and manipulate product recommendations. Our attacks have high success rates: In hundreds of attacks targeting identity manipulation and NSFW evasion, we measure success rates of $22 - 100\%$ across six models. No novel attack algorithm is required: basic techniques known for over a decade suffice, establishing a lower bound on attacker capability that should concern defenders. Our results demonstrate that visual adversarial robustness is now a practical -- and still largely unsolved -- safety problem.

• The paper demonstrates that small adversarial perturbations can manipulate VLMs to confidently produce misleading outputs.
• Empirical results reveal high attack success rates, with models like GPT-5.4 and Grok 4.2 misclassifying adversarial examples in various applications.
• The study highlights the vulnerability of current defenses and calls for ecosystem-level solutions to secure AI authority against misinformation and fraud.

AI Authority Laundering via Adversarial Examples: Technical Summary

Introduction and Context

"Laundering AI Authority with Adversarial Examples" (2605.04261) investigates a new operational threat to deployed vision-LLMs (VLMs): authority laundering. The core insight is that imperceptible adversarial perturbations to image inputs can cause VLMs—including top-tier models like GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro, and Grok 4.2—to produce confident, policy-abiding responses about attacker-chosen semantic content, distinct from what a human observer perceives. This undermines the implicit trust users place in VLMs as factual or compliance authorities on social platforms, search engines, commerce tools, and moderation pipelines. Unlike canonical adversarial “misclassification” demos, authority laundering weaponizes the perceived integrity of VLM outputs to propagate misinformation, evade content filters, disparage identities, and manipulate commercial decisions.

Figure 1: Diverse examples of authority laundering attacks against production VLMs, spanning narrative manipulation, identity manipulation, commercial fraud, and safety filter evasion. Each attack achieves semantic substitution, laundering the attacker’s narrative or request via AI authority.

Threat Model: Authority Laundering

The paper formalizes authority laundering by characterizing two types of authority conferred on VLMs:

• Epistemic authority: VLM outputs shape beliefs; users trust the model's judgments.
• Compliance authority: VLM acceptance or engagement is interpreted as vetting content for safety and policy compliance.

The adversary’s mechanism is a perceptual-discrepancy attack—perturbing an image so the VLM perceives it as a target concept, while the observer sees the benign source content. Success requires (1) the model to further the adversarial objective (e.g., spread misinformation, generate banned content, misidentify products or people), and (2) the observer to view the interaction as legitimate.

Concrete attack families addressed:

• Narrative manipulation: Spreading misinformation (e.g., confirming conspiracy theories).
• Identity manipulation: Misattributing claims to individuals.
• Commercial fraud: Manipulating product recommendations.
• Evasion of safety filters: Bypassing moderation to process/generate disallowed or unsafe content.

The theoretical upper bound is posited via a “perception oracle”: an attacker with perfect control over the model’s perception and prompt. The actual attack instantiates this with adversarial examples, using black-box transfer from ensemble CLIP surrogates to production VLMs, optimized with vanilla PGD and data augmentation.

Empirical Validation: Case Studies

Narrative Manipulation

Authority laundering enables models to validate false narratives. Perturbing images of historical events (e.g., Apollo 11, 9/11, atomic bombings) to match the embedding of "fake news" results in models confidently declaring these events didn't occur.

Figure 2: ChatGPT 5.4 Thinking, when shown an adversarially manipulated 9/11 image, asserts the event is fabricated, echoing conspiracy theory narratives.

Quantitative analysis shows a high attack success rate (ASR): Gemini 3 Pro (100%), Grok 4.2 (97.5%), GPT 5.4 (67.5%), and Llama 4 Maverick (67.5%) misclassify most adversarially manipulated historical images (see Table 1 in paper).

Dangerous Advice and Commercial Fraud

Adversarial examples can induce VLMs to recommend unsafe products. Example: Grok 4.2, given a Tylenol image perturbed toward Roaccutane (unsafe in pregnancy), warns of birth defects, failing to recognize Tylenol's safety profile.

Figure 3: Grok 4.2 gives unsafe medical advice, influenced by an adversarial image matching Roaccutane embedding rather than Tylenol.

Similarly, product recommendation systems are manipulated by adversarially shifting embeddings: ChatGPT advises buying a cheap watch adversarially matched to a Rolex, reversing its original, evidence-based ranking.

Figure 4: ChatGPT recommends a low-quality watch after its image is perturbed toward a Rolex embedding, manipulating product advice.

Identity Manipulation

VLMs are misled to misidentify individuals in images, leading to harmful reputational outcomes. Grok 4.2 identifies Elon Musk as the subject of a drug-dealing news article after the article is perturbed to Musk’s embedding, despite explicit contradictory text.

Figure 5: Grok misidentifies Musk as the subject of a manipulated news article, demonstrating semantic redirection via adversarial perturbation.

The cross-identity manipulation benchmark across ten public figures yields untargeted ASRs up to 95.6%, and targeted ASRs up to 54.4% (Grok 4.2), underscoring broad susceptibility.

Evasion of Moderation and Policy Filters

Adversarial attacks bypass content moderation, allowing circulation of NSFW or policy-violating material. Perturbing explicit images toward doll embeddings evades commercial NSFW detection and causes image-generation VLMs to create cartoon-style versions of disallowed content.

Figure 6: NSFW content evades moderation filters after adversarial perturbation targeting doll embeddings, accepted and processed by VLMs.

Attacks also circumvent gender-asymmetric policy filters and public-figure protections: perturbing female images toward male embeddings allows clothing-removal requests normally denied; perturbing public figure images toward AI-generated faces allows restricted edits or generation.

Figure 7: Gender-asymmetric content filter is bypassed—Grok accepts clothing-removal requests for adversarially perturbed female images previously rejected.

Quantitative Results and Technical Claims

• Attack success rates: Reported 22–100% ASR across six leading VLMs, with high reliability and transferability, requiring only basic PGD/CLIP ensemble attacks.
• No novel algorithm required: Decade-old techniques suffice, defining a minimum attacker capability.
• Evasion of NSFW filters: Commercial moderation tools and VLMs are bypassed with >70% acceptance on adversarially perturbed explicit images.
• Reputation attacks: Identity manipulation leads to misidentification in downstream tasks like reverse image search (see Figure 8).

  Figure 8: Google reverse image search misidentifies an adversarially manipulated Trump image as Elon Musk.

Failure Modes and Limitations

• Attacks are less effective when images have prominent, unperturbed text due to robust OCR pathways; larger perturbations or full-screen attacks are required.
• Perturbed images are not always fully imperceptible and may be detectable by attentive users.
• Verbose VLM outputs sometimes leak semantic traces of the target, betraying the discrepancy to sophisticated observers.

Defensive Landscape

Visual adversarial robustness remains unsolved. Standard defenses (adversarial training, randomized smoothing) yield limited protection, poor scalability, or broken guarantees. Attacks transfer reliably across architectures due to semantically-aligned low-level visual embeddings. Proposed mitigations include exposing VLM reasoning traces and leveraging cryptographic image integrity, but current systems offer no meaningful defense.

Implications and Future Directions

Practical Implications

• Information ecosystems: Trust in AI-output for news, moderation, and product advice is undermined; platforms must recalibrate authority conferred to VLMs.
• Policy and transparency: Restrict reach or tag VLM outputs as potentially manipulated to counteract misinformation propagation.
• Content moderation: Filter pipelines reliant on VLM perception are vulnerable; additional layers or cross-modal inspection required.

Theoretical Implications

• Adversarial robustness: Imperceptible attacks are now operational, not theoretical; adversarial ML research must focus on ecosystem-level interventions beyond model-centric defenses.
• Alignment irrelevance: Authority laundering operates without breaking model alignment—standard safety fine-tuning/behavioral refusal mitigations are ineffective.

Speculation on Future Developments

As VLMs permeate more online ecosystems, authority laundering attacks will likely increase in sophistication, exploiting system trust to scale misinformation, fraud, and policy violations. Further research must address multimodal semantic robustness and the design of end-to-end ecosystem-level defenses, such as cryptographic image provenance and explicit model reasoning disclosures. Regulatory and platform operator responses will become critical as AI authority becomes a channel for adversarial manipulation.

Conclusion

"Laundering AI Authority with Adversarial Examples" presents strong empirical evidence and formal analysis demonstrating that imperceptible adversarial perturbations pose an unsolved threat to deployed VLMs. The attack surface is broad, accessible, and practically exploits the epistemic and compliance authority conferred to VLMs across online information and moderation platforms. Technical and policy responses are urgently needed to secure AI authority in production systems.