Hateful Illusion Dataset: Hidden Hate in Optical Illusions

• The Hateful Illusion Dataset is a benchmark of AI-generated optical illusions that embed hidden hate messages within a benign surface scene and an obscured secondary layer.
• It employs Stable Diffusion and ControlNet to generate 1,860 images, with 1,571 retained after human annotation based on high and low visibility of hidden hate.
• Experiments show that standard moderation classifiers and vision-language models struggle, with accuracy dropping below 0.245, exposing a critical vulnerability in current safety protocols.

Searching arXiv for the dataset paper and closely related work on hateful memes, explainability, and illusion benchmarks. arxiv_search(query="(Qu et al., 30 Jul 2025) hateful illusion dataset", max_results=5) arxiv_search(query="Hateful Illusion dataset (Qu et al., 30 Jul 2025)", max_results=10) The Hateful Illusion Dataset is a benchmark of AI-generated optical illusions that hide hate messages inside otherwise benign-looking scenes, introduced to study whether current moderation systems can detect harmful content that is perceptually split between a visible surface scene and a secondary hidden layer of information (Qu et al., 30 Jul 2025). In this setting, the image’s apparent content is intentionally innocuous, while the hateful content becomes apparent only under alternative viewing conditions such as distance, zooming out, or blurring. The dataset operationalizes a distinct safety problem: not ordinary explicit hate, and not standard multimodal meme reasoning, but moderation of images whose harmful semantics are embedded as optical illusions.

1. Conceptual scope and threat model

The paper defines a hateful illusion as an image with two layers. The first is the surface scene, namely the visible, seemingly harmless content that first draws attention. The second is the hidden hate message, namely a hate speech phrase or hate symbol that becomes apparent only from a distance, when zoomed out, blurred, or with other visual manipulations (Qu et al., 30 Jul 2025). This formulation shifts the moderation problem from overt recognition to layered visual interpretation.

The threat model is explicitly adversarial. Recent text-to-image diffusion systems make it possible to generate optical illusions at scale, and the paper studies how such images may be used to disseminate hate while bypassing content moderation. The risk is not merely that the hate is subtle; it is that the hateful payload is embedded in a perceptual layer that standard image encoders may not prioritize. A central implication of the benchmark is therefore that moderation failure can arise even when a model is competent at explicit hate detection on ordinary images.

This dataset differs structurally from conventional hateful meme benchmarks. In many meme datasets, harmful meaning emerges from the interaction between overlaid text and image context; here, the harmful content is hidden within the image itself as an optical-illusion artifact (Qu et al., 30 Jul 2025). That distinction is important for evaluation, because it targets weaknesses in visual representation rather than only failures of cross-modal fusion.

2. Construction pipeline and generative design

The dataset is generated with a scalable pipeline based on Stable Diffusion and ControlNet (Qu et al., 30 Jul 2025). The paper formalizes the generation process as

$i = \mathcal{F}(x, c)$

where $x$ is a descriptive prompt for the benign-looking surface scene, $c$ is a message image containing the target hate message, $\mathcal{F}$ is the combined Stable Diffusion + ControlNet generator, and $i$ is the resulting image. The adversary first maps a hate message $m$ to a standardized 512×512 message image $c$, and this message image is then used as ControlNet conditioning input.

The hidden-message inventory contains 62 hate messages total, divided into 23 textual hate speech messages and 39 visual hate symbols, with the symbol set spanning 18 distinct symbol classes. Examples mentioned in the paper include Swastika, Happy Merchant, and Confederate Flag (Qu et al., 30 Jul 2025). To diversify the surface scenes, the authors ask GPT-4o to generate 30 scene categories, query Lexica to retrieve high-quality prompts for each category, and randomly sample one prompt per category. This yields 30 descriptive prompts augmented with quality boosters such as “super realistic” and “8K.”

The full generation scale is therefore

$62 \times 30 = 1{,}860$

images. The paper also reports an instructive implementation detail: Stable Diffusion’s built-in safety checker is turned off during generation, and when re-enabled later it blocks only 3.0% of all generated images (Qu et al., 30 Jul 2025). This result functions as an early indication that conventional safeguards are poorly aligned with the hidden-message threat model.

3. Annotation protocol, visibility labels, and dataset composition

After generation, the 1,860 images undergo a two-round human annotation process (Qu et al., 30 Jul 2025). In Round 1, annotators inspect each original image and determine whether they can immediately detect a hidden message from the predefined hate-message set. If the answer is yes, the image is labeled High visibility. Otherwise, it proceeds to Round 2, in which the image is augmented using transformations such as blurring and zooming out. If the hidden message becomes detectable under these conditions, the image is labeled Low visibility; if not, it is labeled No visibility.

Only the high and low visibility images are retained in the final dataset, which the paper collectively names the Hateful Illusion Dataset (Qu et al., 30 Jul 2025). Annotation is performed by 3 expert annotators, final labels are assigned by majority vote, and reliability is reported as Fleiss’ kappa = 0.783 overall. The per-class agreement is 0.860 for hate speech images and 0.725 for hate symbol images.

Aspect	Value	Notes
Total generated images	1,860	62 message images × 30 prompts
Final dataset size	1,571	High + Low visibility
Excluded images	289	No visibility
Hate speech illusions	604	From 23 message images
Hate symbol illusions	967	From 39 message images
High visibility	1,031	Visible in original image
Low visibility	540	Visible under transformations
Annotation agreement	0.783	Fleiss’ kappa overall

The paper further reports an overall success rate of 84.5%, computed as $1571/1860$. The hate speech success rate is 87.5% $(604/690)$, and the hate symbol success rate is 82.6% $x$0 (Qu et al., 30 Jul 2025). These figures indicate that the generation pipeline is effective at producing usable hidden-message images across both textual and symbolic hate targets.

The visibility taxonomy is also methodologically important. High visibility images contain hate messages that are clearly and completely visible in the original image, whereas Low visibility images require transformed viewing conditions. A plausible implication is that the dataset does not merely measure whether a detector can recognize hate; it also measures sensitivity to perceptual accessibility.

4. Benchmarking moderation classifiers and vision-LLMs

The dataset is used to evaluate both six moderation classifiers and nine vision-LLMs (VLMs) (Qu et al., 30 Jul 2025). The moderation classifiers comprise four commercial systems—Omni (OpenAI), SafeSearch (Google), Microsoft Moderation API, and Microsoft Multimodal Moderation API—and two open-source systems—Q16 and the Stable Diffusion safety checker. The metric is accuracy, defined as the percentage of images correctly flagged as hateful out of all tested images.

A key experimental design is the comparison between direct hate-message images and the corresponding hateful illusions. On the direct message images, performance is materially stronger. The best moderation classifier is Q16, with 0.696 accuracy on hate speech and 0.846 on hate symbols. But on the Hateful Illusion Dataset, performance collapses. Q16 remains the best moderation classifier, yet reaches only 0.209 for hate speech and 0.245 for hate symbols; even on High visibility images, detection remains below 0.258 (Qu et al., 30 Jul 2025). The paper notes that SafeSearch, Moderation API, and M-Moderation API are often near 0.0 on these illusion images.

The VLM benchmark is similarly severe. The evaluated families are GPT-Vision (GPT-4V, GPT-4o), Gemini (Gemini-1.5, Gemini-2), LLaVA (LLaVA-1.5, LLaVA-Next), Qwen-VL, and CogVLM (CogVLM, CogVLM-2). Inference is run with temperature = 1.0 and max new tokens = 1024, under three progressively more explicit zero-shot prompts and a chain-of-thought prompting condition (Qu et al., 30 Jul 2025).

On direct hate-message images, VLMs perform reasonably well; for example, GPT-4o reaches 1.000 for both hate speech and hate symbols. On hateful illusions, however, zero-shot accuracy is extremely low: the best hate speech result is 0.034 from Gemini-2, and the best hate symbol result is 0.060 from LLaVA-1.5. Chain-of-thought improves some results, but only modestly: the best hate speech accuracy becomes 0.102 and the best hate symbol accuracy 0.090, both from LLaVA-Next (Qu et al., 30 Jul 2025). The paper summarizes the practical consequence directly: detection accuracy falls below 0.245 for moderation classifiers and below 0.102 for VLMs.

The authors also introduce an explainable detection task in which the model must identify the specific hidden message rather than only answer yes or no. Evaluation uses LLaMA-3.1 as a judge, with reported judge accuracy of 0.991 for hate speech, 0.940 for hate symbols, and 1.000 for digits (Qu et al., 30 Jul 2025). Model performance remains extremely low. For example, GPT-4o scores 0.056 on hidden hate speech, 0.065 on hidden hate symbols, and 0.074 on digit illusions. This indicates that failure is not limited to binary moderation; the models often cannot recover the concealed content itself.

5. Failure analysis and mechanistic interpretation

The paper attributes the benchmark failures primarily to the vision encoders, especially those based on ViT / CLIP-ViT, which focus on surface-level image details while overlooking the secondary hidden-message layer (Qu et al., 30 Jul 2025). This is not framed as a lack of hate knowledge alone, but as a representational bias toward the wrong visual abstraction.

Three forms of evidence are presented. First, using CLIP-L/14 embeddings and t-SNE, the authors show that images generated with the same descriptive prompt cluster together regardless of which hidden message was embedded. Second, cosine-similarity analysis shows that image embeddings from hateful illusions are more similar to their descriptive prompts than to the hidden message text. Third, CLIP attention or relevancy maps emphasize salient scene objects such as “collapse,” “truck,” and “red bricks,” while ignoring the hidden target, including targets such as “female symbol” or “misogyny” (Qu et al., 30 Jul 2025).

The paper offers three reasons for this limitation. The first is training data mismatch: AI-generated optical illusions are underrepresented in the web-scale image-text corpora used to train CLIP-like encoders. The second is training objective bias: contrastive image-text pretraining favors ordinary captions that describe salient surface content, not hidden secondary meanings. The third is static attention: ViT-style attention is described as fixed or static relative to human visual perception, which can shift dynamically between local detail and global pattern.

A broader research context reinforces this interpretation. A related optical-illusion benchmark, IllusionVQA, reports large human-model gaps on illusion comprehension and localization, with GPT-4V reaching 62.99% on comprehension and 49.7% on localization, while humans achieve 91.03% and 100%, respectively (Shahgir et al., 2024). This suggests that optical illusions already stress VLM perception in non-safety settings; the Hateful Illusion Dataset adds a safety-critical hidden-message layer on top of that pre-existing weakness.

6. Mitigation strategies and relation to adjacent datasets

The paper evaluates two mitigation directions: image transformations and training-level strategies (Qu et al., 30 Jul 2025). Among the transformations tested—Gaussian blur, downscaling, grid repetition, grayscale, histogram equalization, gamma correction, gradient magnitude / edge detection, and combinations thereof—the most effective is Gaussian blur followed by histogram equalization. This raises GPT-4o accuracy from 0.03 to 0.70, an improvement of +0.67, and raises Q16 from 0.21 to 0.82, an improvement of +0.61. The result indicates that suppressing local texture while amplifying global structure can make the hidden layer more machine-accessible.

The training-level mitigation uses a specialized CLIP-based classifier trained on the Hateful Illusion Dataset plus clean optical-illusion examples. The training pool combines approximately 1.5K hateful illusion images and approximately 1.5K benign optical illusion images, for a total of approximately 3K images, using an 80/20 train/test split (Qu et al., 30 Jul 2025). The paper compares linear probing and prompt learning, each with frozen CLIP and with full fine-tuning. The best result is fully fine-tuned prompt learning, which achieves 0.938 accuracy, 0.931 precision, 0.947 recall, and 0.939 F1.

Within the broader dataset landscape, the Hateful Illusion Dataset occupies a distinct niche. Datasets such as HatReD add free-text reasons for hateful memes (Hee et al., 2023), MemeXplain/MemeIntel augment hateful memes with explanations (Kmainasi et al., 23 Feb 2025), and later work extends the Hateful Memes benchmark with fine-grained labels, human explanations, and distilled step-by-step chain-of-thought rationales for reasoning-supervised MLLMs (Kmainasi et al., 1 Mar 2026). Other systems use OCR, captioning, retrieval, and VQA to detect subtle latent hate in memes where meaning emerges from image-text interaction (Anaissi et al., 23 Apr 2025). By contrast, the Hateful Illusion Dataset is neither primarily a meme dataset nor primarily an explanation dataset. Its core contribution is a benchmark for hidden-message moderation in AI-generated optical illusions.

A common misconception is to treat it as a generic hateful-image benchmark. The paper’s emphasis is narrower and more technical: it studies images that are deliberately engineered to bypass moderation by hiding hate in a secondary perceptual layer (Qu et al., 30 Jul 2025). Another misconception is that ordinary image-safety filters should suffice. The reported 3.0% block rate from the re-enabled Stable Diffusion safety checker, together with the near-collapse of both moderation APIs and general-purpose VLMs, directly contradicts that assumption.

The dataset therefore functions as both a benchmark and a stress test. It exposes a failure regime in which explicit hate may be detectable, yet hidden hate in optical illusions is missed almost completely (Qu et al., 30 Jul 2025). In that sense, its significance lies less in scale than in problem formulation: it isolates a moderation blind spot that standard hateful meme and hateful image benchmarks do not directly test.