---
title: 'Hateful Illusion Dataset: Hidden Hate in Optical Illusions'
url: https://www.emergentmind.com/topics/hateful-illusion-dataset
type: topic
---

# Hateful Illusion Dataset: Hidden Hate in Optical Illusions

Searching arXiv for the dataset paper and closely related work on hateful memes, explainability, and illusion benchmarks.
arxiv_search(query="2507.22617 hateful illusion dataset", max_results=5)
arxiv_search(query="Hateful Illusion dataset 2507.22617", max_results=10)
The Hateful Illusion Dataset is a benchmark of **AI-generated optical illusions that hide hate messages inside otherwise benign-looking scenes**, introduced to study whether current moderation systems can detect harmful content that is perceptually split between a visible surface scene and a secondary hidden layer of information [2507.22617]. In this setting, the image’s apparent content is intentionally innocuous, while the hateful content becomes apparent only under alternative viewing conditions such as distance, zooming out, or blurring. The dataset operationalizes a distinct safety problem: not ordinary explicit hate, and not standard multimodal meme reasoning, but moderation of images whose harmful semantics are embedded as optical illusions.

## 1. Conceptual scope and threat model

The paper defines a **hateful illusion** as an image with two layers. The first is the **surface scene**, namely the visible, seemingly harmless content that first draws attention. The second is the **hidden hate message**, namely a hate speech phrase or hate symbol that becomes apparent only from a distance, when zoomed out, blurred, or with other visual manipulations [2507.22617]. This formulation shifts the moderation problem from overt recognition to layered visual interpretation.

The threat model is explicitly adversarial. Recent text-to-image diffusion systems make it possible to generate optical illusions at scale, and the paper studies how such images may be used to disseminate hate while bypassing content moderation. The risk is not merely that the hate is subtle; it is that the hateful payload is embedded in a perceptual layer that standard image encoders may not prioritize. A central implication of the benchmark is therefore that moderation failure can arise even when a model is competent at explicit hate detection on ordinary images.

This dataset differs structurally from conventional hateful meme benchmarks. In many meme datasets, harmful meaning emerges from the interaction between overlaid text and image context; here, the harmful content is hidden **within the image itself** as an optical-illusion artifact [2507.22617]. That distinction is important for evaluation, because it targets weaknesses in visual representation rather than only failures of cross-modal fusion.

## 2. Construction pipeline and generative design

The dataset is generated with a scalable pipeline based on **Stable Diffusion** and **ControlNet** [2507.22617]. The paper formalizes the generation process as

$$
i = \mathcal{F}(x, c)
$$

where \(x\) is a descriptive prompt for the benign-looking surface scene, \(c\) is a message image containing the target hate message, \(\mathcal{F}\) is the combined Stable Diffusion + ControlNet generator, and \(i\) is the resulting image. The adversary first maps a hate message \(m\) to a standardized **512×512 message image** \(c\), and this message image is then used as ControlNet conditioning input.

The hidden-message inventory contains **62 hate messages total**, divided into **23 textual hate speech messages** and **39 visual hate symbols**, with the symbol set spanning **18 distinct symbol classes**. Examples mentioned in the paper include **Swastika**, **Happy Merchant**, and **Confederate Flag** [2507.22617]. To diversify the surface scenes, the authors ask **GPT-4o** to generate **30 scene categories**, query **Lexica** to retrieve high-quality prompts for each category, and randomly sample one prompt per category. This yields **30 descriptive prompts** augmented with quality boosters such as “super realistic” and “8K.”

The full generation scale is therefore

$$
62 \times 30 = 1{,}860
$$

images. The paper also reports an instructive implementation detail: **Stable Diffusion’s built-in safety checker is turned off during generation**, and when re-enabled later it blocks only **3.0%** of all generated images [2507.22617]. This result functions as an early indication that conventional safeguards are poorly aligned with the hidden-message threat model.

## 3. Annotation protocol, visibility labels, and dataset composition

After generation, the **1,860** images undergo a **two-round human annotation process** [2507.22617]. In **Round 1**, annotators inspect each original image and determine whether they can immediately detect a hidden message from the predefined hate-message set. If the answer is yes, the image is labeled **High visibility**. Otherwise, it proceeds to **Round 2**, in which the image is augmented using transformations such as **blurring** and **zooming out**. If the hidden message becomes detectable under these conditions, the image is labeled **Low visibility**; if not, it is labeled **No visibility**.

Only the **high** and **low** visibility images are retained in the final dataset, which the paper collectively names the **Hateful Illusion Dataset** [2507.22617]. Annotation is performed by **3 expert annotators**, final labels are assigned by **majority vote**, and reliability is reported as **Fleiss’ kappa = 0.783 overall**. The per-class agreement is **0.860** for hate speech images and **0.725** for hate symbol images.

| Aspect | Value | Notes |
|---|---:|---|
| Total generated images | 1,860 | 62 message images × 30 prompts |
| Final dataset size | 1,571 | High + Low visibility |
| Excluded images | 289 | No visibility |
| Hate speech illusions | 604 | From 23 message images |
| Hate symbol illusions | 967 | From 39 message images |
| High visibility | 1,031 | Visible in original image |
| Low visibility | 540 | Visible under transformations |
| Annotation agreement | 0.783 | Fleiss’ kappa overall |

The paper further reports an **overall success rate of 84.5%**, computed as \(1571/1860\). The **hate speech success rate** is **87.5%** \((604/690)\), and the **hate symbol success rate** is **82.6%** \((967/1170)\) [2507.22617]. These figures indicate that the generation pipeline is effective at producing usable hidden-message images across both textual and symbolic hate targets.

The visibility taxonomy is also methodologically important. **High visibility** images contain hate messages that are clearly and completely visible in the original image, whereas **Low visibility** images require transformed viewing conditions. A plausible implication is that the dataset does not merely measure whether a detector can recognize hate; it also measures sensitivity to perceptual accessibility.

## 4. Benchmarking moderation classifiers and vision-language models

The dataset is used to evaluate both **six moderation classifiers** and **nine vision-language models (VLMs)** [2507.22617]. The moderation classifiers comprise four commercial systems—**Omni (OpenAI)**, **SafeSearch (Google)**, **Microsoft Moderation API**, and **Microsoft Multimodal Moderation API**—and two open-source systems—**Q16** and the **Stable Diffusion safety checker**. The metric is **accuracy**, defined as the percentage of images correctly flagged as hateful out of all tested images.

A key experimental design is the comparison between **direct hate-message images** and the corresponding **hateful illusions**. On the direct message images, performance is materially stronger. The best moderation classifier is **Q16**, with **0.696** accuracy on hate speech and **0.846** on hate symbols. But on the Hateful Illusion Dataset, performance collapses. **Q16** remains the best moderation classifier, yet reaches only **0.209** for hate speech and **0.245** for hate symbols; even on **High visibility** images, detection remains below **0.258** [2507.22617]. The paper notes that **SafeSearch**, **Moderation API**, and **M-Moderation API** are often near **0.0** on these illusion images.

The VLM benchmark is similarly severe. The evaluated families are **GPT-Vision** (GPT-4V, GPT-4o), **Gemini** (Gemini-1.5, Gemini-2), **LLaVA** (LLaVA-1.5, LLaVA-Next), **Qwen-VL**, and **CogVLM** (CogVLM, CogVLM-2). Inference is run with **temperature = 1.0** and **max new tokens = 1024**, under three progressively more explicit zero-shot prompts and a **chain-of-thought prompting** condition [2507.22617].

On **direct hate-message images**, VLMs perform reasonably well; for example, **GPT-4o** reaches **1.000** for both hate speech and hate symbols. On **hateful illusions**, however, zero-shot accuracy is extremely low: the best hate speech result is **0.034** from **Gemini-2**, and the best hate symbol result is **0.060** from **LLaVA-1.5**. Chain-of-thought improves some results, but only modestly: the best hate speech accuracy becomes **0.102** and the best hate symbol accuracy **0.090**, both from **LLaVA-Next** [2507.22617]. The paper summarizes the practical consequence directly: detection accuracy falls below **0.245** for moderation classifiers and below **0.102** for VLMs.

The authors also introduce an **explainable detection task** in which the model must identify the **specific hidden message** rather than only answer yes or no. Evaluation uses **LLaMA-3.1** as a judge, with reported judge accuracy of **0.991** for hate speech, **0.940** for hate symbols, and **1.000** for digits [2507.22617]. Model performance remains extremely low. For example, **GPT-4o** scores **0.056** on hidden hate speech, **0.065** on hidden hate symbols, and **0.074** on digit illusions. This indicates that failure is not limited to binary moderation; the models often cannot recover the concealed content itself.

## 5. Failure analysis and mechanistic interpretation

The paper attributes the benchmark failures primarily to the **vision encoders**, especially those based on **ViT / CLIP-ViT**, which focus on **surface-level image details** while overlooking the **secondary hidden-message layer** [2507.22617]. This is not framed as a lack of hate knowledge alone, but as a representational bias toward the wrong visual abstraction.

Three forms of evidence are presented. First, using **CLIP-L/14 embeddings** and **t-SNE**, the authors show that images generated with the same descriptive prompt cluster together **regardless of which hidden message was embedded**. Second, cosine-similarity analysis shows that image embeddings from hateful illusions are more similar to their **descriptive prompts** than to the hidden **message text**. Third, CLIP attention or relevancy maps emphasize salient scene objects such as “collapse,” “truck,” and “red bricks,” while ignoring the hidden target, including targets such as “female symbol” or “misogyny” [2507.22617].

The paper offers three reasons for this limitation. The first is **training data mismatch**: AI-generated optical illusions are underrepresented in the web-scale image-text corpora used to train CLIP-like encoders. The second is **training objective bias**: contrastive image-text pretraining favors ordinary captions that describe salient surface content, not hidden secondary meanings. The third is **static attention**: ViT-style attention is described as fixed or static relative to human visual perception, which can shift dynamically between local detail and global pattern.

A broader research context reinforces this interpretation. A related optical-illusion benchmark, **IllusionVQA**, reports large human-model gaps on illusion comprehension and localization, with **GPT-4V** reaching **62.99%** on comprehension and **49.7%** on localization, while humans achieve **91.03%** and **100%**, respectively [2403.15952]. This suggests that optical illusions already stress VLM perception in non-safety settings; the Hateful Illusion Dataset adds a safety-critical hidden-message layer on top of that pre-existing weakness.

## 6. Mitigation strategies and relation to adjacent datasets

The paper evaluates two mitigation directions: **image transformations** and **training-level strategies** [2507.22617]. Among the transformations tested—**Gaussian blur**, **downscaling**, **grid repetition**, **grayscale**, **histogram equalization**, **gamma correction**, **gradient magnitude / edge detection**, and combinations thereof—the most effective is **Gaussian blur followed by histogram equalization**. This raises **GPT-4o** accuracy from **0.03 to 0.70**, an improvement of **+0.67**, and raises **Q16** from **0.21 to 0.82**, an improvement of **+0.61**. The result indicates that suppressing local texture while amplifying global structure can make the hidden layer more machine-accessible.

The training-level mitigation uses a specialized **CLIP-based classifier** trained on the Hateful Illusion Dataset plus clean optical-illusion examples. The training pool combines approximately **1.5K hateful illusion images** and approximately **1.5K benign optical illusion images**, for a total of approximately **3K** images, using an **80/20** train/test split [2507.22617]. The paper compares **linear probing** and **prompt learning**, each with frozen CLIP and with **full fine-tuning**. The best result is **fully fine-tuned prompt learning**, which achieves **0.938 accuracy**, **0.931 precision**, **0.947 recall**, and **0.939 F1**.

Within the broader dataset landscape, the Hateful Illusion Dataset occupies a distinct niche. Datasets such as **HatReD** add free-text reasons for hateful memes [2305.17678], **MemeXplain/MemeIntel** augment hateful memes with explanations [2502.16612], and later work extends the **Hateful Memes** benchmark with **fine-grained labels**, **human explanations**, and **distilled step-by-step chain-of-thought rationales** for reasoning-supervised MLLMs [2603.01225]. Other systems use OCR, captioning, retrieval, and VQA to detect subtle latent hate in memes where meaning emerges from image-text interaction [2504.16723]. By contrast, the Hateful Illusion Dataset is neither primarily a meme dataset nor primarily an explanation dataset. Its core contribution is a benchmark for **hidden-message moderation in AI-generated optical illusions**.

A common misconception is to treat it as a generic hateful-image benchmark. The paper’s emphasis is narrower and more technical: it studies images that are **deliberately engineered to bypass moderation by hiding hate in a secondary perceptual layer** [2507.22617]. Another misconception is that ordinary image-safety filters should suffice. The reported **3.0%** block rate from the re-enabled Stable Diffusion safety checker, together with the near-collapse of both moderation APIs and general-purpose VLMs, directly contradicts that assumption.

The dataset therefore functions as both a benchmark and a stress test. It exposes a failure regime in which explicit hate may be detectable, yet **hidden hate in optical illusions is missed almost completely** [2507.22617]. In that sense, its significance lies less in scale than in problem formulation: it isolates a moderation blind spot that standard hateful meme and hateful image benchmarks do not directly test.

Source: https://www.emergentmind.com/topics/hateful-illusion-dataset