Ex-ToxiCN-MM: Chinese Harmful Meme Dataset

• The paper introduces Ex-ToxiCN-MM as the first dataset featuring dual interpretations per meme to enhance the analysis of harmful content in Chinese online culture.
• It employs a paired explanation structure with both harmful and non-harmful readings, augmented by the RIKE framework and C-HarmKB to guide interpretation.
• Empirical evaluations demonstrate significant gains in detection and explanation quality, particularly in handling semantic ambiguity and deep cultural context.

Ex-ToxiCN-MM, short for Explanatory Toxic Chinese Multimodal Memes, is the first Chinese harmful meme explanation dataset. It was introduced to study harmful meme attribution under two conditions that are especially difficult for Chinese online content: semantic ambiguity, in which the same meme can support both harmful and benign readings, and deep cultural grounding, in which correct interpretation depends on Chinese internet slang, sociocultural references, and double entendre. Each meme is paired with two opposing explanations—a harmful interpretation and a non-harmful interpretation—and the dataset is released together with the RIKE attribution analysis framework and the Chinese Harmful Semantic Knowledge Base (C-HarmKB) (Wang et al., 23 May 2026).

1. Research problem and conceptual design

Ex-ToxiCN-MM was constructed around a specific claim about harmful meme analysis in Chinese: a meme’s harmfulness often cannot be determined from surface text or image content alone. The paper identifies two central obstacles. First, accurate assessment depends heavily on deep cultural context. Second, many memes are semantically ambiguous, so harmfulness is often subjective rather than directly observable (Wang et al., 23 May 2026).

The dataset’s defining design choice is to require dual-opposite interpretations per meme. Every sample contains a plausible non-harmful interpretation, which gives a benign or good-faith reading, and a harmful interpretation, which explains how the meme can be toxic in Chinese cultural context. This forces models to reason across competing readings rather than collapse the task into single-label pattern matching.

This design also changes what “interpretability” means in the benchmark. Ex-ToxiCN-MM does not treat explanation as a post-hoc attribution layer added after classification. Instead, explanation is part of the supervision itself: the model is expected to distinguish between two interpretive paths and justify the final decision. A plausible implication is that the benchmark evaluates not only recognition of harmful cues, but also the ability to reject superficially plausible benign readings when cultural knowledge supports a harmful one.

2. Dataset composition and representational schema

The dataset contains 7,042 total memes, with 3,735 labeled harmful and 3,307 labeled non-harmful. Because each meme has both a harmful and a non-harmful explanation, the corpus contains 14,084 explanatory texts. The reported explanation length is 33.29 Chinese characters on average, with standard deviation 7.40 (Wang et al., 23 May 2026).

For modeling, each meme is represented as

$M = (I, T, X),$

where $I$ is the image, $T$ is the embedded text, and $X$ is an image description. Each sample also contains:

• a binary harmful/non-harmful label,
• two opposing explanations,
• and, for harmful memes, a harmful subtype.

The harmful subset is reported with the following per-type counts:

Harmful type	Total	Train / Test
Targeted Harm	889	711 / 178
General Offense	1,078	862 / 216
Sexual Innuendo	1,367	1,093 / 274
Disparaging Culture	1,040	862 / 178

The paper reports 2,988 harmful memes in train and 747 in test. For the 3,307 non-harmful memes, specific train/test counts are not detailed in the provided excerpt.

The paired-interpretation structure is the dataset’s core formal property. The non-harmful interpretation is defined as a plausible reading that minimizes offense, often by treating a meme as humor, self-deprecation, literal meaning, or rhetorical exaggeration. The harmful interpretation is a culturally grounded reading that identifies the derogatory mechanism, such as a slur, stereotype, or metaphor, and articulates the attack target and reasoning chain.

3. Annotation procedure, examples, and quality control

The annotation process used six primary annotators and three senior reviewers, all native Mandarin speakers with deep familiarity with Chinese online culture and strong analytical writing skills. The annotation manual covered definitions of harmful memes, interpretation requirements, tool usage, and ethics. Before full annotation, the team ran three trial rounds of 100 memes each to calibrate judgments (Wang et al., 23 May 2026).

Annotation followed a two-stage “Collect-and-Edit” framework adapted from Wiegreffe & Marasović. In Stage 1, Annotator A drafted both opposing interpretations, $Exp_{n_h}(M)$ and $Exp_h(M)$. In Stage 2, Annotator B verified cultural references and slang accuracy, improved logic and fluency, and negotiated disagreements with Annotator A; unresolved cases were adjudicated by a researcher.

Quality control was unusually explicit. Three reviewers scored each explanation on five 5-point Likert dimensions:

• Informativeness
• Soundness
• Cultural Relevance
• Conciseness
• Persuasiveness

The target length was approximately 30 characters. Any explanation receiving below 2.5 on any dimension from any reviewer was flagged and re-annotated by a separate group until it met the standard. The paper reports strong aggregate scores, including 4.72 / 4.81 / 4.05 / 4.53 / 4.79 for harmful explanations of harmful memes on the five dimensions above, and 4.21 / 4.65 / 3.55 / 4.58 / 4.71 for non-harmful explanations of non-harmful memes (Wang et al., 23 May 2026).

The dataset examples illustrate the intended ambiguity. For “vegetable dog” (菜狗), the non-harmful reading treats the phrase as self-mockery because “dog” can be self-deprecating in Chinese internet slang, whereas the harmful reading interprets it as derogatory labeling of someone as a “loser,” encouraging conflict and negative values. For memes contrasting graduation aspirations with “salted fish” (咸鱼), the non-harmful reading frames the meme as self-deprecating satire about ideals versus reality, while the harmful reading treats it as disparaging people who “accomplish nothing,” with potentially harmful effects on vulnerable audiences.

A common misconception is that the paired explanations merely restate two labels. In the annotation design, they are intended to encode opposing interpretive rationales, not just class-conditioned paraphrases.

4. C-HarmKB and the RIKE attribution framework

Ex-ToxiCN-MM was released together with C-HarmKB, a Chinese Harmful Semantic Knowledge Base containing 2,870 entries. Its purpose is to supply prior cultural knowledge—slang, stereotypes, historical references, and related concepts—that general LLMs and MLLMs often lack (Wang et al., 23 May 2026).

C-HarmKB was expanded from an existing Chinese insult lexicon. Definitions and contexts were crawled from Baidu Baike, Wikipedia, and specialized subculture forums, and the resource was further enriched with contextual items harvested during Ex-ToxiCN-MM annotation. Three native Chinese speakers categorized entries into classes such as Sexism, Racism, Region, LGBTQ, and Others. Each entry includes a definition with cultural context, origin, and harmful connotations.

The associated modeling framework is RIKE, which has two sequential components:

1. AKE (Attribution Knowledge Enhancement) A retrieval-augmented pipeline tailored to harmful meme attribution. It performs dynamic query expansion, hybrid retrieval (BM25 + dense), LLM-based re-ranking, and threshold filtering to construct knowledge $K(M)$ for each meme.
2. RIR (Relative Intent Reasoning) An instruction-tuned reasoning module that takes the meme input, the two opposing interpretations, and the retrieved knowledge, then produces both a label and a concise justification.

The decision function is reported as

$$y^* = C_{\theta}(I, T, X, Exp_{n_h}(M), Exp_h(M), K(M)).$$

The classification and explanation objectives are given as:

$$L_{\text{clf}} = -\log P(y^* \mid M, Exp_{n_h}(M), Exp_h(M), K(M)),$$

and

$$L_{\text{itf}} = -\log P(Exp(M) \mid M, Exp_{n_h}(M), Exp_h(M), K(M)).$$

Training uses supervised fine-tuning with LoRA. The model is instruction-tuned to weigh the harmful and non-harmful interpretive paths relative to the retrieved cultural knowledge. This is the technical meaning of “relative intent reasoning” in the paper: the classifier is not asked to score a meme in isolation, but to resolve a contrast between competing explanations under external cultural evidence.

The AKE stage is also important methodologically. The knowledge base text is chunked, embedded with a pre-trained Chinese embedding model, stored in a vector DB, and retrieved through hybrid BM25 + dense retrieval with LLM-based re-ranking and a relevance threshold. This suggests that the framework treats background knowledge as selective evidence rather than as a static prompt appendix.

5. Evaluation protocol and empirical results

The evaluation has two tasks. The first is harmful meme detection on the full test set. The second is explanation generation, evaluated only on harmful memes classified as harmful, so that explanation quality is not conflated with classification failure (Wang et al., 23 May 2026).

Detection is evaluated with Accuracy, Precision, Recall, and F1 for the harmful class. Explanation generation is evaluated with BLEU-4 and ROUGE-L against gold explanations, plus an LLM-as-a-judge setup using Qwen-VL-Plus over the same five Likert dimensions used in data validation.

The baseline set includes open-source models such as Qwen2.5-VL-3B, Qwen2.5-VL-7B, InternVL3_5-8B, and LLaVA-1.5-7B, as well as closed-source models such as Qwen-VL-Max and GLM-4V-Flash. The paper compares Base, RIR, RIR+BK, SFT, SFT+RIR, and RIKE settings.

Selected detection results under the full RIKE configuration are:

Model	Acc	F1
Qwen2.5-VL-3B	0.948	0.952
Qwen2.5-VL-7B	0.957	0.960
InternVL3_5-8B	0.967	0.969
LLaVA-1.5-7B	0.915	0.915
Qwen-VL-Max	0.757	0.765

The ablations isolate the contribution of the framework’s parts. On Qwen2.5-VL-3B, RIR increases recall from 0.364 to 0.772; adding background knowledge in RIR+BK increases F1 from 0.654 to 0.710; and RIKE reaches 0.952 F1. Across open-source models, SFT+RIR already approaches near-ceiling performance, and RIKE consistently yields the best overall metrics. The paper summarizes this as evidence that relative intent reasoning improves recall on ambiguous cases, while knowledge augmentation improves precision–recall balance.

Explanation generation results show similarly large gains. For Qwen2.5-VL-7B, BLEU-4 improves from 14.17 to 51.69, and ROUGE-L from 29.31 to 62.51 under RIKE. For InternVL3_5-8B, BLEU-4 improves from 8.86 to 53.48, and ROUGE-L from 28.28 to 64.38. LLM-judge scores also improve, though by smaller margins, including Informativeness 2.73 → 2.84 and Cultural Relevance 2.51 → 2.64 for Qwen2.5-VL-7B.

The qualitative analysis explains where these gains come from. In a case involving the phrase “working like a horse,” the knowledge-augmented model links the expression to intense labor and negative connotations instead of reading it as neutral adult effort. In an “Asian parents” stereotype case, the model with C-HarmKB surfaces parental-pressure satire as a harmful aspect rather than defaulting to a benign family-care interpretation. In a visually ambiguous case misread as “mouth-to-mouth resuscitation,” the relative reasoning setup helps recover the intended violent meaning.

6. Limitations, ethics, and position within Chinese toxicity research

Ex-ToxiCN-MM is designed to reduce ambiguity, not eliminate it. The paper explicitly notes that subjectivity persists even with opposing explanations and RIKE, because value-laden judgments differ across cultures and individuals (Wang et al., 23 May 2026). It also notes that C-HarmKB has finite coverage: long-tail slang and rapidly evolving memes may be absent, so routine updates are necessary.

Several implementation details are not fully specified in the paper text, including the precise OCR, captioning, and some hyperparameters. The authors also note that BLEU and ROUGE may under-reward strong closed-source models because stylistic mismatch can depress n-gram overlap even when explanations are substantively good. The paper therefore argues for better human-aligned explanation metrics.

The ethical framing is explicit. The dataset contains potentially offensive content. Annotators were trained on ethics, and multi-reviewer checks were used to reduce harm and bias. At the same time, the release carries clear misuse risk: systems that interpret or generate culturally grounded harmful content could be repurposed for targeted harassment, so the paper frames the resource as intended for research and safety tooling.

Within adjacent Chinese toxicity research, Ex-ToxiCN-MM occupies a distinct position. STATE ToxiCN is a text-only, span-level Chinese hate speech dataset centered on Target-Argument-Hateful-Group quadruples, and its paper explicitly states that the term “Ex-ToxiCN-MM” does not appear there; it is discussed only as a possible multimodal extension of STATE ToxiCN rather than as an existing benchmark (Bai et al., 26 Jan 2025). Ex-ToxiCN-MM instead addresses a different object—multimodal memes—and a different supervision signal—paired opposing explanations. A broader multimodal toxicity parallel exists outside Chinese memes as well: ToxVidLLM studies code-mixed Hindi-English video toxicity with multimodal synchronization and multitask learning, illustrating that multimodal toxicity research increasingly depends on modality-specific context rather than text alone (Maity et al., 2024).

Taken together, Ex-ToxiCN-MM defines a particular research agenda for Chinese harmful content analysis: models should not merely detect toxicity, but should resolve competing interpretations under retrieved cultural knowledge and produce concise, culturally grounded justifications. That combination of paired explanations, knowledge augmentation, and relative reasoning is the resource’s principal contribution.