RIKE: Chinese Harmful Meme Attribution

• RIKE is a comprehensive framework for interpretable detection of Chinese harmful memes, integrating multimodal inputs and culturally anchored knowledge.
• It employs an Attribution Knowledge Enhancement module to dynamically retrieve cultural context and enrich multimodal features through query expansion and hybrid scoring.
• The Relative Intent Reasoning module contrasts harmful and non-harmful explanations, significantly boosting classification accuracy and explanation quality.

Searching arXiv for the RIKE framework and closely related Chinese harmful meme detection work. RIKE is a comprehensive attribution analysis framework for interpretable detection of Chinese harmful memes, introduced in the context of the Ex-ToxiCN-MM benchmark and the Chinese Harmful Semantic Knowledge Base (C-HarmKB). It treats harmful-meme attribution as a joint multimodal classification and explanation-generation problem in which each meme is represented as $M=(I,T,X)$, with $I$ the raw image, $T$ the overlaid text string, and $X$ an auxiliary caption or textual description of the image. The framework is designed for cases in which harmfulness is culturally grounded, semantically ambiguous, and therefore not reducible to surface lexical cues alone. Its central architectural components are the Attribution Knowledge Enhancement module (AKE), which retrieves and injects culturally specific prior knowledge, and the Relative Intent Reasoning module (RIR), which explicitly contrasts harmful and non-harmful interpretations for the same meme (Wang et al., 23 May 2026).

1. Problem setting and motivation

The motivating task is harmful-meme attribution in Chinese, where progress is reported to lag due to two stated difficulties: accurately assessing harmfulness depends heavily on understanding deep cultural context, and many memes are semantically ambiguous, making harmfulness highly subjective (Wang et al., 23 May 2026). RIKE is therefore positioned not merely as a binary classifier, but as a mechanism for distinguishing between competing readings of the same multimodal artifact.

In the underlying formulation, the model receives the meme image $I\in\mathbb{R}^{H\times W\times 3}$, its overlaid text $T$, an auxiliary caption $X$, and a retrieved set of cultural-knowledge snippets $K(M)=\{d_1,\dots,d_k\}$ drawn from C-HarmKB. The target output is two-fold: a harmful/non-harmful label $y\in\{0,1\}$ and a concise explanation $\mathrm{Exp}(M)$. This is formalized through a model $I$0 with classification and explanation heads,

$I$1

The paper’s framing suggests that interpretability is not an auxiliary reporting layer but part of the task definition itself, because correct attribution depends on identifying which culturally situated interpretation is better supported by the evidence (Wang et al., 23 May 2026).

A distinguishing feature of the setup is the use of opposing explanations. Ex-ToxiCN-MM provides both “harmful” and “non-harmful” interpretations for each meme. This makes the task closer to comparative adjudication than to ordinary single-label annotation, and it motivates RIKE’s relative-intent formulation.

2. Data resources and knowledge grounding

RIKE is introduced together with two resources: Ex-ToxiCN-MM and C-HarmKB. Ex-ToxiCN-MM is described as the first Chinese harmful meme explanation dataset and contains 7,042 memes, of which 3,735 are harmful and 3,307 are non-harmful, using an 80/20 train/test split (Wang et al., 23 May 2026). The dataset provides opposing interpretations categorized as “harmful” and “non-harmful” for each meme. This design is structurally aligned with the later RIR module, which scores alternative intent hypotheses rather than assuming a single canonical reading.

C-HarmKB is a specialized knowledge base of Chinese cultural concepts and offensive vocabulary. Each retrieved snippet $I$2 may contain Chinese-specific slang, historical allusions, or definitions. The knowledge base is encoded as a vector store and queried by hybrid attention/keyword scoring. The paper states that this prior knowledge is meant to supply models with essential background that is often absent from the meme text and image alone (Wang et al., 23 May 2026).

The interaction between dataset and knowledge base is central. Ex-ToxiCN-MM operationalizes ambiguity by furnishing competing human-written explanations, while C-HarmKB operationalizes background knowledge by furnishing culturally anchored snippets. A plausible implication is that RIKE is optimized for cases in which harmfulness arises from inferential context rather than explicit insult tokens, especially when textual and visual signals are individually underdetermined.

3. Attribution Knowledge Enhancement

The Attribution Knowledge Enhancement module addresses ambiguity and lack of background knowledge through four stated components: a dynamic query-expansion LLM, a hybrid retriever over C-HarmKB, an encoder for retrieved snippets, and a fusion/gating mechanism that injects knowledge into multimodal features (Wang et al., 23 May 2026).

Dynamic query expansion begins from $I$3 and $I$4. A lightweight LLM proposes additional keywords $I$5, after which the full query set is defined as

$I$6

filtered to non-empty queries. Retrieval then combines sparse and dense scoring. For each candidate document $I$7 and each query $I$8, the hybrid score is

$I$9

A reranker subsequently estimates posterior relevance using yes/no logits:

$T$0

Only documents with $T$1 are retained, and the top-$T$2 unique snippets form $T$3 (Wang et al., 23 May 2026).

The retrieved knowledge is encoded alongside visual and textual features:

$T$4

The snippet embeddings are aggregated either by mean-pooling or attention,

$T$5

Knowledge injection is implemented through gated fusion:

$T$6

$T$7

The result $T$8 initializes the downstream reasoning module (Wang et al., 23 May 2026).

The reported ablation is methodologically notable: AKE alone yields $T$9, below the supervised fine-tuning baseline of $X$0 on Qwen2.5-VL-3B, whereas RIKE as a whole achieves $X$1 (Wang et al., 23 May 2026). This indicates that knowledge enrichment, in isolation, is not sufficient; its value emerges when coupled to the comparative reasoning mechanism.

4. Relative Intent Reasoning

The Relative Intent Reasoning module is the component that explicitly compares the two stance-conditioned explanations provided for each meme. Let $X$2 denote the human-written non-harmful explanation and $X$3 the human-written harmful explanation. These are re-encoded relative to the fused multimodal representation:

$X$4

Their difference,

$X$5

is interpreted as a relative-intent logit: positive $X$6 indicates that the harmful reading is more strongly supported (Wang et al., 23 May 2026).

Training uses binary cross-entropy on $X$7:

$X$8

Conditioned on the selected stance, the framework also generates a unified explanation with loss

$X$9

An optional auxiliary retrieval-reconstruction loss $I\in\mathbb{R}^{H\times W\times 3}$0 can be added, giving the full objective

$I\in\mathbb{R}^{H\times W\times 3}$1

Optimization is then defined by minimizing the expected total loss over memes, labels, and paired explanations (Wang et al., 23 May 2026).

The RIR ablation provides the most direct evidence for the framework’s central hypothesis. On Qwen2.5-VL-3B, adding RIR alone raises $I\in\mathbb{R}^{H\times W\times 3}$2 from approximately $I\in\mathbb{R}^{H\times W\times 3}$3 to approximately $I\in\mathbb{R}^{H\times W\times 3}$4, while RIKE with both RIR and AKE reaches approximately $I\in\mathbb{R}^{H\times W\times 3}$5 (Wang et al., 23 May 2026). This suggests that explicit relative reasoning over opposed interpretations is the dominant contributor to classification performance, with cultural knowledge serving as a complementary support mechanism.

5. Training configuration and empirical performance

The reported implementation fine-tunes Qwen2.5-VL (3B/7B) and other MLLMs using LoRA with rank $I\in\mathbb{R}^{H\times W\times 3}$6 and $I\in\mathbb{R}^{H\times W\times 3}$7, a learning rate of $I\in\mathbb{R}^{H\times W\times 3}$8, batch size $I\in\mathbb{R}^{H\times W\times 3}$9, and 5 training epochs (Wang et al., 23 May 2026). Evaluation covers both classification and explanation quality. The key metrics are Accuracy, Precision/Recall/F1 on the harmful class, BLEU-4 and ROUGE-L for generated explanations, and Qwen-VL-Plus judgments on Informativeness, Soundness, Cultural-Relevance, Conciseness, and Persuasiveness using a 5-point Likert scale (Wang et al., 23 May 2026).

The main quantitative findings for Qwen2.5-VL-3B are summarized below.

Setting	Reported result
Baseline supervised fine-tuning (SFT)	$T$0
+RIR only	$T$1
+AKE only	$T$2
RIKE (RIR+AKE)	$T$3

For explanation generation on the same backbone, RIKE raises BLEU-4 from 14.17 to 51.69 and ROUGE-L from 29.31 to 62.51 (Wang et al., 23 May 2026). Similar gains are reported across InternVL3_5-8B, LLaVA-1.5-7B, and closed-source models. Human-judge scores on explanation dimensions increase by 0.1–0.3 across the board, which the paper interprets as evidence that the framework improves both classification robustness and culture-grounded attribution (Wang et al., 23 May 2026).

These results support a specific architectural reading. RIKE does not simply improve label prediction; it appears to restructure the task so that explanation quality and classification quality co-improve. Because the explanation loss is conditioned on the paired stance representations and retrieved knowledge, the framework links attribution correctness to explanation faithfulness more tightly than ordinary post hoc rationale generation would.

6. Interpretation, scope, and relation to broader research

RIKE belongs to a strand of multimodal harm detection research that emphasizes context sensitivity, explanation, and cultural specificity rather than purely generic toxicity classification (Wang et al., 23 May 2026). Within that framing, its principal novelty lies in converting ambiguity from an annotation nuisance into a supervised reasoning signal. Instead of collapsing disagreement into a single label, the framework models harmfulness through the contrast between a harmful and a non-harmful interpretation.

This design also clarifies a common misconception in harmful meme analysis: ambiguity does not imply annotation noise in the trivial sense. In RIKE’s formulation, ambiguity is task structure. The presence of two plausible readings is precisely what necessitates relative-intent modeling. Likewise, the use of C-HarmKB indicates that cultural knowledge is not treated as optional enrichment but as a required component for resolving historically or socially indexed references.

Another misconception would be to treat AKE as the primary source of performance gains simply because it introduces an external knowledge base. The ablation results do not support that reading. AKE alone underperforms the baseline, whereas RIR alone contributes the major increase in $T$4, and the full RIKE configuration yields the best overall outcome (Wang et al., 23 May 2026). This suggests that retrieved knowledge must be integrated into a comparative attribution process to become useful.

More broadly, RIKE can be understood as an interpretable multimodal reasoning architecture for culturally grounded harm attribution in Chinese. The paper’s results suggest that explanation supervision, stance contrast, and knowledge retrieval are mutually reinforcing when the target phenomenon is subjective, context dependent, and semantically underdetermined (Wang et al., 23 May 2026). A plausible implication is that analogous frameworks may be relevant wherever harmfulness hinges on community-specific references, irony, or contested intent, although such extrapolation would require direct empirical validation beyond the reported Chinese harmful meme setting.