Tri-stream Cross-Reasoning Network (TCRNet)

• The paper introduces TCRNet, a fusion module that integrates OCR text, visual content, and structured reasoning to capture implicit cultural cues in dark humor memes.
• TCRNet employs pairwise cross-attention among text, image, and reasoning streams, yielding a unified 2304-dimensional representation for multimodal classification.
• Empirical results demonstrate that incorporating the reasoning stream significantly outperforms text-only and image-only methods in dark humor detection and target identification.

Tri-stream Cross-Reasoning Network (TCRNet) is the multimodal fusion module introduced in “D-HUMOR: Dark Humor Understanding via Multimodal Open-ended Reasoning” for dark humor understanding in memes (Kasu et al., 8 Sep 2025). Within the D-HUMOR pipeline, a meme is represented through three coordinated streams—OCR text, image content, and a structured explanation generated by a Vision–LLM with an iterative Role-Reversal Self-Loop—and TCRNet fuses these streams through pairwise cross-attention to produce a unified representation for downstream classification. Its design is motivated by the observation that dark humor depends on implicit, sensitive, and culturally contextual cues that are not fully recoverable from raw text or visual input alone (Kasu et al., 8 Sep 2025).

1. Architectural role in the D-HUMOR pipeline

In the D-HUMOR framework, each meme consists of an image $I$ with embedded text. The embedded text is converted into an OCR transcript $T$, while a six-part structured explanation $R$ is generated by a Vision–LLM, specifically Qwen-2.5-7B, together with an iterative Role-Reversal Self-Loop (Kasu et al., 8 Sep 2025). TCRNet operates on top of these three streams and serves as the central fusion mechanism.

The three streams have distinct semantic roles. OCR text captures the literal words in the meme, often including the punchline. Vision features capture visual context such as objects, facial expressions, and backgrounds. Structured reasoning captures the high-level explanation of the meme, including the implied joke, narrative style, emotional effect, taboo cues, and target. TCRNet is designed to align these components so that “what is said,” “what is seen,” and “why it is funny (or offensive)” are jointly modeled (Kasu et al., 8 Sep 2025).

This organization places TCRNet in a reasoning-augmented multimodal setting rather than a standard text-image fusion setting. A plausible implication is that the architecture is intended not merely to aggregate modalities, but to incorporate model-generated interpretive structure as a first-class modality.

2. Stream formulation and feature extraction

The model defines three inputs:

• $T \in \mathrm{Vocab}^\ast$: OCR transcript tokens, with subword-tokenized length $L \approx 197$
• $R \in \mathrm{Text}^\ast$: concatenated six-field explanation, similarly tokenized to length $\approx 197$
• $I \in \mathbb{R}^{H \times W \times 3}$: raw RGB image, with $H=W=224$ for ViT

Each stream is mapped into a sequence of embeddings of length $N=197$ and dimensionality $T$0 (Kasu et al., 8 Sep 2025). The paper specifies the following feature extractors:

$T$1

$T$2

$T$3

Here, each row of $T$4 corresponds to one token embedding for BERT or S-BERT, or one patch embedding for ViT. The architectural regularity of using the same sequence length and embedding dimensionality across all three streams simplifies later cross-attention operations.

Stream	Extractor	Output shape
OCR text $T$5	BERT	$T$6
Reasoning $T$7	S-BERT	$T$8
Vision $T$9	ViT	$R$0

This common embedding geometry is central to TCRNet’s pairwise fusion scheme. It suggests a deliberate effort to make cross-modal interaction structurally symmetric even when the source modalities are semantically heterogeneous.

3. Pairwise cross-reasoning mechanism

TCRNet performs multimodal fusion through one-way cross-attention between selected modality pairs. For modalities $R$1, the model defines learnable projections

$R$2

with $R$3 attention heads and $R$4, often $R$5 per head (Kasu et al., 8 Sep 2025). The projected query, key, and value tensors are

$R$6

with

$R$7

The attention operator is the scaled dot-product form

$R$8

TCRNet computes exactly three attention flows (Kasu et al., 8 Sep 2025):

$R$9

These have specific interpretive functions. $T \in \mathrm{Vocab}^\ast$0 aligns text tokens to visual patches; $T \in \mathrm{Vocab}^\ast$1 aligns text to reasoning concepts; and $T \in \mathrm{Vocab}^\ast$2 aligns visual patches to reasoning. Notably, the formulation is pairwise rather than fully all-to-all. This means that TCRNet encodes selective cross-modal dependencies rather than an unrestricted fusion graph.

A common misconception would be to interpret the reasoning stream as a late auxiliary feature appended after text-image fusion. The specified equations show that reasoning participates directly in attention-based alignment, both from text to reasoning and from image to reasoning, making it part of the core interaction mechanism rather than an external add-on.

4. Unified representation, heads, and optimization objective

Each cross-attended tensor $T \in \mathrm{Vocab}^\ast$3 is reduced to a single vector by average pooling across the sequence dimension:

$T \in \mathrm{Vocab}^\ast$4

The resulting pooled vectors are concatenated:

$T \in \mathrm{Vocab}^\ast$5

This $T \in \mathrm{Vocab}^\ast$6-dimensional vector is the multimodal, reasoning-informed embedding delivered to the classifier (Kasu et al., 8 Sep 2025).

The classification head consists of dropout with $T \in \mathrm{Vocab}^\ast$7, a single LayerNorm marked as optional, and three task-specific linear layers:

• Dark humor, binary:

$T \in \mathrm{Vocab}^\ast$8

• Target identification, $T \in \mathrm{Vocab}^\ast$9 classes:

$L \approx 197$0

• Intensity, $L \approx 197$1 levels:

$L \approx 197$2

Training uses the sum of three cross-entropy losses:

$L \approx 197$3

For dark humor classification, the paper gives the example

$L \approx 197$4

with analogous terms for the other tasks (Kasu et al., 8 Sep 2025).

This multi-task setup ties TCRNet to three prediction problems simultaneously: dark humor detection, target identification, and intensity prediction. A plausible implication is that the shared representation $L \approx 197$5 is encouraged to encode not only whether content is darkly humorous, but also whom it targets and with what severity.

5. Training regime and implementation parameters

The reported training configuration for TCRNet uses AdamW with weight decay $L \approx 197$6, a learning rate of $L \approx 197$7, batch size $L \approx 197$8, and $L \approx 197$9 epochs (Kasu et al., 8 Sep 2025). Dropout within TCRNet is $R \in \mathrm{Text}^\ast$0, and the number of attention heads is $R \in \mathrm{Text}^\ast$1.

No additional data augmentation is used beyond standard image resizing or cropping to $R \in \mathrm{Text}^\ast$2 and standard BERT tokenization. The reasoning LLM is fine-tuned separately via QLoRA before feature extraction, with $R \in \mathrm{Text}^\ast$3 epochs, rank $R \in \mathrm{Text}^\ast$4, $R \in \mathrm{Text}^\ast$5, and LoRA-dropout $R \in \mathrm{Text}^\ast$6 (Kasu et al., 8 Sep 2025).

Component	Setting
Optimizer	AdamW, weight decay $R \in \mathrm{Text}^\ast$7
Learning rate	$R \in \mathrm{Text}^\ast$8
Batch size	$R \in \mathrm{Text}^\ast$9
Epochs	$\approx 197$0
Attention heads	$\approx 197$1
TCRNet dropout	$\approx 197$2

These parameters indicate a conventional fine-tuning regime for the fusion model, while keeping the reasoning generator on a separate QLoRA adaptation path. This separation suggests a modular pipeline in which explanation generation and multimodal classification are coupled functionally but trained in different stages.

6. Empirical performance and interpretive significance

According to Table 3, TCRNet achieves the following results on the D-HUMOR tasks (Kasu et al., 8 Sep 2025):

• Dark humor detection accuracy: $\approx 197$3
• Dark humor detection Weighted-F1: $\approx 197$4
• Target identification accuracy: $\approx 197$5
• Target identification Macro-F1: $\approx 197$6
• Intensity prediction accuracy: $\approx 197$7
• Intensity prediction Pearson corr: $\approx 197$8

The paper states that this performance outperforms all text-only, image-only, zero-shot VLMs, and OCR+explanation baselines. The ablation analysis in Fig. 6 is especially important for understanding TCRNet’s role. Removing the reasoning stream, that is, deleting $\approx 197$9 and its cross-attentions, reduces target Macro-F1 from $I \in \mathbb{R}^{H \times W \times 3}$0 to $I \in \mathbb{R}^{H \times W \times 3}$1, and dark humor Weighted-F1 from $I \in \mathbb{R}^{H \times W \times 3}$2 to $I \in \mathbb{R}^{H \times W \times 3}$3 (Kasu et al., 8 Sep 2025).

These ablation results support the claim that structured reasoning and its cross-modal alignment are critical for capturing implicit and culturally grounded cues in dark humor memes. They also clarify that the gains are not attributable solely to having three encoders; rather, the reported evidence points to the specific value of the reasoning stream within the pairwise attention design.

More broadly, TCRNet can be situated as a reasoning-aware multimodal classifier whose distinctive feature is the explicit integration of generated explanation structure into the fusion process. Within the terms used by the paper, text disambiguates visual incongruities, vision grounds abstract reasoning, and reasoning supplies the high-level intent that neither raw text nor images can convey alone (Kasu et al., 8 Sep 2025). This suggests that TCRNet is best understood not as a generic multimodal architecture, but as a targeted mechanism for domains in which latent social meaning, offense, or taboo interpretation must be inferred from jointly processed textual, visual, and explanatory signals.