BiMind: A Dual-Head Reasoning Model with Attention-Geometry Adapter for Incorrect Information Detection

Abstract: Incorrect information poses significant challenges by disrupting content veracity and integrity, yet most detection approaches struggle to jointly balance textual content verification with external knowledge modification under collapsed attention geometries. To address this issue, we propose a dual-head reasoning framework, BiMind, which disentangles content-internal reasoning from knowledge-augmented reasoning. In BiMind, we introduce three core innovations: (i) an attention geometry adapter that reshapes attention logits via token-conditioned offsets and mitigates attention collapse; (ii) a self-retrieval knowledge mechanism, which constructs an in-domain semantic memory through kNN retrieval and injects retrieved neighbors via feature-wise linear modulation; (iii) the uncertainty-aware fusion strategies, including entropy-gated fusion and a trainable agreement head, stabilized by a symmetric Kullback-Leibler agreement regularizer. To quantify the knowledge contributions, we define a novel metric, Value-of-eXperience (VoX), to measure instance-wise logit gains from knowledge-augmented reasoning. Experiment results on public datasets demonstrate that our BiMind model outperforms advanced detection approaches and provides interpretable diagnostics on when and why knowledge matters.

• The paper introduces a dual-head reasoning model (BiMind) that structurally separates content and knowledge reasoning to mitigate attention collapse.
• It employs an Attention Geometry Adapter and uncertainty-aware fusion to enhance token-level specialization and enable robust, interpretable knowledge integration.
• Extensive experiments demonstrate significant accuracy and efficiency improvements over baselines, validating its effectiveness in detecting incorrect information.

BiMind: Dual-Head Reasoning and Attention Geometry Adaptation for Knowledge-Sensitive Incorrect Information Detection

Introduction and Motivation

The propagation of incorrect information in social media presents substantial challenges for automatic detection methods, primarily due to ambiguous textual cues, frequent occurrence of domain shifts, and the varying relevance or integrity of external knowledge. Conventional models usually entangle content-derived features with knowledge-augmented evidence, leading to hard-to-diagnose failures and frequent attention collapse in deep Transformer architectures. The paper "BiMind: A Dual-Head Reasoning Model with Attention-Geometry Adapter for Incorrect Information Detection" (2604.06022) directly addresses this shortcoming with a framework that structurally separates content-internal reasoning from knowledge-augmented reasoning, introducing mechanisms for quantifying, fusing, and visualizing the impact of external knowledge at both the instance and dataset level.

The research introduces three principal innovations:

• An Attention Geometry Adapter (AGA) explicitly re-parameterizes pre-softmax attention distributions using interpretable POS-driven token-conditioned offsets, systematically counteracting representational collapse while preserving low-rank, structured attention specialization.
• A self-retrieval knowledge module, constructing in-domain semantic memory via SentenceTransformer-based kNN retrieval, and modulating encoder representations with FiLM, ensures knowledge integration is both contextual and locally explainable.
• Uncertainty-aware fusion leveraged by entropy-gated weighting and a trainable agreement head; this not only stabilizes predictions under knowledge ambiguity but concretely quantifies knowledge contribution with the Value-of-eXperience (VoX) metric.

Dual-Head BiMind Architecture

The BiMind framework consists of a dual-head Transformer, as depicted in Figure 1. The content-internal head encodes raw textual features, while a parallel knowledge-augmented head injects retrieved neighbor representations via FiLM, which are adapted by the AGA-modified encoder stack.

Immediately after this architectural bifurcation, the knowledge path retrieves top-$k$ semantically nearest neighbors from an in-domain candidate set (embedding normalization and cosine similarity are used for retrieval), projects the aggregate neighbor into modulation vectors, and applies FiLM to the main content representation. This design enforces that knowledge augmentation does not contaminate content-only reasoning channels and that knowledge’s influence can be dissected both on a per-instance and statistical basis.

Figure 1: BiMind's dual-head architecture, explicit AGA attention shaping, knowledge self-retrieval with FiLM, and adaptive fusion with VoX quantification.

The attention collapse phenomenon in deeper Transformer layers is addressed by the AGA module, which increases entropy, thereby dispersing focus and facilitating robust token-level salience discrimination across heads. The result is a set of heads with structured selectivity—essential for interpretable knowledge integration. The effect of AGA is illustrated in Figure 2.

Figure 2: Attention maps with and without AGA. AGA enforces structured, low-rank, and key-focused attention in deep layers compared to baseline flattening.

Uncertainty-aware Fusion and Knowledge Diagnostic Tools

The uncertainty-aware fusion combines the predictions of the two heads, dynamically weighting them according to the entropy of their individual predictive distributions. If the content-internal head expresses uncertainty (high entropy), the fusion gate shifts toward the knowledge-augmented head, and vice versa. An alternative is a trainable agreement head that captures the interaction and discrepancy between head representations, regularized by a symmetric KL loss to promote consistent predictions while permitting divergence where knowledge is genuinely informative.

Crucially, BiMind operationalizes the quantification of knowledge value via the Value-of-eXperience (VoX) metric, defined as the logit difference on the gold label between the two heads. Positive VoX indicates effective knowledge augmentation; negative VoX signals deleterious or noisy knowledge injection; near-zero VoX marks neutral or redundant knowledge.

Figure 3: Instance-level VoX analysis. Positive knowledge gain (green), knowledge harm (orange), and baseline vs. augmentation trajectories.

The ability to interpret causality in prediction shifts, and to localize knowledge intervention to the reasoning path, substantially increases the transparency and diagnostic power of the system—an aspect rarely present in prior unified models.

Experimental Validation and Analysis

Experiments span four datasets: MM COVID (health misinformation), RoCOVery (COVID-19 news), LIAR (fact-checking), and MC Fake (multi-domain). BiMind is benchmarked against a breadth of content-only, graph-based, and hybrid models, including CNN, GCN, BERT, HAN, and HeteroSGT.

BiMind consistently achieves the highest or second-highest accuracy, precision, recall, and F1 (e.g., 0.951 acc/F1 on MM COVID; 0.918 acc/F1 on ReCOVery), outperforming all baselines with significant margins (e.g., $+$18-27\% F1 over CNN/GCN and $+$3-5\% improvement over HeteroSGT on key datasets). Fusion strategies offer robust performance across domains, dynamically adjusting head predominance based on the data.

AGA and knowledge retrieval are shown through ablations to be critical: removing AGA or the knowledge retrieval function causes accuracy and F1 drops up to 5%. Uncertainty-aware fusion increases performance by 3-4% F1 in adversarial or low-alignment regimes. The efficiency comparison demonstrates that BiMind's non-graph-dependent architecture reduces training and inference time by up to $4\times$ relative to HeteroSGT, demonstrating clear scalability benefits.

Instance-level case studies (see Figure 4) demonstrate that knowledge-augmented reasoning can autocorrect content-induced errors, providing direct evidence for the efficacy and necessity of structured dual-head disentanglement.

Figure 4: Self-correction via knowledge; knowledge-augmented head revises a misclassification made by the content-internal head.

Attention Specialization and Adaptive Knowledge Utility

Analysis of layer-wise attention heads demonstrates that AGA enables select heads to differentiate and focus on grammatically or semantically critical token classes, counteracting depth-induced attention homogeneity.

Figure 5: Layer 9 head specialization—AGA preserves specialized, category-focused attention patterns.

VoX analysis (Figure 3) and dataset-level statistics show that knowledge utility is modulated both by the retrieval quality (cosine similarity) and linguistic complexity (Flesch Reading Ease) of the retrieved corpus. This suggests that adaptive fusion and per-instance knowledge integration, as in BiMind, are necessary for generalization across short/noisy and long/complex input domains.

Implications, Theoretical and Practical

From a theoretical perspective, BiMind provides a concrete instantiation of disentangled feature attribution for knowledge-augmented detection tasks. It operationalizes uncertainty quantification and agreement regularization in a predictive context, linking instance-level behavior to aggregate performance.

Practically, BiMind’s architecture—explicit dual-head reasoning, attention geometry shaping, and in-domain semantic retrieval—offers a domain-agnostic template for high-stakes misinformation detection where both interpretability and accuracy are required. The selective trust of knowledge, tunable fusion, and detailed diagnostics facilitate deployment in dynamic, real-world environments.

As LLMs and multimodal neural architectures increasingly underpin social content curation, frameworks like BiMind point toward hybrid, modular systems that are both robust to feature collapse and transparent in their knowledge utilization mechanisms. Extending BiMind to multimodal or multilingual misinformation regimes and integrating credibility modeling of external knowledge sources are logical next steps.

Conclusion

The BiMind architecture advances the state of incorrect information detection through structured separation of reasoning streams, principled attention geometry adaptation, and instance-level quantification of knowledge benefit. Empirical results validate the necessity of each module for both accuracy and interpretability. BiMind exemplifies a generalizable, efficient, and diagnostic approach for dynamic misinformation environments, setting a technical foundation for subsequent developments in knowledge-reasoning architectures.