Resolving Internal Knowledge Conflicts in Pre-trained LLMs

Develop mechanisms to detect, manage, and resolve internal knowledge conflicts arising within large language models due to contradictory, time-varying, subjective, or uncertain information in web-scale pretraining corpora, enabling reliable selection and synthesis of answers.

Background

Web-scale corpora contain conflicting, outdated, subjective, and uncertain information, which leads to internal knowledge conflicts when models learn via probabilistic language modeling. The authors highlight the need for models to select, synthesize, and handle such conflicts—capabilities that are natural for humans but challenging for current LLMs without structured preprocessing or additional mechanisms.

References

This blog post highlight three critical open problems limiting model capabilities: (1) challenges in knowledge updating for LLMs, (2) the failure of reverse knowledge generalization (the reversal curse), and (3) conflicts in internal knowledge.

— Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigms  (2504.06823 - Ye et al., 9 Apr 2025) in Abstract

We leave to future work trajectory-level analysis of where conflicts are dropped and which harness affordances and scoring choices would preserve and reward uncertainty acknowledgment.

— Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict  (2610.12360 - Sun et al., 8 Oct 2026) in Section 4, subsection “Prompting improves EH but does not consistently improve accuracy”