Knowledge Overshadowing in LLMs

• Knowledge Overshadowing is a phenomenon where dominant data patterns suppress rarer, complementary knowledge in LLMs, leading to systematic errors.
• It arises from factors like data imbalance, shared contextual prefixes, and model architecture, impacting multi-modal fusion and retrieval systems.
• Mitigation techniques such as contrastive decoding, circuit pruning, and adaptive fusion are used to rebalance knowledge integration and reduce hallucinations.

Knowledge overshadowing is a phenomenon in machine learning and, specifically, LLMs, where certain knowledge components—frequently those that are more prevalent, semantically dominant, or externally introduced—overpower or suppress alternative, potentially correct or complementary knowledge sources during representation, inference, or generation. This suppression leads to systematic errors such as amalgamated hallucinations, misattribution, loss of diversity in generated content, or inattention to critical sub-conditions in queries. Knowledge overshadowing is observed across a spectrum of modalities and frameworks, including multi-modal fusion, retrieval-augmented generation (RAG), knowledge graph reasoning, and generative recommendation. Its emergence is closely tied to data imbalance, prompt structure, model size, fusion strategies, and competition between parametric and contextual knowledge.

1. Formal Definitions and Typology

Knowledge overshadowing manifests whenever a dominant knowledge source, pattern, or modality disproportionately dictates model behavior to the detriment of less prominent conditions or signals. In LLMs, knowledge overshadowing can be formally characterized as follows:

• Condition-level: Given conditions $A$ and $B$ in a prompt, overshadowing occurs if $p(\mathbf{y}|A,B) \approx p(\mathbf{y}|A)$, indicating that $B$ is effectively ignored (Zhang et al., 2024, Zhang et al., 22 Feb 2025).
• Fact-level (Hallucination): Given co-occurring facts $KA$ (popular) and $KB$ (rare), overshadowing is observed when prompts resembling $KB$ produce completions predicted from $KA$, rather than respecting their unique suffixes or attributes (Zhang et al., 22 Feb 2025).
• Modality-level: In multi-modal fusion (e.g., concatenation of semantic and collaborative embeddings), the modality with higher representational capacity or signal strength dominates the fused representation, suppressing contributions from the weaker modality (Xiao et al., 10 Feb 2025).
• Source-level (Parametric vs. Contextual): When models must reconcile internal (parametric) knowledge $M$ and retrieved/contextual knowledge $C$, overshadowing is observed whenever the response aligns strictly with either the dominant context $C$ or the entrenched parametric memory $M$, depending on their relative plausibility or prompt influence (Sun et al., 6 Jun 2025, Guo et al., 18 Mar 2025, Zhuo et al., 15 Oct 2025).

Overshadowing-induced amalgamated hallucinations occur when outputs conflate, blend, or misattribute facts from the overshadowing knowledge, even under clean, error-free training corpora (Zhang et al., 2024, Huang et al., 20 May 2025).

2. Mechanisms and Theoretical Drivers

Knowledge overshadowing arises mainly from the interplay between data-driven frequency effects, model architecture, optimization trajectory, and fusion strategy:

• Data Imbalance (Popularity): The ratio $\gamma = M/N$ of dominant to suppressed condition examples directly raises the overshadowing rate. Empirically, as $\gamma$ grows (e.g., from 10:1 to 100:1), hallucination rates for the rare condition increase from 45.7% up to 70.5% on Llama-2-7B (Zhang et al., 2024).
• Shared Context and Prefix Length: The capacity of the dominant pattern to overshadow grows not only with popularity, but also with the descriptive length of the shared prefix or condition (Zhang et al., 22 Feb 2025, Zhang et al., 2024). Longer shared contexts amplify overshadowing, yielding higher error bounds.
• Model Size: Larger models (higher parameter count $S$) more efficiently compress frequent facts, magnifying the memorization of dominant patterns and suppression of rare facts (Zhang et al., 22 Feb 2025, Huang et al., 20 May 2025).
• Training Dynamics: During early training epochs, loss gradients from the dominant pattern outnumber and outscale those from rarer knowledge. Subordinate knowledge is neglected until its associated loss rises, at which point the optimizer gradually recovers balanced attention (Huang et al., 20 May 2025).
• Fusion and Alignment Schemes: Naive concatenation of representations in multimodal systems (e.g., semantic plus collaborative embeddings) leads to semantic domination, wherein the semantic space envelops the joint code, with the collaborative signal nearly erased (e.g., 97.33% similarity with the semantic modality vs. 2.67% with collaborative under simple concat) (Xiao et al., 10 Feb 2025).

Generalization theory formalizes overshadowing risk via Rademacher-complexity-based bounds, demonstrating that hallucination probability increases with data skew and shared feature length (Zhang et al., 2024, Zhang et al., 22 Feb 2025).

3. Empirical Manifestations

Knowledge overshadowing has been demonstrated in a variety of technical contexts:

• Amalgamated Hallucination: When prompted for “famous female AI scientists,” LLMs enumerate well-known male names due to the statistical prevalence of male AI scientist examples (Zhang et al., 2024).
• LLM-RAG Systems: GraphRAG’s unfiltered retrieval can cause the system to prefer incorrect graph-derived answers over correct parametric responses, especially when the retrieved knowledge is noisy or irrelevant. On WebQSP and CWQ, 16.89% of samples that LLM alone answered correctly were mispredicted due to overshadowing after RAG augmentation (Guo et al., 18 Mar 2025).
• Multimodal Fusion in Recommendation: Simple fusion of semantic and collaborative representations yields a fused embedding that is almost entirely determined by the semantic modality, degrading recall and NDCG performance compared to semantic-only representations (Xiao et al., 10 Feb 2025).
• Knowledge Graph Reasoning: Sparse KG contexts can excessively distort the LLM’s posterior, moving it away from priors encoded in pretraining, leading to illogical inferences (e.g., mislabeling a music school as a film genre) (Zhuo et al., 15 Oct 2025).
• Multi-hop Reasoning in RAG: In multi-hop QA, intermediate queries tend to neglect (“overshadow”) critical keyphrases from the user’s original multi-condition query, propagating error across reasoning steps (Ma et al., 12 Jan 2026).
• Attention Circuits: Mechanistically, during overshadowing, attention heads focus nearly exclusively on dominant knowledge pathways, suppressing subordinate traces. Only after sufficient loss gradient from subordinate cases do specific heads “open” subordinate circuits, restoring correct output probabilities (Huang et al., 20 May 2025).

4. Quantitative Laws and Predictive Frameworks

A key theoretical outcome is the derivation and empirical confirmation of an explicit log-linear law for overshadowing-induced hallucination rates (Zhang et al., 22 Feb 2025): $$H \equiv R = \alpha + \beta_1 \cdot \log P + \beta_2 \cdot \log L + \beta_3 \cdot \log S$$ where $P$ is knowledge popularity (dominant-to-rare frequency ratio), $L$ is relative knowledge length (token share of rare fact), and $S$ is model size. Fits across Pythia, LLaMA-2, GPT-J, and Phi series yield $R^2 \approx 0.95$, with only ~8% relative error when predicting hallucination rates on downstream tasks.

Other metrics include:

• Hallucination Rate (HR): $$P[\text{model predicts } Y_a | X_{\text{share}} \circ x_b]$$ (Zhang et al., 22 Feb 2025, Zhang et al., 2024).
• Relative Overshadowing (RO): $$\text{AO} / \text{Dominant Recall} = p(Y_{\text{dom}} | P_{\text{sub}}) / p(Y_{\text{dom}} | P_{\text{dom}})$$ (Huang et al., 20 May 2025).
• Quantification under RAG: Category C (LLM-only correct, GraphRAG wrong) directly measures overshadowing instances (Guo et al., 18 Mar 2025).
• Detection via Perturbation: In multi-hop QA, cosine similarity between pooled model outputs after perturbing candidate keyphrases provides a quantitative overshadowing criterion (Ma et al., 12 Jan 2026).

5. Mitigation Techniques and Architectural Solutions

Addressing knowledge overshadowing requires interventions at data, training, architectural, and decoding levels:

• Contrastive Decoding (CoDA / SCD): Adjusting generation logits at inference by penalizing outputs that depend more on dominant context than the overshadowed condition, boosting tokens that only appear when the rare condition is unmasked. CoDA yields up to 27.9% EM improvement on synthetic and real tasks (Zhang et al., 22 Feb 2025, Zhang et al., 2024).
• Circuit Analysis and Pruning: PhantomCircuit constructs subgraphs of model computation responsible for distinguishing subordinate facts and prunes out connections correlated with overshadowing, recovering correct outputs at up to 90% probability (Huang et al., 20 May 2025).
• Multi-stage Filtering and Integration: In GraphRAG-FI, a two-stage filter prunes distractor knowledge both by attention and LLM-based relevance before blending LLM and retrieval-based answers through confidence-weighted fusion, reducing error due to over-reliance on external knowledge (Guo et al., 18 Mar 2025).
• Fusion Architecture Adaptation: In generative recommendation, cross-modality knowledge alignment (via InfoNCE and adaptive layer normalization) re-balances modalities before fusion, followed by in-modality knowledge distillation to recover complementary information lost in discretization (Xiao et al., 10 Feb 2025).
• Unified Tokenization and Attention: In KG reasoning, KRLM aligns LLM priors and KG context via joint tokenization, custom attention that dynamically blends both sources, and a domain-constrained next-entity predictor (Zhuo et al., 15 Oct 2025).
• Perturbation-based Overshadowing Detection: ActiShade detects under-weighted keyphrases in multi-hop RAG pipelines using sensitivity to Gaussian noise, then guides reasoning to supplement these overlooked facts in subsequent retrieval and query steps (Ma et al., 12 Jan 2026).

Data curation (balancing co-occurrence frequencies), prompt engineering (increasing the contextual weight of rare conditions), and curriculum training (emphasizing less frequent patterns early) are general preventative recommendations (Zhang et al., 2024, Zhang et al., 22 Feb 2025, Huang et al., 20 May 2025).

6. Broader Implications, Limitations, and Open Research Questions

Knowledge overshadowing exposes fundamental limitations in LLMs and related architectures that persist despite high-quality, factual training data or sophisticated retrieval augmentation:

• Resilience to Prompt Instructions: LLMs cannot reliably suppress parametric knowledge even when instructed to ignore it; metacognitive prompt design and rationale generation offer only partial compensation (Sun et al., 6 Jun 2025).
• Generalization vs. Selectivity Trade-off: Overshadowing reflects a broader tension: gradient-based optimization favors compressible, popular patterns, but this overgeneralization undermines model selectivity and conditional faithfulness (Zhang et al., 22 Feb 2025, Huang et al., 20 May 2025).
• Evaluation Pitfalls: Model-based evaluation is inherently biased in the presence of knowledge conflict; human or mixed evaluators are necessary for conflict-laden tasks (Sun et al., 6 Jun 2025).
• Limits of Black-box Mitigation: Most mitigation strategies (self-contrastive decoding, filter-fusion recipes) operate as post-hoc corrections, yet architectural or data-level approaches are likely necessary for robust prevention across tasks (Zhang et al., 2024, Zhang et al., 22 Feb 2025).
• Open Directions: The interaction of overshadowing with instruction-tuning, multimodal/fused representations, continual and lifelong learning, and cross-lingual knowledge retention remains largely unexplored (Guo et al., 18 Mar 2025, Zhuo et al., 15 Oct 2025, Ma et al., 12 Jan 2026).

In sum, knowledge overshadowing unifies a diverse set of failure modes in generative AI, linking hallucination, loss of conditional faithfulness, modality imbalance, and retrieval misalignment to quantifiable, theoretically grounded causes tied to data structure, model capacity, and fusion strategy. Ongoing research leverages both interpretability and architectural innovation to mitigate its impact and forecast error rates, advancing the reliability of knowledge-intensive AI systems.