---
title: Knowledge Overshadowing in LLMs
url: https://www.emergentmind.com/topics/knowledge-overshadowing
type: topic
---

# Knowledge Overshadowing in LLMs

Knowledge overshadowing is a phenomenon in machine learning and, specifically, large language models (LLMs), where certain knowledge components—frequently those that are more prevalent, semantically dominant, or externally introduced—overpower or suppress alternative, potentially correct or complementary knowledge sources during representation, inference, or generation. This suppression leads to systematic errors such as amalgamated hallucinations, misattribution, loss of diversity in generated content, or inattention to critical sub-conditions in queries. Knowledge overshadowing is observed across a spectrum of modalities and frameworks, including multi-modal fusion, retrieval-augmented generation (RAG), knowledge graph reasoning, and generative recommendation. Its emergence is closely tied to data imbalance, prompt structure, model size, fusion strategies, and competition between parametric and contextual knowledge.

## 1. Formal Definitions and Typology

Knowledge overshadowing manifests whenever a dominant knowledge source, pattern, or modality disproportionately dictates model behavior to the detriment of less prominent conditions or signals. In LLMs, knowledge overshadowing can be formally characterized as follows:

- **Condition-level:** Given conditions \(A\) and \(B\) in a prompt, overshadowing occurs if \(p(\mathbf{y}|A,B) \approx p(\mathbf{y}|A)\), indicating that \(B\) is effectively ignored [2407.08039][2502.16143].
- **Fact-level (Hallucination):** Given co-occurring facts \(KA\) (popular) and \(KB\) (rare), overshadowing is observed when prompts resembling \(KB\) produce completions predicted from \(KA\), rather than respecting their unique suffixes or attributes [2502.16143].
- **Modality-level:** In multi-modal fusion (e.g., concatenation of semantic and collaborative embeddings), the modality with higher representational capacity or signal strength dominates the fused representation, suppressing contributions from the weaker modality [2502.06269].
- **Source-level (Parametric vs. Contextual):** When models must reconcile internal (parametric) knowledge \(M\) and retrieved/contextual knowledge \(C\), overshadowing is observed whenever the response aligns strictly with either the dominant context \(C\) or the entrenched parametric memory \(M\), depending on their relative plausibility or prompt influence [2506.06485][2503.13804][2510.13909].

Overshadowing-induced amalgamated hallucinations occur when outputs conflate, blend, or misattribute facts from the overshadowing knowledge, even under clean, error-free training corpora [2407.08039][2505.14406].

## 2. Mechanisms and Theoretical Drivers

Knowledge overshadowing arises mainly from the interplay between data-driven frequency effects, model architecture, optimization trajectory, and fusion strategy:

- **Data Imbalance (Popularity):** The ratio \(\gamma = M/N\) of dominant to suppressed condition examples directly raises the overshadowing rate. Empirically, as \(\gamma\) grows (e.g., from 10:1 to 100:1), hallucination rates for the rare condition increase from 45.7% up to 70.5% on Llama-2-7B [2407.08039].
- **Shared Context and Prefix Length:** The capacity of the dominant pattern to overshadow grows not only with popularity, but also with the descriptive length of the shared prefix or condition [2502.16143][2407.08039]. Longer shared contexts amplify overshadowing, yielding higher error bounds.
- **Model Size:** Larger models (higher parameter count \(S\)) more efficiently compress frequent facts, magnifying the memorization of dominant patterns and suppression of rare facts [2502.16143][2505.14406].
- **Training Dynamics:** During early training epochs, loss gradients from the dominant pattern outnumber and outscale those from rarer knowledge. Subordinate knowledge is neglected until its associated loss rises, at which point the optimizer gradually recovers balanced attention [2505.14406].
- **Fusion and Alignment Schemes:** Naive concatenation of representations in multimodal systems (e.g., semantic plus collaborative embeddings) leads to semantic domination, wherein the semantic space envelops the joint code, with the collaborative signal nearly erased (e.g., 97.33% similarity with the semantic modality vs. 2.67% with collaborative under simple concat) [2502.06269].

Generalization theory formalizes overshadowing risk via Rademacher-complexity-based bounds, demonstrating that hallucination probability increases with data skew and shared feature length [2407.08039][2502.16143].

## 3. Empirical Manifestations

Knowledge overshadowing has been demonstrated in a variety of technical contexts:

- **Amalgamated Hallucination:** When prompted for “famous female AI scientists,” LLMs enumerate well-known male names due to the statistical prevalence of male AI scientist examples [2407.08039].
- **LLM-RAG Systems:** GraphRAG’s unfiltered retrieval can cause the system to prefer incorrect graph-derived answers over correct parametric responses, especially when the retrieved knowledge is noisy or irrelevant. On WebQSP and CWQ, 16.89% of samples that LLM alone answered correctly were mispredicted due to overshadowing after RAG augmentation [2503.13804].
- **Multimodal Fusion in Recommendation:** Simple fusion of semantic and collaborative representations yields a fused embedding that is almost entirely determined by the semantic modality, degrading recall and NDCG performance compared to semantic-only representations [2502.06269].
- **Knowledge Graph Reasoning:** Sparse KG contexts can excessively distort the LLM’s posterior, moving it away from priors encoded in pretraining, leading to illogical inferences (e.g., mislabeling a music school as a film genre) [2510.13909].
- **Multi-hop Reasoning in RAG:** In multi-hop QA, intermediate queries tend to neglect (“overshadow”) critical keyphrases from the user’s original multi-condition query, propagating error across reasoning steps [2601.07260].
- **Attention Circuits:** Mechanistically, during overshadowing, attention heads focus nearly exclusively on dominant knowledge pathways, suppressing subordinate traces. Only after sufficient loss gradient from subordinate cases do specific heads “open” subordinate circuits, restoring correct output probabilities [2505.14406].

## 4. Quantitative Laws and Predictive Frameworks

A key theoretical outcome is the derivation and empirical confirmation of an explicit log-linear law for overshadowing-induced hallucination rates [2502.16143]:
\[
H \equiv R = \alpha + \beta_1 \cdot \log P + \beta_2 \cdot \log L + \beta_3 \cdot \log S
\]
where \(P\) is knowledge popularity (dominant-to-rare frequency ratio), \(L\) is relative knowledge length (token share of rare fact), and \(S\) is model size. Fits across Pythia, LLaMA-2, GPT-J, and Phi series yield \(R^2 \approx 0.95\), with only ~8% relative error when predicting hallucination rates on downstream tasks.

Other metrics include:
- **Hallucination Rate (HR):** \(P[\text{model predicts } Y_a | X_{\text{share}} \circ x_b]\) [2502.16143][2407.08039].
- **Relative Overshadowing (RO):** \(\text{AO} / \text{Dominant Recall} = p(Y_{\text{dom}} | P_{\text{sub}}) / p(Y_{\text{dom}} | P_{\text{dom}})\) [2505.14406].
- **Quantification under RAG:** Category C (LLM-only correct, GraphRAG wrong) directly measures overshadowing instances [2503.13804].
- **Detection via Perturbation:** In multi-hop QA, cosine similarity between pooled model outputs after perturbing candidate keyphrases provides a quantitative overshadowing criterion [2601.07260].

## 5. Mitigation Techniques and Architectural Solutions

Addressing knowledge overshadowing requires interventions at data, training, architectural, and decoding levels:

- **Contrastive Decoding (CoDA / SCD):** Adjusting generation logits at inference by penalizing outputs that depend more on dominant context than the overshadowed condition, boosting tokens that only appear when the rare condition is unmasked. CoDA yields up to 27.9% EM improvement on synthetic and real tasks [2502.16143][2407.08039].
- **Circuit Analysis and Pruning:** PhantomCircuit constructs subgraphs of model computation responsible for distinguishing subordinate facts and prunes out connections correlated with overshadowing, recovering correct outputs at up to 90% probability [2505.14406].
- **Multi-stage Filtering and Integration:** In GraphRAG-FI, a two-stage filter prunes distractor knowledge both by attention and LLM-based relevance before blending LLM and retrieval-based answers through confidence-weighted fusion, reducing error due to over-reliance on external knowledge [2503.13804].
- **Fusion Architecture Adaptation:** In generative recommendation, cross-modality knowledge alignment (via InfoNCE and adaptive layer normalization) re-balances modalities before fusion, followed by in-modality knowledge distillation to recover complementary information lost in discretization [2502.06269].
- **Unified Tokenization and Attention:** In KG reasoning, KRLM aligns LLM priors and KG context via joint tokenization, custom attention that dynamically blends both sources, and a domain-constrained next-entity predictor [2510.13909].
- **Perturbation-based Overshadowing Detection:** ActiShade detects under-weighted keyphrases in multi-hop RAG pipelines using sensitivity to Gaussian noise, then guides reasoning to supplement these overlooked facts in subsequent retrieval and query steps [2601.07260].

Data curation (balancing co-occurrence frequencies), prompt engineering (increasing the contextual weight of rare conditions), and curriculum training (emphasizing less frequent patterns early) are general preventative recommendations [2407.08039][2502.16143][2505.14406].

## 6. Broader Implications, Limitations, and Open Research Questions

Knowledge overshadowing exposes fundamental limitations in LLMs and related architectures that persist despite high-quality, factual training data or sophisticated retrieval augmentation:

- **Resilience to Prompt Instructions:** LLMs cannot reliably suppress parametric knowledge even when instructed to ignore it; metacognitive prompt design and rationale generation offer only partial compensation [2506.06485].
- **Generalization vs. Selectivity Trade-off:** Overshadowing reflects a broader tension: gradient-based optimization favors compressible, popular patterns, but this overgeneralization undermines model selectivity and conditional faithfulness [2502.16143][2505.14406].
- **Evaluation Pitfalls:** Model-based evaluation is inherently biased in the presence of knowledge conflict; human or mixed evaluators are necessary for conflict-laden tasks [2506.06485].
- **Limits of Black-box Mitigation:** Most mitigation strategies (self-contrastive decoding, filter-fusion recipes) operate as post-hoc corrections, yet architectural or data-level approaches are likely necessary for robust prevention across tasks [2407.08039][2502.16143].
- **Open Directions:** The interaction of overshadowing with instruction-tuning, multimodal/fused representations, continual and lifelong learning, and cross-lingual knowledge retention remains largely unexplored [2503.13804][2510.13909][2601.07260].

In sum, knowledge overshadowing unifies a diverse set of failure modes in generative AI, linking hallucination, loss of conditional faithfulness, modality imbalance, and retrieval misalignment to quantifiable, theoretically grounded causes tied to data structure, model capacity, and fusion strategy. Ongoing research leverages both interpretability and architectural innovation to mitigate its impact and forecast error rates, advancing the reliability of knowledge-intensive AI systems.

Source: https://www.emergentmind.com/topics/knowledge-overshadowing