Relational Knowledge in Language Models

• Relational knowledge in LMs is defined by the encoding of subject–relation–object triples through learned distributed patterns in transformer attention and MLP layers.
• Key methodologies include unsupervised pretraining, causal mediation analysis, and targeted editing to reveal layerwise emergence and modular storage of relational facts.
• Advances in neuron-level and hybrid graph approaches enhance interpretability and control, supporting effective recall, precision, and dynamic generalization of relational information.

Relational knowledge in LLMs concerns the acquisition, encoding, recall, evaluation, and manipulation of factual or abstract associations—most commonly formalized as subject–relation–object triples—within large pre-trained neural LMs. Unlike knowledge bases, which represent relations explicitly and symbolically, LMs internalize these associations as distributed patterns in their learned weights and activations. The scientific paper of relational knowledge in LMs addresses questions of storage localization, recall dynamics, editability, representation geometry, task generalization, and limitations, drawing on a convergence of interpretability, knowledge engineering, and cognitive modeling methodologies.

1. Mechanisms for Encoding and Recall of Relational Knowledge

Relational knowledge ($k = (s, r, o)$) is acquired primarily through the unsupervised pretraining of transformer LLMs on large textual corpora. During forward inference on prompts of the form "subject + relation," such as “Marco Reus is a citizen of,” internal representations are formed sequentially over layers and tokens. At each layer $l$ and token position $i$, the hidden state is propagated as: $h_i^l = h_i^{l-1} + a_i^l + m_i^l$ where $a_i^l$ and $m_i^l$ are the multi-head self-attention and MLP sublayer outputs, respectively. The output token probability is: $$P(t_{N+1}|t_1,...,t_N) = \mathrm{softmax}(\phi(h_N^{L-1} + a_N^L + m_N^L))$$ Relational information is not homogeneously distributed. Critical mechanistic studies show that, in auto-regressive LMs, relation-specific attributes are predominantly accumulated at the last relation token in the prompt, and synthesized in the MLP (middle-to-late layers). This layerwise "relation emergence" is distinct from the processing of subject information, which dominate the earlier tokens.

Causal mediation analysis further uncovers a three-stage flow: initial (no relation/subject effect), relational emergence (pure relational effect at specific intermediate layers), and conjoint (subject and relation entanglement at deeper layers). By precisely identifying the intermediate layers and specific token positions mediating relational concepts, one can directly extract or manipulate relation representations for analysis or control (Wang et al., 19 Jun 2024, Liu et al., 27 Aug 2024).

2. Localization, Storage, and Editing: Subject vs. Relation Perspectives

Contrary to previous beliefs that knowledge is localized mainly in MLP weights, current evidence demonstrates that relational knowledge is partially and, for some relations, predominantly stored in attention modules (especially in upper transformer layers), whereas entity knowledge shows stronger concentration in MLPs (Wei et al., 1 Sep 2024). This distinction has direct consequences:

• Editing model parameters at the subject (entity) locus does not reliably modify relational knowledge, and vice versa.
• Causal tracing reveals non-equivalent propagation and storage signatures for subject vs. relation: indirect effects (IE) peak in MLP for entities and in attention modules for relations.

Knowledge editing methods developed under the subject-centric paradigm (e.g., ROME, PMET), which update MLPs at subject tokens, are shown to generalize edits incorrectly: changing a single fact about a subject (e.g., birthplace) can inadvertently corrupt unrelated relations (e.g., spouse), resulting in over-generalization. Relation-centric editing, as in RETS (Liu et al., 27 Aug 2024), applies targeted updates at the MLP of the last relation token in middle-late layers, augmented with explicit "subject constraints" to preserve non-targeted subject–relation facts. This approach achieves high efficacy while dramatically increasing relation-specificity ("R-specificity") and preventing collateral changes to other facts about the subject.

Storage Target	Localization	Editing Effect
Entity knowledge	MLP (lower/middle)	Edits entity only
Relation knowledge	Attention + MLP	Edits relation only

3. Neuron- and Subnetwork-Level Relational Knowledge

At the finest granularity, individual and sets of neurons can be mapped to the representation and manipulation of specific relations. Using statistics-based techniques to track activation patterns across tasks (Liu et al., 24 Feb 2025), relation-specific neurons are identified by their discriminative activation on relation-centric prompts, distinct from activation on other relations. Experimental deactivation of these neurons yields several robust properties:

• Cumulativity: The more relation-specific neurons are deactivated, the more facts of that relation fail to be recalled.
• Versatility: Some neurons are shared across relations, even with low semantic overlap; certain relation-neuron effects transfer cross-lingually.
• Interference: Deactivating neurons for one relation can, in rare cases, improve model accuracy for other, possibly competing, relations.

Discovered relational knowledge subnetworks can be extremely sparse: as little as 1.5–2% of the model’s weights are sufficient to encode (or suppress) collections of relational facts (Bayazit et al., 2023). Removing such subnetworks sharply degrades recall for targeted relational facts while leaving other functional capacities of the model essentially intact, demonstrating modular but non-unique representations.

4. Geometric and Embedding-Based Representations of Relations

Relational knowledge is also accessible via geometric embeddings learned by LMs:

• Fine-tuned masked models (e.g., RelBERT (Ushio et al., 2023)) extract relation vectors for word or entity pairs by feeding templated prompts and pooling token representations (excluding [MASK] tokens). These vectors, when trained with contrastive objectives, cluster similar relation pairs and discriminate fine-grained relation types much more effectively than raw word embeddings or KG-based approaches.
• Many linguistic, factual, and commonsense relations can be approximated by affine mappings (linear relation representations) acting on the hidden state of the subject. Approximately 48% of tested relations in large LMs are robustly decoded by such linear functions (Hernandez et al., 2023). However, more compositional, contextual, or structurally diffuse relations are not captured linearly and require further investigation.

The nature of relational representations supports compositional and analogy reasoning, enabling strong generalization—even to relation types and entity pairings unseen during training.

5. Integration with Structured and Dynamic Knowledge Graphs

Hybrid approaches dynamically integrate external knowledge graphs (KGs) and their relational substructures with LMs. Message-passing architectures, such as in KELM (Lu et al., 2021), utilize hierarchical graphs unifying token-level, entity-level, and multi-hop relational graph structures. Context-sensitive attention and hierarchical relational-GNNs allow flexible, dynamic disambiguation and propagation of knowledge context, outperforming static entity-linking or pretraining-dependent methods in transfer and domain adaptation. Further, conditioning LMs on graph-derived relational memories (triples) or using GNN-encoded structured prompts enhances factual text generation, coherence, and support for logical reasoning and causal interventions (Liu et al., 2022, Wu et al., 6 Jun 2025).

6. Evaluation, Limitations, and Taxonomies

Probing relational knowledge in LMs uses rigorous, architecture-agnostic frameworks. BEAR (Wiland et al., 5 Apr 2024) and LM-PUB-QUIZ (Ploner et al., 28 Aug 2024) enable standardized, bias-mitigated, and balanced evaluation of relational knowledge through log-likelihood ranking of multiple-choice cloze statements, covering multi-token answers and a wide spectrum of relations. This allows cross-model comparison regardless of causal/masked structure and highlights persistently low recall and accuracy on challenging/harder instances. Rank-then-select paradigms for multi-valued relations remain difficult: current LMs exhibit F1 scores below 50% for multi-valued slot-filling (Singhania et al., 2023), with recall limited by token probability calibration and prompt sensitivity.

Recent taxonomies (Safavi et al., 2021) situate LMs’ relational knowledge representation along a supervision axis: from fully implicit (pretraining only), entity-level (entity linking or fusion), to relation-level (explicit relation embeddings/templates). LMs show strong recall for high-frequency, 1-to-1 relations but less so for many-to-many or abstract relations. While symbolic knowledge bases give precision and interpretability but suffer schema rigidity, LMs afford flexibility and open-class relation capture at the cost of opacity and inconsistent inference. Hybrid architectures and continual learning frameworks integrating both paradigms are active research frontiers.

7. Open Challenges and Theoretical Implications

Despite clear evidence that LMs internalize significant relational knowledge, persistent challenges remain:

• Most neural models, including contemporary deep LMs, still fail to implement truly compositional, dynamic binding for arbitrary role-filler pairs or for robust relational generalization outside the training distribution (Puebla et al., 2019).
• The storage of relational knowledge is multifaceted and not easily modifiable: entity and relation knowledge reside and propagate through partially separable, overlapping modules with non-equivalent editability (Wei et al., 1 Sep 2024).
• Human-like predicate abstraction (i.e., symbolic representations enabling arbitrary binding and flexible inference) is not yet reliably realized in neural LMs; association- and interpolation-driven mechanisms are dominant.

Advances in interpretability, knowledge editing, and hybrid symbolic–neural reasoning systems are required to address these fundamental limitations.

---

References

• Relation Also Knows: Rethinking the Recall and Editing of Factual Associations in Auto-Regressive Transformer LLMs (Liu et al., 27 Aug 2024)
• Locating and Extracting Relational Concepts in LLMs (Wang et al., 19 Jun 2024)
• On Relation-Specific Neurons in LLMs (Liu et al., 24 Feb 2025)
• Does Knowledge Localization Hold True? Surprising Differences Between Entity and Relation Perspectives in LLMs (Wei et al., 1 Sep 2024)
• BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked LLMs (Wiland et al., 5 Apr 2024)
• KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs (Lu et al., 2021)
• Relational Memory Augmented LLMs (Liu et al., 2022)
• RelBERT: Embedding Relations with LLMs (Ushio et al., 2023)
• Linearity of Relation Decoding in Transformer LLMs (Hernandez et al., 2023)
• Discovering Knowledge-Critical Subnetworks in Pretrained LLMs (Bayazit et al., 2023)
• The relational processing limits of classic and contemporary neural network models of language processing (Puebla et al., 2019)
• Relational World Knowledge Representation in Contextual LLMs: A Review (Safavi et al., 2021)
• Extracting Multi-valued Relations from LLMs (Singhania et al., 2023)
• LM-PUB-QUIZ: A Comprehensive Framework for Zero-Shot Evaluation of Relational Knowledge in LLMs (Ploner et al., 28 Aug 2024)
• LLMs are Good Relational Learners (Wu et al., 6 Jun 2025)