Multi-Faceted Continual Knowledge Graph Embedding for Semantic-Aware Link Prediction

Abstract: Continual Knowledge Graph Embedding (CKGE) aims to continually learn embeddings for new knowledge, i.e., entities and relations, while retaining previously acquired knowledge. Most existing CKGE methods mitigate catastrophic forgetting via regularization or replaying old knowledge. They conflate new and old knowledge of an entity within the same embedding space to seek a balance between them. However, entities inherently exhibit multi-faceted semantics that evolve dynamically as their relational contexts change over time. A shared embedding fails to capture and distinguish these temporal semantic variations, degrading lifelong link prediction accuracy across snapshots. To address this, we propose a Multi-Faceted CKGE framework (MF-CKGE) for semantic-aware link prediction. During offline learning, MF-CKGE separates temporal old and new knowledge into distinct embedding spaces to prevent knowledge entanglement and employs semantic decoupling to reduce semantic redundancy, thereby improving space efficiency. During online inference, MF-CKGE adaptively identifies semantically query-relevant entity embeddings by quantifying their semantic importance, reducing interference from query-irrelevant noise. Experiments on eight datasets show that MF-CKGE achieves an average (maximum) improvement of 1.7% (2.7%) and 1.4% (3.8%) in MRR and Hits@10, respectively, over the best baseline. Our source code and datasets are available at: https://anonymous.4open.science/r/MF-CKGE-04E5.

• The paper introduces MF-CKGE, which decouples temporal and semantic embeddings to prevent overwriting and enhance continual link prediction.
• It employs pointer-based semantic de-redundancy and query-adaptive importance weighting to optimize memory usage and inference efficiency.
• Experimental results show up to 3.8% improvement in MRR, demonstrating effective knowledge retention across evolving knowledge graphs.

Multi-Faceted Continual Knowledge Graph Embedding for Semantic-Aware Link Prediction: An Expert Review

Introduction and Motivation

Continual Knowledge Graph Embedding (CKGE) faces the challenge of integrating new entities and relations into KGs while preserving prior semantic knowledge. Most existing CKGE methods, relying on shared embedding spaces, inevitably entangle multi-faceted temporal semantics, which leads to suboptimal performance in evolving KGs where entity semantics change dynamically. Addressing the semantic entanglement, the paper "Multi-Faceted Continual Knowledge Graph Embedding for Semantic-Aware Link Prediction" (2604.10947) introduces MF-CKGE, a framework specifically designed to separate, de-redundantize, and adaptively leverage entity semantics for improved continual link prediction.

Figure 1: Distribution of knowledge domains for entities across evolving snapshots on the HYBRID dataset, highlighting the increasing semantic diversity of entities.

Problem Formulation

The authors formalize evolving KGs as sequences of snapshots, where each snapshot contains a set of entities, relations, and factual triples. At each step, new knowledge may be introduced, either as entirely new triples or as expansions to existing entities or relations. The lifelong link prediction task requires a model to incrementally learn from each snapshot and perform link prediction on the union of all test queries up to the current time, under the constraint that prior training data is unavailable in subsequent steps. This stringent protocol measures both knowledge retention and acquisition dynamics.

Figure 2: An evolving KG with three snapshots, illustrating how temporally distinct knowledge facets accumulate for David Beckham, motivating disentangled semantic representation.

Methodological Framework

MF-CKGE comprises two targeted modules:

1. Entity Embeddings Decoupling

Temporal Decoupling: The framework allocates distinct embedding spaces for old and new knowledge per entity to prevent overwriting and loss of temporally-local semantic facets. For completely novel entities or relations, new embeddings are randomly initialized. For partially new entities or relations, embeddings are inherited and extended as necessary. Importantly, static entities and relations that do not appear in new knowledge are excluded from retraining to optimize computation.

Semantic Decoupling: To eliminate redundancy from incremental updates within the same semantic domain, MF-CKGE merges highly similar entity embeddings via cosine-based similarity thresholding ($\theta$). Instead of storing multiple redundant embeddings, identical facets are represented using lightweight pointer redirection, reducing memory consumption without sacrificing performance.

2. Semantic-Aware Link Prediction

At inference, MF-CKGE computes query-specific importance weights for each entity’s semantic facets. For a link prediction query $(h, r, ?)$, it quantifies the alignment between the semantic domain of each snapshot’s embedding (via the associated relations) and the query’s relation. The importance is aggregated using a Top-$k$ strategy, and query predictions are weighted accordingly, suppressing noise from irrelevant domains while amplifying relevant evidence.

Figure 3: Visualization of the average importance assigned to each snapshot embedding across test sets, with diagonal dominance indicating effective query-relevance discrimination.

Experimental Evaluation

Main Results

Extensive experimentation was performed on eight KG benchmarks—ranging from synthetic (ENTITY, RELATION, HYBRID) to large-scale (FB-CKGE, DB-CKGE) and various growth regimes (GraphEqual, GraphHigher, GraphLower). MF-CKGE consistently outperforms state-of-the-art regularization-, architecture-, and replay-based CKGE baselines. Average improvements are 1.7% in MRR and 1.4% in Hits@10, with statistically significant gains up to 3.8% in some settings.

Notably, MF-CKGE even slightly outperforms retraining-from-scratch on heavily updated KGs in some configurations, establishing that semantic decoupling with query-adaptive inference can exceed naïve global retraining in both efficiency and accuracy.

Figure 4: Training time comparisons, demonstrating MF-CKGE's significant reduction in computational cost compared to other CKGE methods and retraining.

Figure 5: GPU memory usage analysis, indicating that semantic decoupling avoids the memory explosion typically seen in multi-branch architecture approaches.

Knowledge Retention versus Acquisition

MF-CKGE robustly preserves performance on older test queries as the KG evolves—empirical results show negligible or even positive drift on "forgotten" knowledge, due to the selective retention and de-redundancy of embeddings.

Figure 6: Effectiveness of MF-CKGE for retaining old knowledge across snapshots, measured by stable MRR versus best CKGE methods.

Ablation and Sensitivity Analysis

Ablation reveals that both temporal/semantic decoupling and semantic-aware importance weighting are critical; removing either module consistently and substantially degrades MRR and Hits@10. Parameter sensitivity experiments further show that aggressive semantic compression (high $\theta$) can reduce model size up to 30% without meaningful loss in link prediction accuracy.

Figure 7: Effect of semantic similarity threshold $\theta$ on model space compression and link prediction effectiveness.

Figure 8: Effect of Top-$k$ on importance aggregation parameter, confirming optimal aggregation balancing semantic focus and noise reduction.

Comparative Analysis and Theoretical Implications

MF-CKGE's core insight is the explicit separation of an entity's multi-faceted, temporally-evolving semantics, challenging the canonical practice of unified entity vector spaces. The evidence shows this approach both mitigates catastrophic forgetting and improves generalization to volatile KG regions, a property highly relevant for reasoning-intensive LLM-based applications, graph-based retrieval, and real-time knowledge graph augmentation.

From a practical viewpoint, the de-redundant, pointer-based embedding storage model significantly lowers both runtime memory and computational footprint, advancing MF-CKGE's scalability to truly web-scale, frequently-updated KGs.

Case Studies

Detailed case studies confirm that MF-CKGE prioritizes query-relevant entity facets (e.g., academic field over graduates for university queries), accurately surfacing ground-truth link predictions. Heatmap visualizations further validate the adaptivity of facet importance by query context.

Conclusion

MF-CKGE establishes a new direction for CKGE by treating entity semantics as multi-faceted and temporally distributed, rather than statically unified. The combined effect of facet decoupling, domain de-redundancy, and adaptive importance weighting yields strong empirical results under evolving KG protocols, together with efficiency gains. The principled approach portends robust integration with dynamic knowledge-augmented LLM systems and real-time semantic reasoning in continual learning environments. Future work may investigate tighter integration with neural-symbolic systems and its impact on knowledge-augmented generative modeling.