Reinforcement Learning based Collective Entity Alignment with Adaptive Features (2101.01353v1)

Abstract: Entity alignment (EA) is the task of identifying the entities that refer to the same real-world object but are located in different knowledge graphs (KGs). For entities to be aligned, existing EA solutions treat them separately and generate alignment results as ranked lists of entities on the other side. Nevertheless, this decision-making paradigm fails to take into account the interdependence among entities. Although some recent efforts mitigate this issue by imposing the 1-to-1 constraint on the alignment process, they still cannot adequately model the underlying interdependence and the results tend to be sub-optimal. To fill in this gap, in this work, we delve into the dynamics of the decision-making process, and offer a reinforcement learning (RL) based model to align entities collectively. Under the RL framework, we devise the coherence and exclusiveness constraints to characterize the interdependence and restrict collective alignment. Additionally, to generate more precise inputs to the RL framework, we employ representative features to capture different aspects of the similarity between entities in heterogeneous KGs, which are integrated by an adaptive feature fusion strategy. Our proposal is evaluated on both cross-lingual and mono-lingual EA benchmarks and compared against state-of-the-art solutions. The empirical results verify its effectiveness and superiority.

Entity alignment (EA) is the crucial task of identifying equivalent entities across different knowledge graphs (KGs). Traditional methods often align entities independently based on pairwise similarity, failing to capture the inherent interdependence between alignment decisions. More recent approaches impose a strict 1-to-1 constraint, but this can lead to suboptimal results due to error propagation and doesn't fully model complex interdependencies. Furthermore, methods using multiple types of features often rely on manual weight assignment for fusion, which is not scalable or adaptable.

This paper proposes the Collective Entity Alignment with Adaptive Features (CEAFF) framework to address these limitations. CEAFF approaches EA as a sequence decision problem and employs a deep reinforcement learning (RL) model to perform collective alignment, explicitly incorporating coherence and exclusiveness constraints. It also introduces an adaptive feature fusion strategy to dynamically combine different similarity signals.

The CEAFF framework consists of three main stages:

1. Feature Generation: This stage extracts representative features from the KGs.
   • Structural Information: Entity embeddings are learned using Graph Convolutional Networks (GCNs). The implementation uses two 2-layer GCNs, one for each KG, with shared weights. Initial features are randomly sampled and L2-normalized. Training uses a margin-based ranking loss function to minimize the distance between embeddings of aligned entities while maximizing it for negative pairs.
   • Semantic Information: Entity names are embedded using averaged pre-trained word embeddings (e.g., fastText, MUSE multilingual embeddings). This captures the semantic similarity between entity names.
   • String Information: The string similarity between entity names is calculated using the Levenshtein distance. This is particularly effective for mono-lingual KGs or KGs with closely related languages and does not require external resources. A string similarity matrix is generated.
2. Adaptive Feature Fusion: This stage fuses the feature-specific similarity matrices ($M^s$ for structural, $M^n$ for semantic, $M^l$ for string) into a single fused similarity matrix $M$.
   • The Bray-Curtis dissimilarity is used to measure the similarity between entity embeddings for structural and semantic features. This measure applies normalization per element pair, which is shown empirically to perform better than common alternatives like Manhattan distance, Euclidean distance, or cosine similarity.
   • An adaptive strategy determines the weights for each feature dynamically without requiring training data or manual tuning. It identifies "confident correspondences" as entity pairs that are top-ranked mutually along both row and column in a feature's similarity matrix. The importance of a feature is related to the confident correspondences it generates. Weights for correspondences are inversely proportional to how many features generate them, with thresholds ($\theta_1, \theta_2$) to bound weights. Feature weights are then derived from the aggregated correspondence weights.
   • The final fused similarity matrix $M$ is a weighted combination of the feature-specific matrices. This outcome-level fusion approach operates on similarity scores directly, aiming to preserve feature characteristics better than representation-level fusion.
3. Collective Entity Alignment (RL-based): The fused similarity matrix $M$ is used as input for the alignment decision process, formulated as a sequence decision problem solved by an Actor-Critic RL model.
   • Environment: Represents the KGs, entities, and the fused similarity matrix.
   • State: At each step (when considering a source entity), the state is a combination of three components:
     • Local similarity vector: Similarity scores from $M$ between the current source entity and all candidate target entities.
     • Exclusiveness vector: Indicates which target entities have already been chosen in previous steps.
     • Coherence vector: Measures the relevance of candidate target entities to target entities previously matched with related source entities (based on KG structure).
   • Action: The RL agent (Actor) selects a target entity for the current source entity from a set of candidates (e.g., top-$\tau$ based on local similarity). The Actor uses a neural network (fully connected + softmax) to output probabilities over candidates.
   • Reward: The reward for taking an action (choosing a target entity) is defined as the product of local similarity and exclusiveness signals, plus the coherence signal. This encourages choosing high-similarity, available, and coherent target entities.
   • Critic: A separate neural network (two fully connected layers) estimates the value of the current state.
   • Optimization: The Actor and Critic networks are trained separately using the Temporal-Difference (TD) error computed by the Critic. The learning sequence of source entities is prioritized by their maximum local similarity score.
   • Preliminary Treatment: Before the main RL process, entities that can be confidently aligned based on mutual top-1 matches in the fused similarity matrix are filtered out. This reduces the search space for the RL agent, speeds up convergence, and provides initial confident matches for the coherence signal. The number of preliminary treatment rounds is a tunable parameter.

Implementation Considerations and Practical Notes:

• Computational Requirements: Feature generation, especially GCN training, can be computationally intensive depending on KG size. Adaptive feature fusion involves matrix operations. The RL training process requires iterating through epochs and entities, and its efficiency depends on hyperparameters like the number of candidates ($\tau$) and epochs. Empirically, while potentially slower than simple stable matching on small datasets, the RL approach with preliminary treatment can be faster than exact combinatorial optimization methods like the Hungarian algorithm on larger KGs.
• RL Training Stability: RL models, particularly Actor-Critic methods, can be sensitive to hyperparameters (learning rates $\alpha, \beta$, decay factor $\gamma$, $\tau$). Careful tuning on a validation set is necessary. The paper uses a relatively small hidden state size (10) in the RL networks, suggesting a simplified model might be sufficient given the state representation.
• Data Handling: Efficient storage and access to adjacency matrices, entity embeddings, and similarity matrices are required. For large test sets, calculating and storing the full similarity matrix might be memory-intensive; optimization like considering only top-$\tau$ candidates helps here.
• Feature Encoding: While the paper uses standard GCN, averaged word embeddings, and Levenshtein distance, replacing these with more advanced encoders (as suggested in future work) could improve initial feature quality.
• Trade-offs: The RL approach implicitly models the 1-to-1 preference through exclusiveness but does not enforce a strict 1-to-1 mapping globally. This can be beneficial by allowing flexibility in edge cases but also risks multiple source entities aligning to the same target entity or leaving some entities unaligned (though the current datasets assume 1-to-1 or matchable entities). The paper analyzes the extent of non-1-to-1 mappings produced by CEAFF and its variants.

Experimental Evaluation Highlights:

• CEAFF consistently achieved state-of-the-art precision on various cross-lingual and mono-lingual datasets (DBP15K, SRPRS, DBP-FB), significantly outperforming previous methods, including collective approaches like CEA and GM-EHD-JEA.
• Ablation studies confirmed the contribution of each main component: collective alignment (RL), adaptive feature fusion, and the different feature types (structural, semantic, string).
• The Bray-Curtis dissimilarity was shown to be effective in measuring similarity between entity embeddings.
• The adaptive feature fusion strategy generally outperformed using fixed equal weights and was more practical than learning-based fusion with limited training data. The use of thresholds $\theta_1, \theta_2$ in adaptive fusion was beneficial.
• The RL-based collective strategy significantly improved performance compared to independent alignment and generally surpassed 1-to-1 stable matching (CEAFF-SM), demonstrating the value of modeling both coherence and exclusiveness.
• The preliminary treatment successfully identified a large percentage of confidently alignable entities, most of which were correct, and improved the performance and convergence of the subsequent RL process.

Limitations and Future Directions:

• The error analysis reveals that a significant portion of errors is not directly attributable to the proposed components but stems from limitations in the initial features or inherent ambiguities in the KGs.
• While the collective strategy improves alignment, it can sometimes lead to error propagation, where one incorrect match influences subsequent decisions negatively. Developing more robust collective algorithms is a direction for future work.
• Establishing more challenging benchmarks, especially mono-lingual datasets that don't rely heavily on identical entity names, is needed for better evaluation.
• Further research into more sophisticated feature encoders is also a clear path for improvement.

In summary, CEAFF provides a practical and effective framework for entity alignment by combining robust multi-feature similarity measures with a novel RL-based collective alignment strategy that moves beyond strict 1-to-1 constraints, demonstrating significant performance gains on standard benchmarks.