ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget

Abstract: Entity Linking (EL) and Relation Extraction (RE) are fundamental tasks in Natural Language Processing, serving as critical components in a wide range of applications. In this paper, we propose ReLiK, a Retriever-Reader architecture for both EL and RE, where, given an input text, the Retriever module undertakes the identification of candidate entities or relations that could potentially appear within the text. Subsequently, the Reader module is tasked to discern the pertinent retrieved entities or relations and establish their alignment with the corresponding textual spans. Notably, we put forward an innovative input representation that incorporates the candidate entities or relations alongside the text, making it possible to link entities or extract relations in a single forward pass and to fully leverage pre-trained LLMs contextualization capabilities, in contrast with previous Retriever-Reader-based methods, which require a forward pass for each candidate. Our formulation of EL and RE achieves state-of-the-art performance in both in-domain and out-of-domain benchmarks while using academic budget training and with up to 40x inference speed compared to competitors. Finally, we show how our architecture can be used seamlessly for Information Extraction (cIE), i.e. EL + RE, and setting a new state of the art by employing a shared Reader that simultaneously extracts entities and relations.

• The paper introduces ReLiK, a unified architecture that integrates retrieval and reading to streamline entity linking and relation extraction tasks.
• It leverages dense passage retrieval and a single forward pass with DeBERTa-v3, achieving EL processing up to 40 times faster than competitors.
• Empirical results show state-of-the-art performance with improvements up to 8.3 percentage points, making the system accessible for research groups with limited resources.

ReLiK: Retrieve and Link, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget

The paper "ReLiK: Retrieve and Link, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget" introduces a novel Retriever-Reader architecture named ReLiK. This architecture is designed for efficient and effective Entity Linking (EL) and Relation Extraction (RE), and it integrates these tasks seamlessly within a unified framework. The authors aim to address the limitations of existing EL and RE systems by focusing on three fundamental properties: inference speed, flexibility, and performance. ReLiK achieves state-of-the-art results while being computationally efficient, making it accessible for research groups with limited resources.

Key Innovations and Contributions

ReLiK is composed of two main components: the Retriever and the Reader.

1. Retriever: This component identifies candidate entities or relations from a given input text. It employs a dense passage retrieval paradigm to encode queries (input text) and passages (textual representations of entities or relations) into dense vector representations. The Retriever ranks the most relevant entities or relations based on these dense representations.
2. Reader: This component takes the input text along with the retrieved candidate entities or relations and establishes connections between the textual spans and the entities or relations. Importantly, the Reader performs this task in a single forward pass, leveraging modern LLMs such as DeBERTa-v3 to efficiently contextualize the input along with the candidate entities or relations.

Strong Numerical Results

The paper reports strong numerical results across several benchmarks:

• Entity Linking: ReLiK sets new state-of-the-art results on both in-domain (AIDA) and out-of-domain datasets (MSNBC, Derczynski, KORE50, N3-Reuters-128, N3-RSS-500, OKE-15, OKE-16). Specifically, the large version of ReLiK (ReLiK_L) achieves the highest performance, significantly outperforming previous state-of-the-art systems by demonstrating an improvement of up to 8.3 percentage points on certain out-of-domain datasets.
• Relation Extraction: ReLiK shows superior performance on popular RE benchmarks such as NYT and CONLL04, matching or surpassing other state-of-the-art systems. ReLiK is particularly notable for its performance on the REBEL dataset for closed Information Extraction (cIE), achieving high F1 scores for both EL and RE components.

Comparative Analysis and Implications

Compared to existing EL systems, particularly those that rely on sequence-to-sequence models or retriever-reader pairs, ReLiK's unified architecture offers substantial efficiency gains. For example, the system achieves a remarkable advancement in speed, processing EL tasks up to 40 times faster than its closest competitors due to its ability to handle multiple candidate entities in a single forward pass. This efficiency makes ReLiK suitable for real-time and large-scale applications.

Moreover, the flexible design of the Retriever component enhances the model’s capability to handle unseen entities or relations, ensuring robust performance across diverse datasets and scenarios. This flexibility is particularly critical for out-of-domain tasks where traditional methods often fail.

Speculations on Future Developments

Given ReLiK's strong performance and efficiency, future developments could explore its application to broader Information Extraction (IE) tasks beyond EL and RE. Potential areas include automatic text summarization, knowledge base construction, and semantic parsing. Moreover, integrating dynamic entity and relation indices, as well as leveraging continual learning paradigms, could enhance the model's adaptability to evolving knowledge bases.

Furthermore, addressing challenges such as emerging entities and the automated generation of textual representations for entities and relations can reduce dependency on static knowledge bases, fostering improvements in the Retriever component's effectiveness.

Conclusion

ReLiK represents a significant step forward in the field of Information Extraction, offering a fast, flexible, and high-performing solution for Entity Linking and Relation Extraction. The architecture's efficiency allows for cost-effective training and deployment, making state-of-the-art IE accessible to a wider range of research groups and practical applications. By unifying EL and RE within a single framework, ReLiK sets the stage for future innovations that could further enhance the capabilities and applicability of Information Extraction systems.