- The paper introduces WAKA, a tool that combines automated NLP and human supervision to construct accurate knowledge graphs.
- It details a three-phase algorithm integrating entity discovery, mREBEL-based relationship extraction, and RDF-formatted knowledge fusion.
- Quantitative evaluation on the RED FM dataset shows satisfactory recall rates while highlighting the need for improved precision through expert input.
Introduction
Marcel Gohsen and Benno Stein from Bauhaus-Universität Weimar present a comprehensive approach to alleviating the challenges of knowledge graph (KG) construction from unstructured text—a complex but essential process in various information retrieval applications. They introduce WAKA, a web application enabling domain experts to create knowledge graphs using natural language, thereby levering experts' familiarity with their own domain language.
WAKA: The Web Application
WAKA is designed with a user-friendly authoring interface comprising two primary components: a text editor and an interactive graph visualization. Its main advantage is the ability to link entities and relationships to Wikidata entries while permitting the addition of new elements. Users can write or import text which WAKA processes to construct a proposed KG that can be further edited and expanded upon through the interface. This integration of automated processes and human oversight ensures that domain experts can refine the automatically generated KGs accurately and efficiently.
Knowledge Graph Construction Algorithm
The paper discusses the algorithmic intricacies of WAKA's knowledge graph creation, detailing a three-part process encompassing entity discovery, relationship extraction, and knowledge fusion. To start, the entity discovery pipeline identifies named entities in the text, seeking high recall to provide an extensive pool of entities for the later stages. Then, the relationship extraction pipeline, spearheaded by the mREBEL tool, extracts triples representing semantic relations grounded in Wikidata's ontology. In the knowledge fusion phase, these components are synthesized to build the resulting KG, represented in the RDF format.
The entity discovery process employs various named entity recognition models and ranks entities by relevance, considering the match quality of mention and context. For relationship extraction, the approach uses mREBEL due to its alignment with Wikipedia abstracts and Wikidata relations. The knowledge fusion step completes the construction by drawing subjects and objects from the entity pool to form precise RDF triples.
Quantitative Evaluation and Conclusion
The paper's quantitative evaluation highlights the performance of distinct KG construction components using the RED FM dataset, which aligns Wikipedia abstracts with Wikidata triples. It reveals that while the recall rates are satisfactory, precision requires enhancement—a finding that underscores the algorithm's complexity and ratifies the necessity for human input. The evaluation offers insights into the performance at the level of entity retrieval, reranking, and the natural language inference employed to check if triples can be inferred from text, which enhances the selection of highest-scoring triples for knowledge fusion.
In conclusion, WAKA serves as a significant step towards facilitating the creation of knowledge graphs from natural language, acknowledging the limitations of automated construction and emphasizing the essential role of human expertise in the refinement process. The introduction of this tool promises to streamline KG authoring, fostering subsequent advancements in domain-specific information systems.