Building Contextual Knowledge Graphs for Personalized Learning Recommendations using Text Mining and Semantic Graph Completion (2401.13609v1)

Abstract: Modelling learning objects (LO) within their context enables the learner to advance from a basic, remembering-level, learning objective to a higher-order one, i.e., a level with an application- and analysis objective. While hierarchical data models are commonly used in digital learning platforms, using graph-based models enables representing the context of LOs in those platforms. This leads to a foundation for personalized recommendations of learning paths. In this paper, the transformation of hierarchical data models into knowledge graph (KG) models of LOs using text mining is introduced and evaluated. We utilize custom text mining pipelines to mine semantic relations between elements of an expert-curated hierarchical model. We evaluate the KG structure and relation extraction using graph quality-control metrics and the comparison of algorithmic semantic-similarities to expert-defined ones. The results show that the relations in the KG are semantically comparable to those defined by domain experts, and that the proposed KG improves representing and linking the contexts of LOs through increasing graph communities and betweenness centrality.

The paper "Building Contextual Knowledge Graphs for Personalized Learning Recommendations using Text Mining and Semantic Graph Completion" addresses the limitations of hierarchical data models commonly used in digital learning platforms for enabling personalized learning. These hierarchical structures (e.g., curriculum -> course -> topic -> material) effectively represent the detail of a learning objective but struggle to capture the broader context of Learning Objects (LOs) and their relations across different domains or curricula, which is crucial for personalized recommendations beyond simple recall-level learning.

The core proposal is to transform these hierarchical data models into Knowledge Graphs (KGs) to better represent the context of LOs and facilitate personalized learning recommendations. The KG represents LOs as nodes and relations between them as edges. The transformation involves not only preserving the original hierarchical relations but also extracting new semantic relations between LOs based on their textual descriptions.

The practical implementation involves a customized Text Mining Pipeline (TMP) designed to analyze the textual metadata (titles and descriptions) of LOs, which can be in multiple languages (specifically English and German in this work). The TMP includes steps for:

1. Language Detection: Identifying the language of the text using a pre-trained model.
2. Text Cleaning: Removing special characters while preserving sentence structure.
3. Topic Extraction: For longer description texts, KeyBERT is used to extract main topics. This step is not performed for shorter titles.
4. Text Embedding: Generating vector representations (embeddings) for titles and extracted topics from descriptions using Sentence-BERT and the SpaCy library. For multilingual handling, German titles are translated to English using the DeepL API before embedding calculation to ensure comparability across languages.
5. Semantic Similarity Calculation: Computing the cosine similarity between text embeddings (titles with titles, and topic sets from descriptions). For descriptions, an intersection matrix of topics determines which topic pairs are compared, and the final similarity is an average across relevant topic pairs.
6. Relation Creation: Based on these similarity scores and experimentally defined, expert-tuned thresholds, a new type of relation, "has_semantic_relation_to", is created between LO nodes in the KG.

The KG is then constructed by integrating these newly mined semantic relations alongside the existing hierarchical relations (Journey-Course, Course-Topic, etc.). This creates a graph where LOs are nodes, and edges represent either hierarchical connections or semantic connections found through text analysis. This allows the KG to model cross-domain or cross-curriculum connections between LOs that share a similar context, even if they are not directly linked in the original hierarchy.

The evaluation assesses both the quality of the extracted semantic relations and the resulting KG structure.

• Semantic Relation Evaluation: This is done quantitatively by comparing the semantic similarity scores of the newly extracted relations between LOs from different Journeys to the average semantic similarity within expert-curated Journeys. The assumption is that if relations between different Journeys have similar semantic scores as relations within Journeys, they are likely meaningful. The results showed that 79% of the mined semantic relations had similarity scores comparable to or higher than the average within-journey similarity, indicating their relevance.
• KG Structure Evaluation: Standard graph quality metrics are used to compare the constructed KG with the original hierarchical model. The selected metrics, interpreted in the context of technology-enhanced learning (TEL) and vocational education and training (VET), include:
  • Average Degree Centrality (ADC): Increased from 1.079 (hierarchical) to 2.262 (KG), indicating better connectedness of LOs to other potential learning contexts.
  • Clustering Coefficient (CC): The number of communities increased from 253 to 541, suggesting more distinct learning contexts are represented. The average modularity decreased from 0.779 to 0.636, implying better inter-connectedness between these communities.
  • Weakly Connected Components (WCC): Reduced from 63 to 35, showing that fewer groups of LOs are isolated, improving overall graph connectivity.
  • Betweenness Centrality (BC): Increased significantly from 1.57 to 15.1, highlighting the emergence of nodes (LOs) that act as crucial bridges between different parts of the graph, which can represent transferrable skills or concepts connecting disparate learning goals.
• Qualitative Expert Evaluation: Focus groups with VET and researcher-training experts validated the approach. Experts confirmed the importance of connecting learning goals and recognized KGs as valuable contextual models. They agreed that textual descriptions are rich sources for finding contextual relations. Feedback also highlighted the need for content creators to understand the underlying process to improve descriptions and the necessity for continuous KG updates due to the dynamic nature of learning domains and learner contexts.

In summary, the paper presents a practical approach to building contextual KGs for personalized learning by leveraging text mining on LO metadata. The method, including a multilingual TMP, successfully transforms rigid hierarchical structures into a more flexible and interconnected graph representation that better captures the contextual relationships between LOs. The evaluation demonstrates that the resulting KG exhibits structural properties conducive to personalized recommendations by enhancing connectivity and identifying bridging concepts.

Implementation considerations include the dependency on the quality and volume of existing textual data for LOs, the need for efficient processing of multilingual text, and managing potential redundancy when recommending multilingual content. Future work involves improving TMP robustness for sparse data and incorporating additional domain-specific features for richer contextualization.