- The paper introduces a hybrid microservice that combines a KG-first approach with LLM-fallback to ensure precise and provenance-aware skill search and explanation.
- It demonstrates that fusing dense semantic retrieval with BM25-based lexical matching yields superior performance (nDCG@5 up to 0.948) with sub-200ms latencies.
- The study highlights that schema-constrained LLMs improve citation precision, underscoring the trade-off between fluency and verifiability in educational applications.
KG-First, LLM-Fallback: Hybrid Microservice for Grounded Skill Search and Explanation
Motivation and Problem Statement
Competency-based educational frameworks such as ESCO, ROME, and O*NET provide foundational labor-market skill corpora essential for aligning learning outcomes with occupational requirements. However, their structural heterogeneity, technical complexity, and variable multilingual coverage severely impede direct adoption by educational stakeholders. Existing access paradigms—primarily relying on SPARQL endpoints and ad hoc keyword queries—fail to offer transparent, provenance-preserving retrieval, rendering them inaccessible to non-expert users and undermining curricular design trustworthiness. The paper addresses this challenge by formalizing the requirements for an interoperable service: robust skill discovery across frameworks, explicit graph context (prerequisites, sub-skills), and audience-adaptive explanations, all rigorously grounded and provenance-aware.
Architectural Overview
The proposed SkillGraph-Service microservice adopts a strictly KG-first, LLM-fallback architectural pattern. The system first utilizes a unified, provenance-preserving RDF/SKOS Knowledge Graph integrating ESCO, ROME, and O*NET corpora (with mapping links rather than destructive merges) structured via a Competency Management Ontology (CMO). Symbolic retrieval is performed using hybrid search: lexical matching with BM25 via SQLite FTS5 and dense semantic similarity using Sentence-BERT-like embeddings indexed with HNSW for efficient nearest-neighbor search. Dense and sparse scores are fused via convex combination with parameter tuning, enabling both cross-lingual paraphrase handling and terminological precision. Graph traversal exposes explicit prerequisite and sub-skill relationships, preserving full provenance metadata.
The LLM component is strictly relegated to augmentation tasks where symbolic retrieval is insufficient: constrained re-ranking among closed candidate sets, and template- or JSON-constrained audience-aware explanation generation (which must cite evidence IDs). All LLM generations are tightly schema-bound, with deterministic template fallback if constraints are breached. Direct skill generation or free-form augmentation without grounding is strictly prohibited, reducing hallucination risk.
Experimental Evaluation
Retrieval Effectiveness
Empirical evaluation utilizes a dataset of 100 educator queries (50 EN, 50 FR), targeting fine-grained skill discovery. The BM25 lexical baseline is outperformed by dense retrieval (nDCG@5: 0.474 → 0.875), confirming the necessity of semantic embeddings for vocabulary mismatch mitigation. The hybrid fusion achieves peak effectiveness (nDCG@5: 0.948; P@5: 0.958), especially in English (nDCG@5: 0.970), demonstrating that combining lexical and dense signals yields optimal distinction—particularly for closely related but distinct skills (e.g., "Java" vs. "JavaScript"). Notably, introducing cross-encoder re-ranking degrades performance (nDCG@5 drop to 0.814), contradicting established retrieval pipeline conventions and highlighting a domain shift issue; computational overhead (>2s latency) further obviates its utility in interactive microservices.
Hybrid retrieval sustains sub-200ms response times (median latency: 188ms), demonstrating suitability for real-time digital learning ecosystem integration without the burden of distributed search clusters.
Explanation Generation
Three explanation modalities are compared: deterministic template (C1), schema-constrained JSON LLM (C2), and free-form LLM (C3). Template (C1) guarantees perfect evidence coverage and near-zero latency, but yields rigid, repetitive output. JSON-constrained LLM (C2) achieves perfect citation precision (1.0), but suffers from incomplete citation coverage (0.32), indicating that enforcing schema constraints can reduce completeness despite eliminating unsupported assertions. Free-form LLM (C3) produces fluent output but demonstrates severe faithfulness breakdown (unsupported rate 0.82), confirming that unconstrained generative approaches are unsuitable in trust-critical contexts. These results underscore the trade-off between fluency and verifiability; production interfaces should default to symbolic templates, with asynchronous LLM summaries clearly labelled as probabilistic.
Theoretical and Practical Implications
The KG-first, LLM-fallback paradigm rigorously prioritizes symbolic correctness and auditability, reserving sub-symbolic flexibility strictly for bounded augmentation. The hybrid retrieval stack (FTS5+HNSW) proves that scalable, low-latency fusion of lexical and semantic signals is sufficient for educator-facing workflows, invalidating the need for resource-intensive cross-encoder re-rankers in this domain. LLMs, while valuable for audience adaptation, must be tightly schema-bound in trust-sensitive applications, as unconstrained hallucination remains unsolved.
This architecture fosters practical integration-readiness for a range of platforms (LMSs, serious games, collaborative knowledge systems) with minimal API surface complexity. The approach advances semantic interoperability by eschewing lossy merges, enforcing explicit mappings, and preserving source integrity. Theoretical implications include the viability of KG-first retrieval for structured competency standards and the necessity of constrained generative models for factual explanation.
Speculation and Future Directions
Further work should address cross-lingual embedding discrepancies, especially optimizing semantic retrieval for languages with complex morphology and administrative nomenclature (e.g., French). Local LLM fine-tuning is required to reduce explanation module latency (<2s) without sacrificing schema adherence. Expanding the KG to integrate additional frameworks (e.g., O*NET) and incorporating user feedback-driven retraining may enhance granularity and personalization. System-level research should examine adaptive fusion parameterization and dynamic candidate set formation to further improve precision, especially for edge cases where explicit prerequisite data is sparse.
Long-term, this paradigm lays groundwork for explainable, trustworthy AI interfaces in educational and occupational domains, anchoring generative components to explicit provenance. Methodologies for formal schema constraint enforcement in LLM generation, beyond hard JSON, merit investigation. Finally, task-specific benchmarks for faithfulness vs. fluency in generative explanations would be valuable for broader adoption.
Conclusion
KG-First, LLM-Fallback establishes a robust architecture for skill search and explanation, strictly prioritizing provenance, precision, and responsiveness. Results highlight that hybrid retrieval can supersede complex re-ranking pipelines in real-world educator scenarios, and that constrained LLMs, while offering readability, require further progress to maximize evidence coverage without sacrificing trust. The system presents a scalable, auditable pathway for integrating heterogeneous skill frameworks into digital learning, with immediate practical utility and substantive implications for future AI-assisted edtech services (2605.01582).