GRISP: Guided Recurrent IRI Selection over SPARQL Skeletons

Abstract: We present GRISP (Guided Recurrent IRI Selection over SPARQL Skeletons), a novel SPARQL-based question-answering method over knowledge graphs based on fine-tuning a small LLM (SLM). Given a natural-language question, the method first uses the SLM to generate a natural-language SPARQL query skeleton, and then to re-rank and select knowledge graph items to iteratively replace the natural-language placeholders using knowledge graph constraints. The SLM is jointly trained on skeleton generation and list-wise re-ranking data generated from standard question-query pairs. We evaluate the method on common Wikidata and Freebase benchmarks, and achieve better results than other state-of-the-art methods in a comparable setting.

• The paper introduces GRISP as an efficient framework for KGQA by decomposing query synthesis into SPARQL skeleton generation and recurrent IRI selection.
• It employs beam search and guided re-ranking to generate robust queries, achieving notable F₁-score improvements on both Freebase and Wikidata benchmarks.
• The approach offers fast inference on moderate hardware while providing error recovery through backtracking and schema-constrained candidate selection.

Guided Recurrent IRI Selection over SPARQL Skeletons for Efficient Knowledge Graph QA

Introduction

The paper "GRISP: Guided Recurrent IRI Selection over SPARQL Skeletons" (2604.21133) introduces GRISP, a SPARQL-based knowledge graph question-answering (KGQA) approach that leverages fine-tuned small LLMs (SLMs). GRISP is positioned as a method that efficiently bridges the gap between agentic, zero-shot KGQA using LLMs—known for their accuracy but high latency—and fine-tuned, generate-then-retrieve SLM approaches, which offer fast, high-quality structured query synthesis when annotated question-query pairs are available.

The GRISP framework integrates two critical innovations: (1) natural-language SPARQL skeleton generation and (2) iterative, guided IRI selection by listwise re-ranking. The method yields state-of-the-art effectiveness and inference latency on complex benchmarks for Freebase and Wikidata, maintains compatibility with moderate hardware (e.g., a single GPU), and demonstrates a robust ablation profile across inference and retrieval mechanisms.

Methodology

GRISP decomposes KGQA into sequential semantic parsing and knowledge graph lookup sub-tasks, both mediated by a fine-tuned SLM. Given a question, the model first produces multiple candidate SPARQL skeletons—queries in which concrete IRIs are replaced by descriptive natural-language placeholders. For each skeleton, IRIs are recurrently resolved and selected using re-ranking conditioned on the knowledge graph's structure and content.

Figure 1: The data synthesis process for both the skeleton generation and re-ranking stages, visualizing the iterative substitution and candidate selection protocol employed by GRISP.

Key components of GRISP include:

• Beam Search for Skeleton Generation: SLMs with beam search produce a diverse set of SPARQL skeleton queries, filtering on parseability and uniqueness, before taking top candidates for downstream resolution.
• Iterative Re-Ranking and Selection: For every placeholder, an index search is invoked over KG entities or properties, and the top-k results are re-ranked with SLM-based scoring to select replacements. Alternatives are encoded using single-token identifiers, and re-ranking leverages next-token logit scores.
• Backtracking for Dead-End Recovery: When selection leads to invalid or empty queries (e.g., no results or ill-formed queries), GRISP backtracks to prior states and resumes search from untried alternatives, enabling robust error correction.
• Knowledge Graph Guidance: Constraints derived from partially resolved skeletons restrict the candidate search space, enforcing schema coherence and contextual accuracy at each selection step.
• Final Validation: After resolving all skeleton placeholders with IRIs, the resulting SPARQL query is executed on the target knowledge graph for validity and result non-emptiness.

The entire approach is trained using paired question-query data, where the model learns both skeleton synthesis (with placeholders substituted 80% with main labels and 20% with aliases) and listwise re-ranking via cross-entropy loss on correct candidate identifiers. Data augmentation (random top-k, alias swapping, shuffling, target-dropping) further improves generalization and robustness.

Experimental Evaluation

GRISP is extensively evaluated against contemporary KGQA systems, notably ChatKBQA and WikiSP, across benchmarks from both Freebase (CWQ, WQSP) and Wikidata (QALD-7/10, WWQ, WDQL, LC-QuAD 2.0, SimpleQuestions, QAWiki, SPINACH). The SLMs are based on the Qwen2.5 family ranging from 0.5B to 7B parameters, always under consumer-level hardware constraints.

GRISP achieves the highest F₁-scores among all compared SLM approaches in all tested datasets. Noteworthy relative improvements on Wikidata benchmarks include:

• QALD-7: GRISP F₁ = 46.0, outperforming WikiSP (43.6, 38.9)
• WWQ (dev): GRISP F₁ = 85.3, versus WikiSP's max of 76.5
• WDQL: GRISP F₁ = 57.6, versus WikiSP at 16.0

On Freebase datasets, it exceeds ChatKBQA by several points, with query times averaging 3–6 seconds—an order of magnitude faster than agentic LLM methods.

Component ablation confirms the additive benefit of GRISP's search, re-ranking, KG guidance, and backtracking mechanisms. The most significant performance drops occur when disabling beam search and re-ranking—core to GRISP's reliable skeleton generation and selection. Model scaling also reflects predictable improvements, with diminishing returns after 7B parameters.

Implications and Theoretical Significance

The architecture of GRISP highlights the practical synergy between SLM fine-tuning and KG-guided retrieval for KGQA. By abstracting SPARQL synthesis into skeleton generation and IRI resolution, the approach achieves both:

• Controlled Semantic Parsing: The skeleton intermediary constrains the model to produce structurally valid queries, drastically reducing semantic hallucinations or schema violations that hinder end-to-end generative approaches. This scaffolding enables more effective downstream selection and correction.
• Efficient, High-Coverage Retrieval: Iterative, SLM-guided re-ranking allows context-sensitive constraint enforcement and error recovery through backtracking. This method mimics the agentic recursive reasoning of LLMs while maintaining computational feasibility for moderate hardware.
• Task Transfer and Modularity: Skeleton-based decomposition is naturally extensible to other structured query tasks (e.g., SQL, Cypher), signifying theoretical portability across KGQA paradigms.

Given high training data regimes, GRISP matches or outperforms far larger zero-shot models with drastically reduced deployment cost, suggesting future directions for hybrid KGQA systems that combine explicit symbolic scaffolding with SLM reasoning and lookups. However, in extremely low-data settings (e.g., QALD benchmarks), the advantages of zero-shot LLM agents persist.

Future Directions

Possible extensions involve further refinement of the skeleton representation to fully support advanced SPARQL constructs (e.g., UNION), tighter integration with large-scale dynamic knowledge graphs, and the use of more sophisticated data augmentation or negative sampling for re-ranking. Given theoretical guarantees of valid query synthesis, GRISP-like frameworks may also serve as controllable front-ends for neuro-symbolic search agents, or as modular sub-components for more complex multi-hop reasoning and KG navigation systems.

Conclusion

GRISP introduces a rigorous, modular framework for KGQA by combining SLM-driven SPARQL skeleton generation with guided, recurrent IRI selection, outperforming competing SLM-based and agentic LLM approaches on efficacy and speed given sufficient training data. Its design offers an efficient and robust blueprint for practical, high-quality KGQA in real-world settings, and provides a foundation for future research into the next generation of hybrid, fine-tuned, and interpretable QA systems over large-scale knowledge graphs.