OntoRAG: Ontology-Guided QA Pipeline

• OntoRAG is an automated pipeline that transforms unstructured PDFs into a queryable ontology for multi-hop, cross-document question answering.
• It employs web scraping, PDF parsing, semantic chunking, and graph clustering to preserve ontological integrity and improve global sensemaking.
• Its methodology yields broader, diverse answers compared to vector RAG, though it may trade off some directness in responses.

OntoRAG is an automated ontology-learning and question-answering pipeline that transforms unstructured knowledge bases into a queryable ontology and then uses that ontology to drive retrieval-augmented generation. It was introduced for a large corpus of electrical relay documentation from ABB, including manuals, datasheets, and application notes, but is presented as a general methodology for turning “a pile of unstructured PDFs” into an ontology-backed QA system. Its central objective is to eliminate manual ontology engineering while preserving ontological integrity and improving global sensemaking, especially for multi-document and multi-hop questions that standard vector-based retrieval handles poorly (Tiwari et al., 31 May 2025).

1. Problem setting and conceptual scope

OntoRAG is motivated by three linked difficulties in ontology-backed QA. First, manual ontology construction is described as slow, error-prone, expensive, and non-scalable for large, dynamic corpora. Second, vector RAG retrieves local, isolated chunks and is effective for local questions but weak on global sensemaking, where relevant information is scattered across many documents. Third, GraphRAG builds entity-relation graphs and communities, but its clustering does not explicitly preserve class hierarchies or learn a reusable ontology; it clusters primarily for summarization rather than ontology derivation (Tiwari et al., 31 May 2025).

The system’s domain choice is consequential. Electrical relay documentation contains knowledge distributed across heterogeneous document types and organized around highly relational concepts such as relays, protection functions, settings, components, failure modes, and operating conditions. Queries in this domain often require cross-document integration, for example asking about common failure modes across relays or about interactions among protection functions. OntoRAG is therefore positioned not as a generic chunk retriever with improved ranking, but as an ontology-first architecture for multi-hop, cross-document reasoning (Tiwari et al., 31 May 2025).

Within the broader literature, the name “OntoRAG” is sometimes used narrowly for this relay-document pipeline and more broadly for ontology-guided retrieval over structured knowledge graphs. A related comparative study treats retrieval over ontology-guided KGs as effectively an OntoRAG-style system, even though it does not explicitly use the term, which indicates that the label has broadened into a family of ontology-mediated RAG architectures (Cruz et al., 8 Nov 2025).

2. End-to-end pipeline from PDFs to structured knowledge

The OntoRAG pipeline has six stages: web scraping, PDF parsing, chunking, information extraction and mapping, knowledge graph construction, and ontology creation. Web scraping uses BeautifulSoup and Scrapy to crawl manufacturer websites or technical repositories, handle dynamic pages, and download PDFs. The output is an organized raw corpus of relay documents (Tiwari et al., 31 May 2025).

PDF parsing converts these documents into structured elements such as titles, headings, paragraphs, tables, and images. The system uses the Unstructured library, backed by PDFMiner, for OCR, object detection, and digital parsing. Because Unstructured can detect tables but has difficulty extracting tabular structure, OntoRAG re-extracts table regions with PyMuPDF using coordinate mapping from Unstructured units $U$ to PyMuPDF units $P$, then adds padding to capture headings and footnotes. These regions are passed to Gemini 2.5 Flash for table interpretation (Tiwari et al., 31 May 2025).

Chunking is designed to produce “First Retrieval Granular Units” that remain semantically coherent while fitting LLM context windows. Hybrid chunking starts new chunks when a Title element appears and the current chunk exceeds a minimum length threshold, while also respecting an upper length limit. Semantic chunking then refines boundaries by embedding combined sentences from adjacent chunks, computing cosine distances between neighboring embeddings, and merging adjacent sentences when similarity exceeds a manually tuned threshold. The result is a set of retrieval units aligned with document structure and semantic continuity (Tiwari et al., 31 May 2025).

Information extraction and mapping use Gemini 2.5 Flash in an in-context-learning setup. Each chunk undergoes text cleaning, disambiguation of pronouns and references, named entity recognition, atomic fact extraction into subject–predicate–object triples, and emission of a local “Chunk Graph JSON” containing nodes and edges with properties such as "part-of-speech", "name", "relationship-type", "source", and "target". Local entities are then canonicalized into “global canonical entities” by vector similarity under a strict confidence threshold. For each key term, the LLM generates a short definition, and both the name and the definition are embedded for later semantic search and ontology construction (Tiwari et al., 31 May 2025).

3. Ontology derivation and internal representation

OntoRAG’s ontology learning begins from the set $K=\{k_1,\dots,k_n\}$ of canonicalized key elements, each with a name embedding and a definition embedding. The paper defines a binary similarity function over key elements that requires both name similarity and definition similarity to exceed thresholds $\theta_{\text{name}}$ and $\theta_{\text{def}}$. Candidate classes are then formed through a disjoint-set union structure and a K-means-inspired clustering procedure, implemented in batches for scalability. These candidate classes $C_1,\dots,C_m$ become the nodes of a knowledge graph $C=(V,E)$, where edges are inherited from relations observed among the original local graph fragments (Tiwari et al., 31 May 2025).

Ontology creation proceeds by community detection over this candidate-class graph. The top-level class structure is induced with the Leiden algorithm, implemented via graspologic, producing disjoint communities $O_1,\dots,O_k$. The paper gives modularity in the standard form

$$Q=\frac{1}{2m}\sum_{i,j}\left(A_{ij}-\frac{k_i k_j}{2m}\right)\delta(c_i,c_j),$$

and notes that a human-in-the-loop supervises the choice of $k$ to obtain meaningful domain classes such as components, settings, measurements, faults, and functions. For each community, properties are aggregated from member candidate classes and intra-community relations, and Gemini 2.5 Flash synthesizes generalized class properties by merging synonyms and naming variations into coherent descriptions (Tiwari et al., 31 May 2025).

Relations between ontology classes are extracted from inter-community edges. Intra-community relations are absorbed into class properties, while repeated relations in opposite directions may be simplified into a single undirected relation. Ontology hierarchy is obtained through recursive sub-community detection on induced subgraphs, giving subclass structures $P$0. In the experiments, these levels are used as ontology levels O0–O3, ranging from root-level abstraction to fine-grained hierarchy (Tiwari et al., 31 May 2025).

The resulting representation distinguishes classes, instances, relations, and attributes in a specific way. Communities and sub-communities act as ontology classes; the original key elements function as instances or fine-grained members; inter-class edge types define relations such as “measures,” “controls,” “trips,” “has-setting,” and “is-part-of”; and generalized aggregated descriptions supply attributes or properties. Canonicalization, strict similarity thresholds, property aggregation, community detection, and human tuning collectively serve as the system’s mechanisms for conflict reduction and structural coherence (Tiwari et al., 31 May 2025).

4. Ontology-aware retrieval and question answering

Once the ontology exists, OntoRAG performs retrieval through ontology classes rather than through chunk similarity alone. Gemini 2.5 Flash first extracts key elements from the user query. The key element’s surface form is treated as its “name,” while the entire query is treated as its “definition.” These are embedded and matched, by cosine similarity, against ontology class names and definitions at a configurable ontology level from O0 to O3. For the top-matching ontology classes, OntoRAG retrieves the source chunks from which their instances and properties were derived, extends each chunk with an expanded context window such as $P$1 tokens, concatenates the resulting context, and asks Gemini 2.5 Flash to generate the answer (Tiwari et al., 31 May 2025).

This retrieval strategy differs materially from both standard vector RAG and GraphRAG. In the vector-based semantic-search baseline, retrieval is chunk $P$2 embedding $P$3 vector store, followed by nearest-neighbor search. In GraphRAG, community detection is used directly for retrieval-oriented summarization. In OntoRAG, retrieval is explicitly mediated by learned classes and hierarchy: query $P$4 key elements $P$5 ontology-class similarity $P$6 relevant classes $P$7 supporting chunks. The paper argues that this makes retrieval concept-aware and hierarchy-aware, and facilitates multi-hop reasoning by traversing class relations rather than only retrieving semantically similar chunks (Tiwari et al., 31 May 2025).

The ontology level matters. The comparison across O0–O3 suggests that intermediate ontology levels, particularly O2, provide the best trade-off between specificity and coverage. Coarser levels are more abstract, while finer levels are more granular; the paper reports that higher granularity yields slightly more claims and clusters, but judged win rates are strongest at O2 (Tiwari et al., 31 May 2025).

5. Evaluation protocol and reported performance

The evaluation uses a proprietary ABB corpus of approximately 1M tokens, chunked into 1600 chunks of 600 tokens with 100-token overlaps. Nine retrieval setups are compared: OntoRAG at ontology levels O0–O3, GraphRAG at community levels C0–C3, and semantic search (SS) as the vector RAG baseline. The benchmark contains 125 sensemaking questions generated by Gemini 2.5 Flash following GraphRAG methodology through personas and tasks, then manually validated by domain experts (Tiwari et al., 31 May 2025).

Two evaluation regimes are used. The first is qualitative head-to-head judging with Gemini 2.5 Flash over four criteria: comprehensiveness, diversity, empowerment, and directness. Each pairwise comparison is repeated five times, and statistical significance is assessed with the Wilcoxon signed-rank test with Holm–Bonferroni correction. The second is claim-based automatic evaluation: Claimify extracts unique factual claims from each answer, and agglomerative clustering with ROUGE-L distance approximates diversity via cluster counts and comprehensiveness via claim counts (Tiwari et al., 31 May 2025).

In the detailed pairwise results for OntoRAG O2 versus SS, OntoRAG wins on comprehensiveness at 88% and diversity at 86%, while SS wins on directness at 92%; empowerment is 55% in OntoRAG’s favor. Against GraphRAG C2, described as the best GraphRAG configuration, OntoRAG O2 wins on comprehensiveness at 65%, diversity at 62%, and directness at 58%, while empowerment at 52% is not significant. Across levels 0–3, OntoRAG’s comprehensiveness is reported at approximately 68–72% and diversity at approximately 66–70%, versus GraphRAG’s approximately 64–67% and 62–65% respectively. In the claim-based analysis, OntoRAG ranges from 34.9 to 35.3 claims and 13.9 to 14.2 clusters across O0–O3, GraphRAG from 34.5 to 34.8 claims and 13.6 to 13.8 clusters across C0–C3, and SS at 26.5 claims and 8.0 clusters (Tiwari et al., 31 May 2025).

The paper’s abstract also summarizes OntoRAG as achieving a comprehensiveness win rate of 85% against vector RAG and 75% against GraphRAG’s best configuration. The detailed table values reported elsewhere in the same description are 88% against SS and 65% against GraphRAG C2. The coexistence of both sets of numbers is part of the record as presented in the source material (Tiwari et al., 31 May 2025).

The empirical interpretation offered in the paper is consistent: OntoRAG tends to produce broader answers that integrate multiple aspects of complex queries, vector RAG tends to be focused and short with high directness, and GraphRAG improves breadth over vector RAG but is described as less aligned with human-interpretable class structure. This supports the view that ontology-aware retrieval changes the balance among breadth, diversity, and concision rather than uniformly dominating every answer quality dimension (Tiwari et al., 31 May 2025).

6. Broader research context and variant formulations

The relay-document system is the clearest named instance of OntoRAG, but related work shows that ontology-guided RAG has already diversified. A comparative study of knowledge-graph construction strategies evaluates RAG over KGs built from ontologies learned either from relational databases or from text. It reports that GraphRAG, text ontology KG with chunks, and RDB ontology KG with chunks each achieve 18/20 correct answers, while vector RAG achieves 12/20, and ontology-guided KGs without chunk nodes collapse to 3/20 or 4/20 correct. That study’s central result is that ontology structure alone is insufficient; chunk-level textual grounding is critical (Cruz et al., 8 Nov 2025).

A different line of work, “OntologyRAG,” applies ontology-enhanced retrieval to biomedical code mapping rather than document QA. In that system, ontologies and unrefined mappings are stored as RDF graphs in Oxigraph, an NL2SPARQL module translates natural-language questions into SPARQL, and a reasoning module assigns mapping proximity levels A, B, or C with natural-language rationale. The paper reports that direct LLM code mapping performs poorly, with GPT-4 below 10% accuracy on a 500-mapping dataset, while mapping-level prediction grounded in the KG reaches as high as $P$8 overall accuracy for GPT-4 under CoT at $P$9 (Feng et al., 26 Feb 2025).

Another adjacent formulation is RIGOR, which treats retrieval-augmented generation as a method for generating OWL ontologies from relational databases. RIGOR retrieves from the database schema and documentation, a repository of external domain ontologies, and a growing core ontology, then generates successive delta ontology fragments table by table. On two real-world databases, its competency-question quality scores rise from roughly 1.3–1.6 in baseline settings to approximately 4.2–4.6 under the full iterative RAG pipeline, showing that ontology generation itself can be the primary RAG target rather than merely a retrieval scaffold for QA (Nayyeri et al., 2 Jun 2025).

Further extensions include a Socratic RAG agent that maps free-text research-topic descriptions to Knowledge Organization Systems and then projects them into bibliometric graphs, and an ontology-guided GraphRAG architecture that uses predefined schema, multi-dimensional community clustering, and dual-channel fusion for multi-hop reasoning. These systems suggest that OntoRAG has become a general design pattern in which ontology, schema, taxonomy, or KOS structure mediates retrieval, disambiguation, and reasoning rather than serving only as offline background knowledge (Lefton et al., 20 Feb 2025, Wang et al., 26 Mar 2026).

7. Limitations, trade-offs, and future directions

OntoRAG’s reported strengths are automated ontology construction, preservation of ontological integrity, improved global sensemaking, and a reusable knowledge structure. Its limitations are also explicit. For a 1M-token corpus, indexing and ontology creation took approximately 300 minutes on a virtual machine with 16 GB RAM and an Intel Xeon Platinum 8171M processor, slightly heavier than GraphRAG indexing at approximately 281 minutes. The pipeline relies on multiple LLM passes for extraction, canonicalization, and property synthesis, in addition to clustering and community detection (Tiwari et al., 31 May 2025).

The system is also domain-dependent. Entity extraction, ontology creation, and question generation rely on domain-specific prompts, and porting to a new domain requires re-engineering prompts and possibly thresholds. Automatically derived ontologies can contain noisy or inconsistent atomic facts, and clustering thresholds can produce spurious merges or splits. Human supervision remains necessary, particularly in tuning the number of communities in Leiden and in avoiding pathological fragmentation or over-merging (Tiwari et al., 31 May 2025).

A substantive trade-off appears in directness. OntoRAG’s answers are typically more comprehensive and diverse, but sometimes less concise and direct than those produced by vector RAG. The paper’s future directions therefore include hybrid retrieval that blends ontology-level retrieval with local vector retrieval, more efficient clustering and indexing strategies, more domain-agnostic prompts, incremental ontology updates for frequent corpus changes, and improved production readiness through lower runtime and smaller computational footprint. A plausible implication is that later OntoRAG systems may converge toward mixed architectures in which ontology-aware structure provides global organization while chunk-level retrieval preserves answer specificity and brevity (Tiwari et al., 31 May 2025, Cruz et al., 8 Nov 2025).