GridCodex: RAG for Grid Code Compliance

• GridCodex is a retrieval-augmented generation framework for power grid code compliance that integrates dual knowledge bases and multi-stage query refinement.
• It employs a dedicated terminology base and a factual repository to disambiguate queries and enable precise retrieval in complex regulatory texts.
• Empirical evaluation shows up to a 26.4% improvement in answer quality and a ten-fold increase in recall on regulatory Q&A benchmarks.

Searching arXiv for GridCodex and closely related work to ground the article in current literature. GridCodex is a retrieval-augmented generation framework for power grid code reasoning and compliance. It is introduced as an end-to-end system for interpreting grid codes, the regulations governing grid operations, under conditions in which the global shift towards renewable energy has increased regulatory complexity and made manual interpretation costly, inconsistent, and error-prone. Its defining technical features are multi-stage query refinement, dual knowledge bases for terminology and factual regulatory content, and enhanced retrieval with RAPTOR; the reported evaluation shows a 26.4% improvement in answer quality and more than a 10 fold increase in recall rate on regulatory question answering benchmarks (Shi et al., 18 Aug 2025).

1. Definition, scope, and problem setting

GridCodex is presented as the first end-to-end RAG-driven AI framework specifically for grid code reasoning and compliance. The motivating premise is that LLMs excel at general QA but lack sufficient domain-specific knowledge and are prone to hallucinations, whereas fine-tuning requires expensive, hard-to-acquire labeled data and intensive compute. In that setting, GridCodex adopts RAG as a domain-grounding strategy: context retrieved from domain-specific knowledge bases is appended to queries at inference time, improving factuality and grounding without retraining the base model (Shi et al., 18 Aug 2025).

The framework is situated in the electricity industry, where grid codes are lengthy, hierarchical, cross-referential regulatory documents. The paper treats compliance reasoning not as a generic document QA task but as a specialized retrieval-and-synthesis problem requiring terminology normalization, cross-lingual handling, and multi-hop access to long-form regulatory structure. A central misconception that the framework explicitly rejects is that ordinary “vanilla RAG” is sufficient for this domain. GridCodex is instead defined by an optimized RAG workflow in which retrieval quality, query reformulation, and document structure are all treated as first-class components.

2. Knowledge organization and document representation

The knowledge layer is divided into two distinct repositories. The terminology knowledge base contains technical terms, definitions, and translations in JSON or Markdown, while the factual knowledge base contains full regulatory documents, clauses, tables, and figures extracted through OCR, Table Structure Recognition, and Document Layout Recognition (Shi et al., 18 Aug 2025).

This separation is operational rather than merely organizational. Terminology retrieval is used to clarify the intent and vocabulary of the incoming query before factual retrieval is attempted, which is important in a domain with specialized nomenclature and multilingual usage. On the factual side, the system uses adaptive chunking aligned to lowest-level section titles in order to retain semantic integrity and prevent clause cutoffs. That design choice reflects the structure of regulatory texts, where the inferential unit is often a clause, exception, or nested subsection rather than a fixed-length text span.

The paper’s architectural diagram organizes the system into three components: Industry Knowledge, which stores terminology and factual databases after preprocessing and embedding; the RAG Pipeline, which manages chunking, embedding, and RAPTOR-based index construction; and the Q&A Pipeline, which orchestrates query refinement, retrieval, and answer generation. This componentization locates GridCodex between document engineering and inference-time reasoning rather than reducing it to a prompt wrapper around an LLM.

3. Retrieval pipeline and query refinement

The core procedural novelty of GridCodex is a multi-stage query refinement pipeline. For a user query $Q$, the system first retrieves domain definitions, denoted $\mathcal{K}_{term}(Q)$. It then forms a refined query

$Q' = f(Q,\mathcal{K}_{term}(Q)),$

where the refinement function enriches the query with terminology explanations. If necessary, the refined query is translated, after which factual retrieval returns $\mathcal{K}_{fact}(Q')$ for answer synthesis (Shi et al., 18 Aug 2025).

The workflow therefore proceeds in three stages: definition injection, query rewriting and translation, and factual retrieval. The ordering matters. The summaries in the terminology base are not simply appended as background context; they are used to disambiguate the query before the search over factual regulation begins. Because most documents are in English, the translation step is applied after clarification, which is intended to bridge cross-lingual gaps without losing technical precision.

Retrieval itself is enhanced with RAPTOR. Grid codes are described as lengthy and deeply hierarchical, so the system avoids purely flat retrieval over contiguous chunks. RAPTOR recursively clusters semantically related text chunks using Gaussian Mixture Models, summarizes each cluster via general LLMs, and re-embeds those summaries to form a tree-structured index. Query-time retrieval can then operate at varying abstraction levels, from clause to section to document, enabling multi-hop retrieval over hierarchical regulation. The embedding model used for vectorization is Linq-Embed-Mistral, selected for high-precision retrieval.

4. Answer generation and evaluation protocol

The evaluation uses a proprietary HUAWEI grid code dataset containing 148 QA pairs spanning four regulatory authorities: Hong Kong (China), Netherlands, European Union, and Bangladesh. Each QA pair has a ground-truth reference answer approved or curated by domain experts, and the system is compared against two baselines: a general LLM without retrieval and a vanilla RAG pipeline without multi-stage refinement or RAPTOR (Shi et al., 18 Aug 2025).

Assessment is reported on three axes. Answer Quality measures correctness, completeness, and usefulness, with correctness prioritized. Faithfulness measures consistency of generated answers with retrieved context. Recall@30 measures the fraction of queries for which answer-relevant information appears among the top 30 retrieved chunks. The use of LLM-based evaluators together with expert-curated references standardizes scoring across models and jurisdictions, which is particularly relevant in a regulatory setting where answer adequacy is not reducible to lexical overlap.

The evaluation protocol is notable because it treats retrieval as part of the compliance system rather than as a hidden preprocessing step. Recall@30 is elevated to a first-order metric, reflecting the view that regulatory reasoning quality is bounded by evidence coverage. This suggests that GridCodex is best understood as a retrieval-and-reasoning stack whose performance depends on the interaction between indexing, reformulation, and generation.

5. Empirical performance and ablation results

The reported quantitative results show large gains over both the general-LLM and vanilla-RAG baselines. In Hong Kong (China), the optimized RAG configuration reaches Answer Quality 0.946, Faithfulness 0.978, and Recall@30 1.000, compared with 0.759, 0.978, and 0.182 for vanilla RAG and 0.798 Answer Quality for the general LLM. In the Netherlands, the optimized system reaches 0.852, 0.968, and 0.917, compared with 0.519, 0.978, and 0.100 for vanilla RAG and 0.602 for the general LLM. In the European Union benchmark, it reaches 0.843, 0.968, and 0.913, compared with 0.645, 0.941, and 0.100 for vanilla RAG and 0.602 for the general LLM. In Bangladesh, it reaches 0.877, 0.962, and 0.900, compared with 0.805, 0.984, and 0.000 for vanilla RAG and 0.784 for the general LLM. The paper also cites an F1 of 0.972 in the Hong Kong region (Shi et al., 18 Aug 2025).

These results are interpreted in the paper as evidence that GridCodex improves answer quality by up to 26.4%–30% relative to vanilla RAG while maintaining high faithfulness and achieving near-perfect or near-perfect Recall@30 across jurisdictions. The retrieval gains are especially pronounced: the abstract reports more than a 10 fold increase in recall rate, and the regional breakdown shows this concretely in settings such as Hong Kong, where Recall@30 rises from 0.182 to 1.000.

The ablation study isolates the role of base-model selection. A reasoning-enabled Qwen3-235B-A22B model improves answer quality by 11.2% relative to Qwen3-30B-A3B on the Netherlands test, although the summary notes a slight reduction in faithfulness. A reasoning-enabled variant also outperforms an instruction-following variant with reasoning chains disabled by 5.8% in answer quality. The reported interpretation is that high-capacity reasoning models are advantageous for multi-hop reasoning and ambiguity resolution in regulatory documents, but that faithfulness remains a deployment constraint.

6. Significance, limitations, and terminological breadth

Within the power-systems context, GridCodex is significant because it frames compliance interpretation as a domain-specific RAG problem rather than as generic legal QA or pure model fine-tuning. The architecture is described as offering foundations for automated violation detection, simulation input generation, and proactive regulatory monitoring. A plausible implication is that its design pattern—terminology-aware reformulation coupled to hierarchical retrieval—could transfer to other safety- or compliance-critical sectors, although that extension is presented as broader implication rather than as a benchmarked result (Shi et al., 18 Aug 2025).

The framework also has clear boundaries. It is built around retrieved evidence and prompt-grounded synthesis, so it does not claim autonomous statutory interpretation outside its knowledge bases. Its empirical claims are tied to a benchmark of 148 QA pairs across four regulatory authorities, and its reported faithfulness scores, while high, indicate that answer generation remains conditioned by retrieval coverage and model behavior rather than being guaranteed by formal reasoning.

The term “GridCodex” is not unique to the power-grid compliance literature. In an earlier distributed data-management context, xDGDL is described as providing a foundation for a “GridCodex,” understood as a Grid-wide data or document catalog that unifies data content, metadata, and location information (Felder et al., 2016). In higher category theory, a summary of the lax-grid model for $(\infty,n)$-categories characterizes its broader “GridCodex” vision as an explicit, computable theory built from cubical pasting diagrams (Keidar et al., 25 Jun 2026). A summary of work on Grid-like Code Quantization also refers to GCQ as “GridCodex” in a brain-inspired world-modeling context (Peng et al., 16 Oct 2025). This suggests that the label is polysemous across disciplines; in current arXiv usage, however, the most specific and technically developed sense is the RAG-driven framework for power grid code reasoning and compliance (Shi et al., 18 Aug 2025).