KG-Augmented Executable CoT Framework

• KG-Augmented Executable Chain-of-Thought is a framework that integrates knowledge graphs, RAG, and executable code to improve the reliability of multi-step reasoning.
• It decomposes complex tasks into structured graphs, enabling effective retrieval of domain-specific context and verifiable execution via pseudo-programs or code.
• Empirical evaluations show notable improvements in mathematical reasoning, code generation, and multi-hop symbolic tasks over traditional chain-of-thought methods.

KG-Augmented Executable Chain-of-Thought (KGA-ECoT) is a reasoning framework that unifies knowledge graph (KG)-guided decomposition, retrieval-augmented generation (RAG), and executable code or pseudo-program prompting to enhance complex multi-step reasoning in LLMs. It addresses two core limitations of standard chain-of-thought (CoT) prompting: (1) the lack of reliability and logical rigor when relying solely on LLM-generated reasoning chains, and (2) the inability to integrate precise, domain-specific knowledge and computational verification. KGA-ECoT operationalizes task decomposition as a structured graph, leverages knowledge-informed retrieval for context augmentation, and generates executable artifacts interpretable as either pseudo-programs or verifiable code. This architecture has established record performance in mathematical reasoning, code generation, and multi-hop symbolic tasks (Li et al., 18 Apr 2025, Chen et al., 6 Aug 2025).

1. Formal Structure: Knowledge Graphs and Task Graphs

KGA-ECoT instances—such as CoT-RAG (Li et al., 18 Apr 2025) and its mathematical coding variant (Chen et al., 6 Aug 2025)—ground the reasoning process in a directed acyclic graph encoding subtasks and their dependencies.

Knowledge Graph Construction

• The system begins with a domain-specific Decision Tree (DT), in practice often expert-provided.
• Each DT node is decomposed by the LLM into KG entities, formally $G = (V, E)$, with each node $v\in V$ specified as a 4-tuple:

$v = (q_v, c_v, d_v, a_v)$

where $q_v$ is a sub-question, $c_v$ a sub-case (example/case specification), $d_v$ a dynamically filled description (context/fact extraction), and $a_v$ the answer slot.

• Edges $(u \rightarrow v) \in E$ encode information flow: $a_u$ is piped as context into $d_v$.

Task Graphs for Mathematical Coding

• For code generation and mathematical reasoning, the Structured Task Graph decomposes a problem $v\in V$0 into a graph $v\in V$1.
• Each node $v\in V$2 represents a subtask; an edge $v\in V$3 means $v\in V$4 depends on $v\in V$5's output.
• The solution construction pipeline includes:
  • Goal Analysis
  • Condition Extraction
  • Problem Decomposition into subtasks
  • Dependency Modeling
  • Task Sorting/topological ordering

This structure establishes a formal, interpretable scaffold guiding the downstream reasoning and execution pipeline (Li et al., 18 Apr 2025, Chen et al., 6 Aug 2025).

2. Retrieval-Augmented Generation (RAG) and GraphRAG

KGA-ECoT incorporates RAG at the subtask/node level. Each node's context is dynamically filled using retrieval mechanisms, ensuring that reasoning steps are both grounded and case-aware.

RAG for Generic Reasoning

• For each node $v\in V$6, a retrieval operator $v\in V$7 fills $v\in V$8 by extracting the most relevant facts/examples for $v\in V$9 from the user input $v = (q_v, c_v, d_v, a_v)$0:

$v = (q_v, c_v, d_v, a_v)$1

• The LLM sees prompts of the form $v = (q_v, c_v, d_v, a_v)$2 at each reasoning step.

GraphRAG for Mathematical Coding

• Augments each subtask with precise library call references using a GraphRAG index.
• The SymPy knowledge graph $v = (q_v, c_v, d_v, a_v)$3 contains catalogue nodes (directories) and callable nodes (functions/classes), connected by category and parent-child edges.
• Encoded via hierarchical embeddings: For node $v = (q_v, c_v, d_v, a_v)$4 with parent $v = (q_v, c_v, d_v, a_v)$5,

$v = (q_v, c_v, d_v, a_v)$6

• Retrieval matches subtask embeddings to callable node embeddings using cosine similarity, returning top-$v = (q_v, c_v, d_v, a_v)$7 candidates to inform code synthesis.

The RAG/GraphRAG mechanism grounds each reasoning or code generation step in accurate, context-specific knowledge.

3. Executable and Pseudo-Program Prompting

To mitigate the brittleness of natural language CoT, KGA-ECoT advances toward executability and verifiability by emitting either pseudo-programs or actual code templates.

Pseudo-Program Prompting

• For symbolic or general tasks (e.g., CoT-RAG), every node $v = (q_v, c_v, d_v, a_v)$8 is transformed into a standardized callable: $q_v$5
• The execution driver loops topologically, invoking LLM for each $v = (q_v, c_v, d_v, a_v)$9, piping answers as prescribed by graph $q_v$0.

Executable Code for Mathematical Reasoning

• For mathematical coding tasks, the system generates Python code (with SymPy or domain-specific libraries).
• Template (for node $q_v$1): $q_v$6
• Execution is conducted in isolation (e.g., Docker container) with a verification loop:
  1. Execute code.
  2. On error, fallback to text-based answer.
  3. Success: extract result_i and propagate.

This duality—pseudo-code for general interpretability and compiled code for measurable verification—anchors both rigor and transparency (Li et al., 18 Apr 2025, Chen et al., 6 Aug 2025).

4. End-to-End Pipeline and Operational Workflow

The canonical KGA-ECoT pipeline comprises three principal stages:

Stage	Key Operations	Output
KG/Task Graph Construction	Expert/DT input, decomposition, subtask and dependency formation	Directed graph $q_v$2
RAG/GraphRAG Augmentation	Node-level retrieval (facts, code APIs, cases) via LLM or library index	Each node $q_v$3 enhanced with $q_v$4 and utility APIs
Execution	Emit and step through pseudo-program/code templates, verify output	Final answer, verified intermediate states

A plausible implication is that this modularity enables straightforward adaptation to new domains by swapping out the knowledge graph base and retrieval index while retaining the logical scaffold and executable driver.

5. Empirical Evaluations and Ablation Insights

KGA-ECoT has demonstrated consistent gains across mathematical and general reasoning tasks.

Main Empirical Results

On arithmetic (GSM8K), symbolic (MATH-500), and commonsense (SVAMP) benchmarks, KGA-ECoT exceeds advanced prompting baselines:

Dataset	DS-7B	DS-14B	L3.2-3B	L3.1-8B
GSM8K	87.49%	92.80%	63.91%	75.97%
MATH-500	89.40%	88.80%	41.10%	46.60%
SVAMP	89.00%	92.33%	79.09%	84.67%

Absolute improvement over strongest baseline: GSM8K (+2.36 pts), MATH-500 (+3.59 pts), SVAMP (+2.37 pts) (Chen et al., 6 Aug 2025).

On general reasoning tasks, CoT-RAG (KGA-ECoT instance) yields accuracy gains of 4–23 points over CoT methods, with up to 15.3 points on complex reasoning chains (≥8 entities). On ERNIE-Speed-128K, accuracy rises from 77.3% to 89.1% (+11.8 pts); on GPT-4o mini, from 94.9% to 99.5% (+4.6 pts) (Li et al., 18 Apr 2025).

Ablation Analysis

Removal studies on (Chen et al., 6 Aug 2025) show:

• Dropping GraphRAG: modest performance loss (up to 4 pts on advanced datasets).

• Dropping code execution: significant degradation (up to 10 pts), highlighting the centrality of external verification.

Additionally, using LLM retrieval for RAG reduces runtime by 14–27% versus vector-store methods and increases accuracy by 9–12% (Li et al., 18 Apr 2025).

6. Generalization and Prospective Extensions

KGA-ECoT’s architecture generalizes across model families (DeepSeek, LLaMA, ERNIE, GPT-4o mini) and reasoning domains (arithmetic, algebra, geometry, coding, commonsense). Its modular structure supports:

• Construction of custom KGs (e.g., NumPy, networkx) or code library graphs for new domains.
• Integration with symbolic theorem provers for formal proof steps.
• Enablement of multi-agent systems via parallel subtask execution and verification.

This suggests that KGA-ECoT is positioned as an extensible platform for interpretable, verifiable reasoning in both general and domain-constrained tasks (Chen et al., 6 Aug 2025).

---

References

• "CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in LLMs" (Li et al., 18 Apr 2025)
• "KG-Augmented Executable CoT for Mathematical Coding" (Chen et al., 6 Aug 2025)