---
title: 'Graph-of-Thoughts: A New Reasoning Paradigm'
url: https://www.emergentmind.com/topics/graph-of-thoughts-got-4b78edd3-5791-45f5-81f7-74eb602e13fc
type: topic
---

# Graph-of-Thoughts: A New Reasoning Paradigm

Graph-of-Thoughts (GoT) represents a paradigm shift in language model reasoning, extending earlier prompting schemes such as Chain-of-Thought (CoT) and Tree-of-Thoughts (ToT) by allowing the intermediate computations of a large language model (LLM) to be organized as a flexible, explicit, directed graph. Rather than forcing solutions through linear or strictly hierarchical pathways, GoT enables the model to branch, merge, aggregate, and recursively refine its thoughts, mirroring the non-linear and recurrent nature of human and algorithmic reasoning processes. This graph-centric approach has been realized in a diverse set of domains—from algebraic problem solving and multi-hop retrieval to multimodal abstraction generation, autonomous driving, chart question answering, recommendation, reward engineering, and interactive assistants—consistently demonstrating gains in solution quality, cost efficiency, interpretability, and knowledge integration.

## 1. Formal Foundations and Topology

GoT formalizes reasoning as a directed graph $G = (V, E)$:
- **Vertices $V$:** Each node $v \in V$ encodes a “thought”—an intermediate reasoning state, partial solution, extracted fact, or subgoal, expressed as a text payload $\tau(v) \in \Sigma^*$ or embedding $c(v) \in \mathbb{R}^d$ [2308.09687, 2401.14295].
- **Edges $E$:** Directed edges $(u,v) \in E$ indicate dependencies, i.e., which thoughts inform the generation of others. Weights or labels $w(u \to v)$ may encode transformation type or confidence [2308.09687].
- **Aggregation $\alpha(\cdot)$:** Enables merging multiple upstream thoughts into new synthesis nodes, supporting dynamic programming–style reuse and data fusion.
- **Scoring and Selection:** Each node can carry a numerical score $s(v)$ evaluating its promise or correctness.

GoT generalizes CoT (a path) and ToT (a tree) to arbitrary graphs, supporting both branching (out-degree $>1$) and merging (in-degree $>1$). Self-loops encode feedback or refinement [2308.09687, 2401.14295, 2404.04538].

## 2. Reasoning Algorithms and Construction

GoT frameworks realize computation through an iterative transformation and search process:
- **Initialization:** The graph starts from the root node representing the original input or query.
- **Expansion:** For each “active” node, one or more child thoughts are generated by the LLM using prompt templates representing operations such as “Generate,” “Merge,” “Refine,” or domain-specific operators (e.g., “locate,” “sum,” “compare”) [2308.09687, 2409.02611].
- **Aggregation and Merging:** When multiple subpaths reach comparable states, aggregation nodes synthesize their contents via a learned function or direct LLM instruction [2401.14295].
- **Scoring and Pruning:** Child nodes are evaluated and ranked. Beam search or heuristic-guided traversal may retain only high-scoring thoughts to cap computation [2308.09687].
- **Termination:** The process halts when a solution node is reached, either by depth, convergence, evaluator score, or explicit halting conditions.

Algorithmic strategies include breadth-first, depth-first, or best-first search across the evolving graph. Adaptive GoT (AGoT) and Dynamic GoT (DGoT) frameworks apply data-driven per-instance criteria for node expansion and stopping, optimizing efficiency and allocating computation where most needed [2502.05078, 2403.17491].

## 3. Domain-Specific Extensions and Architectures

GoT has been extended across multiple research domains, each with tailored graph structures and integration patterns:

- **Hierarchical Graph of Thoughts (HGOT):** Multilayered graphs for retrieval-augmented factuality, decomposing complex queries into subquestions, integrating citation-aware voting and evidence scoring [2402.09390].
- **Compositional Reasoning in Multimodal Tasks:** Operator-typed nodes (localization, numerical, logical) drive transformer-based reasoning over charts or visual inputs [2409.02611].
- **Multimodal Aggregation:** Each reasoning step embeds a subgraph of weighted meta-prompts, with adaptive gating and aggregation (AGoT), interfacing with visual models for robust text-image alignment and VQA [2404.04538].
- **Cooperative Autonomous Driving (V2V-GoT):** LLM-structured graphs orchestrate perceptual fusion, occlusion-aware sensing, and planning-aware prediction among connected autonomous vehicles [2509.18053].
- **Knowledge Graph of Thoughts (KGoT):** Persistent, tool-enhanced knowledge graphs integrate external reasoning steps, web retrieval, and real-time code execution for cost-effective assistant agents [2504.02670].
- **Reward Evolution (RE-GoT):** Bi-level graphs decompose RL tasks into text-attributed nodes and edges, with iterative LLM/VLM-driven reward synthesis and rollout refinement [2509.16136].
- **Sequential Recommendation (GOT4Rec):** Parallel GoT subgraphs model short-term, long-term, and collaborative behaviors, with LLM-driven generation and aggregation for improved user-oriented predictions [2411.14922].

## 4. Comparative Analysis: GoT vs CoT and ToT

GoT subsumes CoT and ToT under a spectrum of expressive power:
- **Latency and Volume:** GoT achieves latency $O(\log_k N)$ and full volume $N$ (number of solution-influencing nodes), combining fast convergence and maximal context reuse [2308.09687, 2401.14295].
- **Adaptive Computation:** Dynamic and hierarchical GoT variants concentrate computation on complex subproblems and avoid redundant work on simple paths, scaling efficiently across problem difficulty [2502.05078, 2403.17491].
- **Aggregation and Feedback:** GoT supports cross-path merging, dynamic programming, and recurrent feedback, which are not possible in tree-based reasoning [2401.14295, 2308.09687].
- **Cost Trade-offs:** Benchmarking shows consistent reductions in error and inference cost across sorting, intersection, keyword counting, multimodal retrieval, and recommendation [2308.09687, 2411.14922].

| Scheme         | Latency      | Volume   | Aggregation | Adaptive Expansion |
|---------------|-------------|----------|-------------|-------------------|
| CoT           | $N$         | $N$      | No          | No                |
| ToT           | $\log_k N$  | $O(\log_k N)$ | No      | No                |
| GoT           | $\log_k N$  | $N$      | Yes         | Yes (DGoT/AGoT)   |

## 5. Empirical Benchmarks and Quantitative Results

GoT frameworks demonstrate substantial improvements across diverse benchmarks:
- **Sorting & Logical Reasoning:** Error reductions up to 62%, cost savings up to 31% versus ToT; accuracy boosts of 89.7%, 86%, and 56% over direct prompting for complex games and algebraic problems [2308.09687, 2308.08614].
- **Retrieval-Augmented QA:** HGOT outperforms Retrieve-then-Read, Self-Ask, and DSP baselines on FEVER (+7%), Open-SQuAD (+1.6%), HotPotQA (+3.4%), with ablation studies confirming importance of hierarchical planning and citation-aware weighting [2402.09390].
- **Multimodal Reasoning:** Stagewise GoT yields multi-point gains in VQA and image-text retrieval tasks, outperforming chain-based soft prompting baselines, with improved domain generalization [2404.04538].
- **Cooperative Autonomous Driving:** GoT reduces 3s-waypoint average $\ell_2$ trajectory error by 3.4m and collision rate by 1.5pp, outperforming geometric and flat-LLM fusion [2509.18053].
- **Recommendation:** GOT4Rec improves HR@K and NDCG@K by 15–75% versus sequential and chain-based models [2411.14922].
- **RL Reward Engineering:** RE-GoT boosts RoboGen and ManiSkill2 task success rates by 32.25% and 93.73%, respectively, exceeding expert and prior LLM-based reward designs [2509.16136].
- **Cost-Effectiveness:** DGoT achieves the best ROUGE-1 (0.358) for scientific abstract generation at only 43–56% the cost of fixed graph approaches [2403.17491]. KGoT reduces AI assistant cost per task by up to 36$\times$ vs. GPT-4 [2504.02670].

## 6. Extensibility, Open Challenges, and Future Directions

GoT is architecturally modular and extensible:
- **Auto-Learned Graphs:** Future GoT frameworks may synthesize optimal graph blueprints via meta-prompting or reinforcement learning [2401.14295].
- **Integration with GNNs:** Graph neural network layers facilitate more efficient evaluation and ranking over thought graphs [2305.16582].
- **Persistent Graph Memory:** Knowledge Graph instantiations extend GoT out of the LLM context, enabling scalable, multi-agent, hybrid retrieval [2504.02670].
- **Dynamic Cost Control:** DGoT and AGoT approaches dynamically bound graph expansion, trading compute for quality per-instance [2403.17491, 2502.05078].
- **Multi-Agent Coordination:** Meta-level coordination among multiple LLM instances updating a shared graph opens new research in decentralized reasoning [2504.02670].
- **Theory and Scaling:** Analysis of graph topology, latency, volume, and cost scaling informs the principled design of future GoT systems [2401.14295].

Challenges include prompt-length constraints, optimal scheduling, adaptive graph derivation, distributed-memory support, robust tool integration, and subgraph prediction in persistent knowledge graphs.

## 7. Synthesis and Impact

Graph-of-Thoughts advances the expressivity, interpretability, and robustness of structured LLM reasoning. By treating intermediate computations as nodes and their dependencies as edges within a graph, GoT frameworks support parallel, modular, and recurrent reasoning patterns, enable aggregation and reuse of sub-results, systematically reduce cost and error rates in diverse applications, and provide blueprints for combining LLMs with external tools, multimodal models, and advanced retrieval systems. The empirical and theoretical results position GoT as a foundational structure in contemporary prompting research, with continuing generalizations in hierarchical, persistent, adaptive, and multi-agent forms across natural language processing, vision, autonomous systems, knowledge engineering, and reinforcement learning [2308.09687, 2401.14295, 2402.09390, 2404.04538, 2409.02611, 2411.14922, 2502.05078, 2504.02670, 2509.18053, 2509.16136].

Source: https://www.emergentmind.com/topics/graph-of-thoughts-got-4b78edd3-5791-45f5-81f7-74eb602e13fc