---
title: Framework of Thoughts (FoT)
url: https://www.emergentmind.com/topics/framework-of-thoughts-fot
type: topic
---

# Framework of Thoughts (FoT)

Framework of Thoughts (FoT) is a foundation framework for building, executing, and optimizing dynamic reasoning schemes for large language models. Its defining move is to separate reasoning structure from execution mechanics, so that chain-, tree-, and graph-based prompting methods can be represented as programmable operations over evolving graphs rather than as static, task-specific prompt templates. In this formulation, FoT is not itself another reasoning prompt; it is a modular, open framework with built-in hyperparameter tuning, prompt optimization, parallel execution, and intelligent caching, intended to make dynamic reasoning schemes faster, cheaper, and more optimizable by default [2602.16512].

## 1. Motivation and conceptual scope

FoT is introduced against three recurring limitations in prior reasoning schemes such as Chain of Thought (CoT), Tree of Thoughts (ToT), and Graph of Thoughts (GoT). First, existing schemes often rely on static, manually defined structures: CoT is essentially linear and prompt-driven, while ToT and GoT add branching or graph structure but usually still depend on structures specified ahead of time. The paper argues that this hinders generalization to dynamic or unseen problem types, especially when the ideal reasoning structure is not known a priori and must be discovered during execution. Second, many schemes are under-optimized: prompts may be suboptimal, hyperparameters poorly chosen, and exhaustive optimization too expensive to perform thoroughly. Third, extended test-time reasoning is often inefficient, because many methods execute sequentially, repeat identical LLM calls, and incur unnecessary waiting and inference costs [2602.16512].

FoT therefore presents its contribution as infrastructural rather than heuristic. Its main advance is not a new reasoning policy analogous to CoT or ToT, but a meta-framework that makes such policies dynamic rather than static, parallelizable rather than sequential, cache-aware rather than repetitive, and optimizable rather than hand-tuned [2602.16512].

A related theoretical perspective comes from work on why few-shot CoT can produce coherent intermediate reasoning at all. That account models CoT prompting as helping an LLM infer a latent reasoning context, so that the generated chain becomes closer to a true contextual distribution [2310.13571]. This suggests that FoT’s emphasis on explicit intermediate thoughts is compatible with a latent-structure view of reasoning: structured traces are not merely formatting artifacts, but a control surface for steering inference.

## 2. Formal model: thoughts, operations, and dual graphs

FoT defines a **thought** as any unit of information in a universe of discourse $\mathcal{D}$, and an **operation** as any executable function, including an LLM call, retrieval, code execution, tool use, or other external processing. The term **graph** is used broadly to include chains, trees, and general graphs [2602.16512].

The core formal distinction is between the **execution graph** and the **reasoning graph**. At step $i$, the execution graph is

$$
G_i^X = (O_i, E_i^X, s_i^X, t_i^X, \pi_i)
$$

where $O_i$ is the set of operations, $E_i^X$ the execution edges, $s_i^X$ and $t_i^X$ the source and target maps, and $\pi_i$ maps each edge either to the thought produced on it or to $\bot$ if execution has not yet produced an output. This graph captures the full computational history, including completed operations, currently executing operations, and planned future operations [2602.16512].

The derived reasoning graph is

$$
G_i^R = (T_i, E_i^R, r_i)
$$

with

$$
T_i = \{ \pi_i(e) \mid e\in E_i^X, \pi_i(e) \neq \bot \},
$$

where $T_i$ contains all thoughts produced so far, $E_i^R$ records dependencies among thoughts, and $r_i$ maps each thought to the operation that generated it. The reasoning graph is thus a simplified dependency view over produced thoughts, whereas the execution graph is the richer operational object [2602.16512].

Operations are themselves graph-transforming functions:

$$
o: \mathcal{D}^* \times G^X \to \mathcal{D}^* \times G^X.
$$

An operation may consume thoughts, output thoughts, and also modify the execution graph itself. Graph evolution is written as

$$
O_{i+1} = (O_i \cup O_i^+) \setminus O_i^-,
$$

$$
E_{i+1}^X = (E_i^X \cup E_i^+) \setminus E_i^-.
$$

This capacity to add or remove operations and edges is the paper’s formal mechanism for dynamic reasoning. The user supplies at least an initial execution graph $G_0^X$ with at least one operation, after which later structure may be derived during runtime [2602.16512].

## 3. Execution architecture and optimization mechanisms

FoT’s system architecture centers on a **Scheduler** and a **Controller**. The Scheduler determines which operations are ready and safe to run; the Controller executes those operations either sequentially or in parallel. Because operations may modify the graph while other operations are running, the paper introduces dynamic execution constraints to avoid race conditions [2602.16512].

These constraints are expressed through the sets of **ancestors**, **descendants**, and **exclusive descendants** of an operation. Operationally, an operation may see only its ancestors, itself, and descendants; it may not modify ancestors or non-exclusive descendants; it may modify exclusive descendants; and it may add new edges from ancestors to exclusive descendants or reroute connections within allowed regions. The point of these rules is to preserve logical consistency while still enabling concurrency [2602.16512].

Optimization is built into the framework rather than treated as an external afterthought. For hyperparameter optimization, FoT integrates **Optuna** with a **Tree-structured Parzen Estimator (TPE)** strategy. The search space can include prompt variants, parser choices, graph-modification decisions, initial graph structure, search strategy such as BFS or DFS, and allowed concurrency. For prompt optimization, FoT integrates **DSPy** through **Cooperative Prompt Optimization (COPRO)**, which iteratively generates prompt variants, selects the best ones, and refines them further. The optimization objective may combine accuracy or task score, runtime, token count, inference cost, or any weighted combination, and the paper explicitly notes that such objectives are often noisy black-box functions, making gradient-based optimization unsuitable [2602.16512].

FoT also includes two caching layers. The **Process Cache** is temporary within a single execution or sample, while the **Persistent Cache** is reusable across samples or problems. Because many reasoning schemes repeatedly invoke identical subroutines across branches and instances, caching is presented as essential not only for runtime efficiency but also for making prompt and hyperparameter optimization tractable [2602.16512].

## 4. Chains, trees, and graphs as executable reasoning structures

FoT is designed to support all three topology classes. A **chain** is a simple sequence of operations, and CoT-like prompting fits this case naturally. A **tree** arises when an operation creates multiple child operations, as in ToT-style proposal/evaluation/filter cycles, hierarchical decomposition, or recursive subproblem solving. A **graph** is the most general case, with multiple parents or children and the possibility of revising dependencies over time. FoT explicitly allows adding and removing operations and dependencies during execution, which makes adaptive graph reasoning possible [2602.16512].

This places FoT in direct dialogue with GoT. GoT models units of information as vertices in an arbitrary directed graph and defines transformations such as generation, aggregation, and refinement or feedback loops over that graph [2308.09687]. FoT can be read as shifting the emphasis from a graph-shaped prompting abstraction to a more general execution substrate in which such graph transformations are operations within a dynamic runtime. A common misconception is therefore to treat FoT as one more prompting topology alongside chains, trees, and graphs. The paper’s formulation is narrower and more infrastructural: FoT is the framework in which these topologies are expressed, executed, optimized, and accelerated [2602.16512].

## 5. Empirical evaluation and reported benefits

The empirical study reimplements **ToT**, **GoT**, and **ProbTree** within FoT and evaluates them on five tasks: **Game of 24 (Go24)** for ToT, **Sorting** for ToT and GoT, **Document Merging (DM)** for GoT, **HotpotQA** for ProbTree, and **MuSiQue** for ProbTree [2602.16512].

Across combinations of sequential versus parallel execution and no cache versus process cache versus persistent cache, reported runtime acceleration ranges from **1.9x to 35.4x**. With parallel execution and persistent caching, the reported **average acceleration** is **10.7x**. Caching reduces costs on **Go24**, **DM**, and **MuSiQue** by about **14–46%** on average, while **Sorting** and **HotpotQA** show little or no caching benefit in those runs [2602.16512].

Optimization also improves task scores while lowering optimized cost:

| Scheme and task | Reported change | Metric |
|---|---:|---|
| ToT on Go24 | 63.0% to 66.0% | accuracy |
| ToT on Sorting | 18.4 to 18.2 | mistakes |
| GoT on Sorting | 12.7 to 12.1 | mistakes |
| GoT on DM | 8.4 to 8.8 | F1 |

The optimization process itself is expensive, but the paper reports that persistent caching reduces total optimization cost to **9–36%** of the no-cache cost, depending on task, and reduces optimization duration by up to **8.7x–50.2x**. The central empirical conclusion is that reasoning optimization is often only practical when combined with parallel execution and caching [2602.16512].

The appendix further reports **HotpotQA F1 = 53.8%** and **MuSiQue F1 = 24.7%** for the FoT-based ProbTree implementation, demonstrating that ProbTree can also be expressed within the same framework [2602.16512].

## 6. Position within the broader “thought-structured” literature

The phrase and acronym **FoT** are not unique in recent literature, and distinguishing them is important. **Framework of Thoughts** is the 2026 foundation framework discussed here [2602.16512]. **Forest-of-Thought** uses multiple reasoning trees, sparse activation, dynamic self-correction, and Consensus-Guided Expert Decision as a test-time compute scaling method [2412.09078]. **Federation over Text** uses the same acronym for a federated-learning-like system in which agents share reasoning traces to build a cross-task insight library rather than a dynamic execution graph [2604.16778].

FoT also sits among a wider family of thought-structured methods that differ in what they structure. **Graph of Thoughts** structures dependencies among thoughts as arbitrary graphs [2308.09687]. **HGOT** builds a hierarchical DAG of subqueries for retrieval-augmented factuality evaluation and couples thought quality to citation-aware passage scoring [2402.09390]. **SR-FoT** imposes a multi-stage syllogistic pipeline centered on major premise, minor premise, and final deduction for knowledge-based reasoning [2501.11599]. **“Table as Thought”** organizes the content within each thought step through a tabular schema of rows and columns rather than only structuring transitions between thoughts [2501.02152]. **Dual Engines of Thoughts** targets open-ended analysis by separating breadth exploration from depth investigation [2504.07872]. Domain-specific descendants extend the same general orientation: **CoFFT** couples reasoning and visual focus for VLMs [2509.22010], while **Protein Thoughts** reframes protein-protein interaction discovery as interpretable Tree-of-Thoughts-style search with explicit biochemical signals [2605.21522].

Taken together, these works suggest that “framework of thoughts” has become a broader research direction concerned with explicit intermediate structure, decomposition, and control over reasoning trajectories. Within that landscape, the distinctive contribution of Framework of Thoughts is its formalization of dynamic reasoning as operations over an evolving execution graph, together with concurrency constraints, integrated optimization, and multi-layer caching as a reusable foundation layer [2602.16512].

Source: https://www.emergentmind.com/topics/framework-of-thoughts-fot