Framework of Thoughts (FoT)

• Framework of Thoughts (FoT) is a dynamic meta-framework that modularizes reasoning by separating structure from execution in large language models.
• It employs evolving execution graphs, concurrent operations, caching, and hyperparameter optimization to enhance reasoning speed and efficiency.
• Empirical evaluations show FoT significantly accelerates processing and reduces computational costs across diverse reasoning tasks.

Framework of Thoughts (FoT) is a foundation framework for building, executing, and optimizing dynamic reasoning schemes for LLMs. Its defining move is to separate reasoning structure from execution mechanics, so that chain-, tree-, and graph-based prompting methods can be represented as programmable operations over evolving graphs rather than as static, task-specific prompt templates. In this formulation, FoT is not itself another reasoning prompt; it is a modular, open framework with built-in hyperparameter tuning, prompt optimization, parallel execution, and intelligent caching, intended to make dynamic reasoning schemes faster, cheaper, and more optimizable by default (Fricke et al., 18 Feb 2026).

1. Motivation and conceptual scope

FoT is introduced against three recurring limitations in prior reasoning schemes such as Chain of Thought (CoT), Tree of Thoughts (ToT), and Graph of Thoughts (GoT). First, existing schemes often rely on static, manually defined structures: CoT is essentially linear and prompt-driven, while ToT and GoT add branching or graph structure but usually still depend on structures specified ahead of time. The paper argues that this hinders generalization to dynamic or unseen problem types, especially when the ideal reasoning structure is not known a priori and must be discovered during execution. Second, many schemes are under-optimized: prompts may be suboptimal, hyperparameters poorly chosen, and exhaustive optimization too expensive to perform thoroughly. Third, extended test-time reasoning is often inefficient, because many methods execute sequentially, repeat identical LLM calls, and incur unnecessary waiting and inference costs (Fricke et al., 18 Feb 2026).

FoT therefore presents its contribution as infrastructural rather than heuristic. Its main advance is not a new reasoning policy analogous to CoT or ToT, but a meta-framework that makes such policies dynamic rather than static, parallelizable rather than sequential, cache-aware rather than repetitive, and optimizable rather than hand-tuned (Fricke et al., 18 Feb 2026).

A related theoretical perspective comes from work on why few-shot CoT can produce coherent intermediate reasoning at all. That account models CoT prompting as helping an LLM infer a latent reasoning context, so that the generated chain becomes closer to a true contextual distribution (Tutunov et al., 2023). This suggests that FoT’s emphasis on explicit intermediate thoughts is compatible with a latent-structure view of reasoning: structured traces are not merely formatting artifacts, but a control surface for steering inference.

2. Formal model: thoughts, operations, and dual graphs

FoT defines a thought as any unit of information in a universe of discourse $\mathcal{D}$, and an operation as any executable function, including an LLM call, retrieval, code execution, tool use, or other external processing. The term graph is used broadly to include chains, trees, and general graphs (Fricke et al., 18 Feb 2026).

The core formal distinction is between the execution graph and the reasoning graph. At step $i$, the execution graph is

$G_i^X = (O_i, E_i^X, s_i^X, t_i^X, \pi_i)$

where $O_i$ is the set of operations, $E_i^X$ the execution edges, $s_i^X$ and $t_i^X$ the source and target maps, and $\pi_i$ maps each edge either to the thought produced on it or to $\bot$ if execution has not yet produced an output. This graph captures the full computational history, including completed operations, currently executing operations, and planned future operations (Fricke et al., 18 Feb 2026).

The derived reasoning graph is

$G_i^R = (T_i, E_i^R, r_i)$

with

$i$0

where $i$1 contains all thoughts produced so far, $i$2 records dependencies among thoughts, and $i$3 maps each thought to the operation that generated it. The reasoning graph is thus a simplified dependency view over produced thoughts, whereas the execution graph is the richer operational object (Fricke et al., 18 Feb 2026).

Operations are themselves graph-transforming functions:

$i$4

An operation may consume thoughts, output thoughts, and also modify the execution graph itself. Graph evolution is written as

$i$5

$i$6

This capacity to add or remove operations and edges is the paper’s formal mechanism for dynamic reasoning. The user supplies at least an initial execution graph $i$7 with at least one operation, after which later structure may be derived during runtime (Fricke et al., 18 Feb 2026).

3. Execution architecture and optimization mechanisms

FoT’s system architecture centers on a Scheduler and a Controller. The Scheduler determines which operations are ready and safe to run; the Controller executes those operations either sequentially or in parallel. Because operations may modify the graph while other operations are running, the paper introduces dynamic execution constraints to avoid race conditions (Fricke et al., 18 Feb 2026).

These constraints are expressed through the sets of ancestors, descendants, and exclusive descendants of an operation. Operationally, an operation may see only its ancestors, itself, and descendants; it may not modify ancestors or non-exclusive descendants; it may modify exclusive descendants; and it may add new edges from ancestors to exclusive descendants or reroute connections within allowed regions. The point of these rules is to preserve logical consistency while still enabling concurrency (Fricke et al., 18 Feb 2026).

Optimization is built into the framework rather than treated as an external afterthought. For hyperparameter optimization, FoT integrates Optuna with a Tree-structured Parzen Estimator (TPE) strategy. The search space can include prompt variants, parser choices, graph-modification decisions, initial graph structure, search strategy such as BFS or DFS, and allowed concurrency. For prompt optimization, FoT integrates DSPy through Cooperative Prompt Optimization (COPRO), which iteratively generates prompt variants, selects the best ones, and refines them further. The optimization objective may combine accuracy or task score, runtime, token count, inference cost, or any weighted combination, and the paper explicitly notes that such objectives are often noisy black-box functions, making gradient-based optimization unsuitable (Fricke et al., 18 Feb 2026).

FoT also includes two caching layers. The Process Cache is temporary within a single execution or sample, while the Persistent Cache is reusable across samples or problems. Because many reasoning schemes repeatedly invoke identical subroutines across branches and instances, caching is presented as essential not only for runtime efficiency but also for making prompt and hyperparameter optimization tractable (Fricke et al., 18 Feb 2026).

4. Chains, trees, and graphs as executable reasoning structures

FoT is designed to support all three topology classes. A chain is a simple sequence of operations, and CoT-like prompting fits this case naturally. A tree arises when an operation creates multiple child operations, as in ToT-style proposal/evaluation/filter cycles, hierarchical decomposition, or recursive subproblem solving. A graph is the most general case, with multiple parents or children and the possibility of revising dependencies over time. FoT explicitly allows adding and removing operations and dependencies during execution, which makes adaptive graph reasoning possible (Fricke et al., 18 Feb 2026).

This places FoT in direct dialogue with GoT. GoT models units of information as vertices in an arbitrary directed graph and defines transformations such as generation, aggregation, and refinement or feedback loops over that graph (Besta et al., 2023). FoT can be read as shifting the emphasis from a graph-shaped prompting abstraction to a more general execution substrate in which such graph transformations are operations within a dynamic runtime. A common misconception is therefore to treat FoT as one more prompting topology alongside chains, trees, and graphs. The paper’s formulation is narrower and more infrastructural: FoT is the framework in which these topologies are expressed, executed, optimized, and accelerated (Fricke et al., 18 Feb 2026).

5. Empirical evaluation and reported benefits

The empirical study reimplements ToT, GoT, and ProbTree within FoT and evaluates them on five tasks: Game of 24 (Go24) for ToT, Sorting for ToT and GoT, Document Merging (DM) for GoT, HotpotQA for ProbTree, and MuSiQue for ProbTree (Fricke et al., 18 Feb 2026).

Across combinations of sequential versus parallel execution and no cache versus process cache versus persistent cache, reported runtime acceleration ranges from 1.9x to 35.4x. With parallel execution and persistent caching, the reported average acceleration is 10.7x. Caching reduces costs on Go24, DM, and MuSiQue by about 14–46% on average, while Sorting and HotpotQA show little or no caching benefit in those runs (Fricke et al., 18 Feb 2026).

Optimization also improves task scores while lowering optimized cost:

Scheme and task	Reported change	Metric
ToT on Go24	63.0% to 66.0%	accuracy
ToT on Sorting	18.4 to 18.2	mistakes
GoT on Sorting	12.7 to 12.1	mistakes
GoT on DM	8.4 to 8.8	F1

The optimization process itself is expensive, but the paper reports that persistent caching reduces total optimization cost to 9–36% of the no-cache cost, depending on task, and reduces optimization duration by up to 8.7x–50.2x. The central empirical conclusion is that reasoning optimization is often only practical when combined with parallel execution and caching (Fricke et al., 18 Feb 2026).

The appendix further reports HotpotQA F1 = 53.8% and MuSiQue F1 = 24.7% for the FoT-based ProbTree implementation, demonstrating that ProbTree can also be expressed within the same framework (Fricke et al., 18 Feb 2026).

6. Position within the broader “thought-structured” literature

The phrase and acronym FoT are not unique in recent literature, and distinguishing them is important. Framework of Thoughts is the 2026 foundation framework discussed here (Fricke et al., 18 Feb 2026). Forest-of-Thought uses multiple reasoning trees, sparse activation, dynamic self-correction, and Consensus-Guided Expert Decision as a test-time compute scaling method (Bi et al., 2024). Federation over Text uses the same acronym for a federated-learning-like system in which agents share reasoning traces to build a cross-task insight library rather than a dynamic execution graph (Yao et al., 18 Apr 2026).

FoT also sits among a wider family of thought-structured methods that differ in what they structure. Graph of Thoughts structures dependencies among thoughts as arbitrary graphs (Besta et al., 2023). HGOT builds a hierarchical DAG of subqueries for retrieval-augmented factuality evaluation and couples thought quality to citation-aware passage scoring (Fang et al., 2024). SR-FoT imposes a multi-stage syllogistic pipeline centered on major premise, minor premise, and final deduction for knowledge-based reasoning (Wan et al., 20 Jan 2025). “Table as Thought” organizes the content within each thought step through a tabular schema of rows and columns rather than only structuring transitions between thoughts (Sun et al., 4 Jan 2025). Dual Engines of Thoughts targets open-ended analysis by separating breadth exploration from depth investigation (Yu et al., 10 Apr 2025). Domain-specific descendants extend the same general orientation: CoFFT couples reasoning and visual focus for VLMs (Zhang et al., 26 Sep 2025), while Protein Thoughts reframes protein-protein interaction discovery as interpretable Tree-of-Thoughts-style search with explicit biochemical signals (Yeon et al., 19 May 2026).

Taken together, these works suggest that “framework of thoughts” has become a broader research direction concerned with explicit intermediate structure, decomposition, and control over reasoning trajectories. Within that landscape, the distinctive contribution of Framework of Thoughts is its formalization of dynamic reasoning as operations over an evolving execution graph, together with concurrency constraints, integrated optimization, and multi-layer caching as a reusable foundation layer (Fricke et al., 18 Feb 2026).