Novelty-based Tree-of-Thought Search for LLM Reasoning and Planning

Abstract: Although advances such as chain-of-thought, tree-of-thought or reinforcement learning have improved the performance of LLMs in reasoning and planning tasks, they are still brittle and have not achieved human-level performance in many domains, and often suffer from high time and token costs. Inspired by the success of width-based search in planning, we explore how the concept of novelty can be transferred to language domains and how it can improve tree-of-thought reasoning. A tree of thoughts relies on building possible "paths" of consecutive ideas or thoughts. These are generated by repeatedly prompting an LLM. In our paper, a measurable concept of novelty is proposed that describes the uniqueness of a new node (thought) in comparison to nodes previously seen in the search tree. Novelty is estimated by prompting an LLM and making use of embedded general knowledge from pre-training. This metric can then be used to prune branches and reduce the scope of the search. Although this method introduces more prompts per state, the overall token cost can be reduced by pruning and reducing the overall tree size. This procedure is tested and compared using several benchmarks in language-based planning and general reasoning.

• The paper introduces binary LLM novelty estimation for pruning Tree-of-Thought search and formalizes action generation, state mapping, verification, and novelty estimation as separate tasks.
• The approach reduced Blocksworld token use from 330,000 to 18,000 on average while solve rates fell only from 47/50 to 42/50, and achieved 48/50 in Logistics at 9,000 tokens versus 69,000 for base ToT.
• The results show that pruning benefits depend heavily on prompt and search configuration, while current models often detect duplicates rather than reason about true width-based novelty, limiting theoretical guarantees and performance on MATH.

Overview

This paper, "Novelty-based Tree-of-Thought Search for LLM Reasoning and Planning" (2605.06040), addresses the prohibitive computational cost of Tree-of-Thought (ToT) reasoning by importing the width-based pruning machinery of classical planning into LLM-driven search. The authors, Hamm and Ajanovic of RWTH Aachen, replace the classical integer novelty measure $w(s)$ with a binary, LLM-estimated novelty query, and use the resulting pruning signal to shrink the search tree explored by ToT. The work makes three contributions: a formalization of LLM planning within ToT as four sub-tasks (action generation, successor state mapping, plan verification, and novelty estimation), an evaluation framework inspired by PlanBench that tests each sub-task against simulator ground truth, and an empirical demonstration that, in well-tuned configurations, novelty pruning reduces token cost dramatically at comparable solve rates.

Motivation and method

The starting point is the width-based planning literature. Iterative Width (IW) prunes any state whose novelty exceeds a threshold $k$, yielding runtime exponential only in the problem's width $w(P)$; since most serialized planning domains have width at most two, IW is highly effective in practice. The paper's hypothesis is that LLM reasoning and planning tasks likewise exhibit low latent width, so novelty pruning should transfer. The obstacle is that LLMs cannot reliably map novelty differences to integer values, and natural-language states lack explicit atoms. The proposed solution is deliberately coarse: the LLM is asked a binary yes/no question — "Is '{new_state}' a novel state compared to this list of states?" — and non-novel states are pruned directly. This removes the need for prescribed output formats or few-shot exemplars; the only domain-specific input is a handwritten context prompt describing the environment rules.

The ToT framework is extended so that each thought is a natural-language state, with the LLM performing action generation (via repeated sampling), successor state mapping, plan verification, and the new novelty estimation. Two tree-construction variants are considered: an explicit action-then-state decomposition (ESA) and a combined direct mapping; both BFS and DFS traversal are evaluated. The authors note a limitation of their own design: unlike IW, no guarantees are provided on solution quality or on a faithful computation of novelty, and the method's success depends on the LLM's internalized notion of salience.

Latent width of reasoning benchmarks

To establish that the approach is applicable beyond formal planning, the authors formalize the Game of 24 as a classical planning problem and compute its width empirically. The result is striking: the benchmark has an average effective width of 1.74, no instance exceeds width 3, and 88.2% of states are pruneable on average, across state trees averaging 3,698 states. This confirms that a benchmark widely used to evaluate LLM reasoning possesses the low-width structure that makes width-based search tractable, providing the theoretical basis for the entire approach. The implication is that many LLM reasoning tasks may be far more tractable under structured pruning than their apparent combinatorial size suggests.

Sub-task evaluation

Each sub-task was evaluated on Qwen3 14B across two axes: prompting style (PDDL-like versus natural language) and the model's built-in thinking mode. Several findings stand out:

• Goal verification is nearly solved: the model scores 47–48 of 50 across all conditions, indicating that verifying a complete plan is well within current capability.
• Novelty pruning succeeds only in the easy variant: the model reaches 50/50 with thinking mode on standard instances, but collapses to 0/50 (1/50 in one condition) on the harder variant where all query states are non-duplicates and must be pruned strictly on width grounds. The authors conclude the model performs good duplicate detection but has poor genuine novelty reasoning — a significant concession, since true width-based pruning requires exactly the latter.
• Enumerating valid actions is harder than choosing one: in thinking mode the model names all valid actions correctly 50/50 times but identifies a single valid next action only 41/50 times, and an optimal one 33/50 times.
• Prompt fragility is severe: a recurring failure mode — confusion between "pick-up" and "unstack" in Blocksworld — was fixed by appending a single targeted example, raising single-action validity from as low as 22/50 to 50/50 in the best condition. The authors report that bad prompts caused complete performance collapse and that their final prompts were themselves found through iterative testing, which undercuts claims of domain-independence.

Complete approach evaluation

The full system was benchmarked against naive prompting and base ToT on Blocksworld, Logistics, and MATH Level 5. The naive baseline is weak: Qwen3 14B solves only 3/50 Blocksworld instances without thinking mode and 23/50 with it. Base ToT without prompt extensions solved zero instances; after prompt tuning, Direct ToT with thinking mode reached 47–49/50, at an average cost of 330k tokens (DFS) or as little as 15–22k (BFS).

The central result concerns the interaction between pruning and configuration quality. In the strongest configuration (DFS, Direct, thinking), novelty pruning cut average token usage from 330k to 18k — an 18-fold reduction — while performance dropped only modestly from 47/50 to 42/50. In Logistics, the same configuration achieved 48/50 with 9k average tokens versus 69k for the base, and required no prompt retuning for the domain change beyond the context description. Conversely, in poorly configured settings the added novelty queries increased cost by over 100% while degrading accuracy, sometimes to near-zero solve rates (e.g., ESA + BFS + normal mode: 5/50 at 1,038k tokens versus 21/50 at 13k for the base).

On MATH Level 5, the picture is more sober. Boolean novelty pruning had negligible impact on both performance and token cost: savings from pruned states were offset by the cost of the novelty queries themselves, and in one DFS configuration cost actually rose from 55k to 128k tokens. The authors attribute this to prompts frozen after Blocksworld tuning, and concede that optimal token efficiency in distinct reasoning domains likely requires domain-specific prompt tuning — a direct tension with the method's domain-agnostic design goal. A further empirical finding contradicts the intuition that decomposing the search into explicit action and state steps helps: ESA generally underperformed Direct generation, plausibly because more LLM queries per tree multiply opportunities for cascading errors. The authors also document "disastrous chain reactions," where one erroneous response degrades subsequent completions until the model breaks output limits — a failure mode inherent to recursive LLM prompting rather than to novelty pruning specifically.

Limitations and open questions

The paper is explicit about its constraints. The binary novelty query is only weakly analogous to the classical width measure: no correlation guarantee of the form $\rho(n(s), w(s)) > 0$ is verified empirically, and the sub-task evaluation shows the estimator degenerates to duplicate detection on the hard variant, meaning the pruning realized in practice may be closer to duplicate elimination than width-based search. Configuration sensitivity is substantial and unpredictable, and the reported prompt-tuning process was itself iterative and manual, so the headline cost reductions are conditional on tuning effort that the paper does not fully systematize. Evaluations use a single 14B open-weights model, with GPT-4o/4o-mini and GPT-5 used only for the naive baseline, leaving the interaction between novelty pruning and larger or stronger reasoning models unexamined. The Game of 24 width analysis covers one benchmark, and the hypothesis that LLM reasoning tasks generally have low latent width remains untested at scale. Open questions the paper leaves include whether goal serialization, alternative novelty estimators, or larger base models can move pruning beyond duplicate detection, and whether prompt-tuning-free configurations can retain token savings on non-planning domains.

Conclusion

The paper demonstrates that LLM-estimated binary novelty can be integrated into Tree-of-Thought search and, under favorable configurations, reduces token cost by up to an order of magnitude or more at near-constant solve rates, while generalizing across planning domains with minimal manual adaptation. The empirical width analysis of Game of 24 substantiates the low-width hypothesis for at least one prominent reasoning benchmark. At the same time, the work is candid that the method's benefits are configuration-dependent, that the LLM novelty estimator largely reduces to duplicate detection, and that no optimality or completeness guarantees survive the transfer from classical width-based planning. The result is best read as evidence that structured, novelty-informed pruning is a viable cost-reduction lever for LLM tree search, with the gap between binary LLM novelty and true width-based pruning as the central open problem.