STATe-of-Thoughts: Structured Action Templates for Tree-of-Thoughts

Abstract: Inference-Time-Compute (ITC) methods like Best-of-N and Tree-of-Thoughts are meant to produce output candidates that are both high-quality and diverse, but their use of high-temperature sampling often fails to achieve meaningful output diversity. Moreover, existing ITC methods offer limited control over how to perform reasoning, which in turn limits their explainability. We present STATe-of-Thoughts (STATe), an interpretable ITC method that searches over high-level reasoning patterns. STATe replaces stochastic sampling with discrete and interpretable textual interventions: a controller selects actions encoding high-level reasoning choices, a generator produces reasoning steps conditioned on those choices, and an evaluator scores candidates to guide search. This structured approach yields three main advantages. First, action-guided textual interventions produce greater response diversity than temperature-based sampling. Second, in a case study on argument generation, STATe's explicit action sequences capture interpretable features that are highly predictive of output quality. Third, estimating the association between performance and action choices allows us to identify promising yet unexplored regions of the action space and steer generation directly toward them. Together, these results establish STATe as a practical framework for generating high-quality, diverse, and interpretable text. Our framework is available at https://github.com/zbambergerNLP/state-of-thoughts.

• The paper introduces structured action templates that enable discrete, interpretable control over high-level reasoning choices in LLM outputs.
• It employs a modular loop of planning, generation, evaluation, and selection to achieve up to twice as many semantically distinct outputs compared to traditional stochastic methods.
• The framework supports rigorous attribution analysis and targeted generation, resulting in improved argument persuasiveness and traceability.

STATe-of-Thoughts: Structured Action Templates for Tree-of-Thoughts

Introduction and Motivation

The STATe-of-Thoughts (STAT) framework (2602.14265) directly addresses two major shortcomings present in mainstream inference-time compute (ITC) methods for LLMs: insufficient controllability over reasoning strategies and inadequate semantic diversity in generated outputs. Traditional approaches such as Best-of-N sampling, Chain-of-Thought (CoT), and Tree-of-Thoughts (ToT) typically rely on high-temperature stochastic sampling at the token level to induce output variation. However, this mechanism often results in superficial, functionally redundant forms of diversity, while keeping the underlying high-level reasoning patterns homogeneous. Furthermore, these approaches provide minimal interpretability in terms of how specific reasoning choices or rhetorical strategies affect downstream performance (e.g., argument persuasiveness), hampering scientific investigations and control for complex textual tasks.

STAT introduces a new ITC paradigm where branching in the response space is not defined by stochastic temperature but by explicit, discrete, and interpretable textual interventions—structured action templates—that encode high-level reasoning or rhetorical choices at each step. This action-space search paradigm enables (1) systematic control over output features, (2) stronger and more meaningful response diversity, and (3) post-hoc traceability for mechanistic attribution and targeted generation.

Methodology

STAT generalizes the ToT architecture by dividing the generative process into a modular loop of discrete planning, generation, evaluation, and selection operations, each of which can be traced and analyzed. At each search step, the controller module chooses explicit actions from a pre-defined action space (templates encompassing content themes, structural moves, and other linguistic interventions). These actions serve as auditable textual prefixes or internal generation instructions, driving the generator module to produce candidate continuations that are guaranteed to differ across high-level features rather than solely in surface forms.

Figure 1: STAT's workflow for argument generation, where action templates control structural and content-level features, branched outputs are generated through tree search, and branches are evaluated for downstream performance.

Figure 2: Stylized STAT Plan→Generate→Evaluate→Select cycle, with the controller exploring actions, generator expanding states, evaluator scoring, and beam selection retaining the best trajectories.

Candidates are evaluated using LLM-based reward models (process or outcome) or deterministic verifiers. The most promising candidate states are then propagated in the beam for subsequent expansion, supporting both breadth and depth scaling. Importantly, STAT natively supports early stopping by introducing a FINISH action, mitigating the overthinking and context saturation documented in unconstrained-depth tree methods.

The design of the action space is modular and task-adaptable; in persuasive argument generation, action templates can comprise rhetorical structure (discourse relations, e.g., contrast, cause, concession) and topical or thematic content axes, which together enable systematic exploration of the strategy/product space.

Empirical Results

Diversity Evaluation with NoveltyBench

STAT demonstrates substantial gains over existing ITC regimes on the NoveltyBench suite, a rigorous benchmark designed to quantify both response novelty (Mean Distinct) and utility (Mean Utility) across a variety of creative, factual, and subjective tasks. Notably, across four LLM architectures (Qwen3, Nemotron-3, Ministral-3) and three sampling temperature regimes, STAT achieves up to twice as many semantically distinct outputs as any best-of-n, CoT, or ToT baseline, without sacrificing output utility.

This pattern holds even when baselines are enhanced with explicit action space exposure in prompts; controller-based action selection is required to fully realize the gains (Mean Distinct: up to 5.02/9 for STAT vs. 2.44 for standard CoT and 3.36 for action-space CoT at $T = 0.7$ for Qwen3-30B). The findings directly contradict the assumption that temperature-based sampling is sufficient for output space coverage, especially when the goal is to surface reasoning-diverse outputs instead of minor paraphrases.

Attribution and Controllability in Argument Generation

In a controlled study on generating persuasive arguments for banning single-use plastics, STAT supports detailed mechanistic tracing: every final output is associated with a sequenced trajectory of controller action choices. Multiple synthesis regimes are explored: strict (verbatim reasoning step concatenation), faithful (order-preserving paraphrasing), and restructured (free reorganization).

Attribution analyses reveal that the sequence of controller actions (not merely the bag-of-actions) explains significant variance in argument quality, as measured by LLM-judge-based pairwise preference (Bradley-Terry). The full sequential model (with position and transition features; M2) consistently outperforms presence-only models (M1a/b/c), achieving up to $R^2 = 0.57$ test-set predictability under strict synthesis.

Figure 3: The sequential model (M2) yields substantially higher $R^2$ than presence-based models, especially under strict synthesis.

When using the learned M2 model to extrapolate and prioritize unvisited regions of the action trajectory space, STAT can generate previously unseen argument trajectories whose judged persuasiveness matches or exceeds those of the best arguments from observed samples. In length-matched pairwise evaluations, up to 81.2% of targeted arguments outperformed random baselines, and in strict synthesis, 51.7% outperformed the original top 5% arguments by BT rank—demonstrating the practical value of trace-based attribution and targeted control.

Analysis of Length and Synthesis Effects

Argument length is shown to correlate positively with persuasiveness judgments (as expected). Length effects are controlled and isolated in both attribution and targeted-generation analyses, confirming that action-sequence quality is not simply an artifact of longer generations.

Figure 4: Distribution of argument lengths per synthesis mode, with strict and restructured synthesis producing unimodal distributions, while faithful synthesis yields bimodal tendencies.

The coupling between action traces and generated text is strongest under strict synthesis and decreases as the synthesis mechanism becomes more flexible. Predictability (as measured by $R^2$) declines from strict ($0.57$) to faithful ($0.38$) to restructured ($0.37$) synthesis, confirming the claim that traceability and direct attribution rely on output faithfulness to controller action histories.

Implications and Theoretical Contributions

Practical Takeaways

1. Controllability: The STAT framework enables direct, discrete, and interpretable control over generative mechanisms at inference time by branching over action templates rather than relying on token-level randomness.
2. Diversity: Discrete action-space branching systematically increases the functional novelty of response candidates, and outperforms all tested temperature-based or prompt-structured sampling baselines.
3. Attribution and Exploration: The structured trace enables rigorous attribution analysis, supporting both correlational and, prospectively, causal research on the mechanisms underlying persuasive argumentation and other creative tasks.
4. Targeted Generation: Rich action traces can be used for targeted search, allowing generators to systematically explore or optimize for unvisited and potentially high-utility regions of the design space.

Theoretical and Research Implications

STAT positions tree search in language generation within the broader literature of search in structured decision spaces (cf. RAP [hao-etal-2023-reasoning], AlphaZero [silver_etal_2018_alphazero], MCTS [kocsis_and_szepesvari_2006_bandit_monte_carlo_planning, browne_etal_2012_mcts_survey]). By augmenting generative models with explicit, interpretable action spaces, STAT strengthens the connection between symbolic/structured planning and neural text generation. The traceability of controller actions supports causal inference methodologies (cf. Egami et al., 2022; Feder et al., 2022), opening pathways for experimental manipulation and mediation analyses of rhetoric or reasoning strategies in text as treatment paradigms.

On the modeling side, these findings challenge the assumption that increased model scale or instruction tuning alone is sufficient for coverage of reasoning space diversity in LLMs. Instead, structured search over explicit conceptual or reasoning interventions is required for both research explainability and practical optimization.

Potential Directions for Future Work

• Causal Attribution: Formalizing sequential actions as treatments enables use of marginal structural models, g-computation, and mediation analysis for causal inference on LLM decision traces.
• Optimization and Search: Integrating policy-gradient or MCTS variants for adaptive action-space exploration during generation.
• Application Domains: Expanding structured action templates into domains such as program synthesis, multi-turn negotiation, or adversarial debate frameworks.
• Human Alignment: Rigorous human evaluation of controllable action-trace interventions, especially in domains sensitive to manipulation (e.g., persuasion, misinformation, or negotiation).
• Defense and Red-Teaming: Using controllable argument space exploration for both red-teaming (discovering effective manipulative strategies) and defense (identifying and mitigating problematic rhetorical moves).

Conclusion

STATe-of-Thoughts constitutes a substantive advance in ITC methods for LLMs by combining controllable, interpretable, and semantically meaningful search over high-level reasoning actions. It overcomes the limitations of token-level temperature sampling, achieves superior diversity without loss of quality, and enables detailed analysis of the relationships between reasoning strategies and downstream outcomes. The framework not only enriches the empirical study of LLM behavior but also provides concrete mechanisms for targeted generation, theoretical inquiry, and applied optimization in complex textual tasks.