Chain-of-Steps (CoS): A Unified Reasoning Paradigm

• Chain-of-Steps (CoS) is a framework that decomposes complex tasks into ordered intermediate steps, boosting interpretability in reasoning, planning, and formal proofs.
• It encompasses multiple variants—Chain-of-Thought, Chain-of-Symbol, Chain-of-Solving, and Vision-Language CoS—tailored to natural language, symbolic, tool-use, and multimodal tasks.
• Empirical findings show that CoS methods enhance accuracy and efficiency by isolating critical reasoning steps using metrics like True Thinking Score and streamlined tactic calls.

Chain-of-Steps (CoS) is an umbrella term for methodologies that elicit, structure, and operationalize reasoning or decision-making as explicit, ordered sequences of intermediate steps in large models—including language, vision-language, planning, and formal proof domains. The CoS paradigm encompasses verbalized chains (e.g., Chain-of-Thought), symbolic or tool-using sequences, and formalized state transitions, aiming to increase interpretability, facilitate causal analysis, and enable fine-grained control or reward assignment in complex tasks.

1. Formal Definitions and Core Variants

The foundational abstraction of Chain-of-Steps is a decomposition of problem-solving into a tuple

$$\langle\,\text{Input},\,s_1,\,s_2,\,\ldots,\,s_n,\,\text{Output}\rangle$$

where each $s_i$ represents an explicit intermediate reasoning step or state. This structure generalizes across modalities and model classes:

• Chain-of-Thought (CoT): The canonical instantiation, eliciting natural-language stepwise rationales from LLMs (Xia et al., 2024). Each $s_i$ is a text span, typically marked (“Step i: …”).
• Chain-of-Symbol (CoS): Chained symbolic representations, often tuples $(o_i, \sigma_k, o_j)$, compactly encode environments or plans and are well suited for spatial/planning tasks (Hu et al., 2023).
• Chain-of-Solving (CoS): Explicit interaction with external tools or code, where the chain interleaves natural language planning and code execution (Qian et al., 2023).
• Chain of States (CoS) in Formal Proof: Each $s_i$ is a formal goal or context (e.g., Lean 4 goal states), and the chain encodes the transitions reconstructing a proof (Wang et al., 11 Dec 2025).
• Multimodal CoS: Stepwise reasoning over both text and vision inputs, each step partitioned into structured fields (e.g., Name, Thought, Reflection) (Chen et al., 23 Sep 2025).

The chain may embody interpretable text, formal objects, or abstract latent states; the unifying property is explicit serialization of reasoning or action as stepwise transitions.

2. Methodological Frameworks and Algorithms

Canonical CoS is instantiated in various prompting and training workflows:

• Prompt Pattern (Chain-of-Thought style) (Xia et al., 2024):

  Q: [question] Let's think step by step. Step 1: ... Step 2: ... ... Answer: ...

• Symbolic Planning (Chain-of-Symbol) (Hu et al., 2023):
  • Input scenarios parsed into symbolic triples and output in a compact chain.
  • Prompts request the model to emit chains like A//B//C rather than verbose NL rationale.
• CoS for Tool Use (Chain-of-Solving) (Qian et al., 2023):
  • Toolkit creation: Decompose task $T$ into functions $k_i$.
  • CoS-Planning: List tools and plan sequence.
  • CoS-Calling: Write and execute code, returning answer.
• CoS in Formal Proof (Wang et al., 11 Dec 2025):
  • Extract formal “Chain of States” by elaboration tree traversal of Lean proofs.
  • Each adjacent pair $(s_p, s_{n})$ becomes a local tactic-synthesis subproblem.
• Vision-Language CoS (Chen et al., 23 Sep 2025):
  • Each step is a triple (Name, Thought, Reflection); chains are represented as
  • $\mathbf{p} = (p^{(1)}, p^{(2)}, ..., p^{(K)})$.
  • Process Reward Models (PRMs) can assign stepwise correctness scores, enabling reinforcement learning and step-level inference-time selection.

3. Causal Faithfulness and Analysis of Step Contribution

A critical concern is whether generated steps reflect computation actually used by the model.

• True Thinking Score (TTS) (Zhao et al., 28 Oct 2025):

$$TTS(s) = \frac{1}{2}\left(\big|S_1(1) - S_0(1)\big| + \big|S_1(0) - S_0(0)\big|\right)$$

where $$S_x(c) = \Pr(y^*|\mathbf{C}=c,\mathrm{do}(\mathbf{X}=x))$$ refers to the model’s confidence in the final answer after “surgical” perturbations of step $s$ and its context. TTS measures stepwise necessity and sufficiency. Empirically, only ≈2.3% of steps in CoT chains have TTS ≥ 0.7 on datasets like AIME/Qwen-2.5, with the majority “decorative” (TTS≈0).

• Latent Steering: “TrueThinking” direction in Model’s residual stream—a difference vector between mean activations for high-TTS (true-thinking) and low-TTS (decorative) steps—enables causal interventions: forcing or suppressing attention to a given step can flip model output in ≈50–90% of cases, including previously decorative “aha” steps.
• Key Step Discovery via Dual Chains: The EDIT method aligns correct/incorrect CoT pairs using edit distance, identifying the ≈4.7% key tokens as pivotal reasoning steps. Optimizing for their accurate generation in students boosts both in-domain and out-of-domain accuracy (Dai et al., 2024).

4. Empirical Performance, Task Domains, and Comparative Analyses

CoS-type methods have been robustly validated on a diversity of reasoning, planning, and formal logic tasks:

• Natural-Language Reasoning: CoS/CoT improves LLM performance by 5–20% on arithmetic, commonsense, symbolic, and logical benchmarks relative to direct answer prompting (Xia et al., 2024).
• Planning via Symbolic Chains: On Brick World (planning), CoS achieves 92.6% accuracy (vs. 75.1% for CoT and 31.8% for zero-shot CoT). Prompt token usage can be reduced by ≈66% (Hu et al., 2023).
• Tool-Using and Program Synthesis: LLaMA-CoS trained on CoS-GPT matches ChatGPT on tool-planning/calling benchmarks and outperforms both CoT fine-tuned and directly prompted LLaMA by 50–60 percentage points in accuracy on BIG-bench tasks (Qian et al., 2023).
• Formal Proofs: CoSProver achieves 69.2–70.3% accuracy on MiniF2F while using 1–2 orders of magnitude fewer tactic calls than leading automated theorem provers (Wang et al., 11 Dec 2025).
• Vision-Language Tasks: Applying CoS-like SFT and RL improves average accuracy by 3–5% over SFT-only baselines, with step-level process rewards yielding further gains (Chen et al., 23 Sep 2025).

Table 1: Comparison of CoS with Related Step-Based Paradigms

Paradigm	Step Semantics	Typical Domains
Chain-of-Thought	Natural language steps	Math, logic, commonsense
Chain-of-Symbol	Symbolic spatial/state triples	Planning, navigation, manipulation
Chain-of-Solving	Tool planning/calling	Tool-use, code-gen, pipelines
Chain of States	Formal proof states	Automated theorem proving
Vision-Language CoS	Structured multimodal steps	VQA, chart reasoning, MMQA

5. Theoretical Foundations and Design Principles

Rigorous theoretical work has clarified when CoS yields maximal gains:

• Markovian Analysis (Wang et al., 27 Feb 2026): Model stepwise reasoning as a Markov chain

$$s_0 \xrightarrow{r_1} s_1 \xrightarrow{r_2} \ldots \xrightarrow{r_T} s_T$$

with transition kernels $P^{(t)}$. When kernels are shared across steps (“transition alignment”), CoS reduces inference-time sample complexity proportionally to $1/T$. Under “misalignment” (heterogeneous steps), these benefits are limited to logarithmic improvement. Increased noise in local transitions exponentially decreases the advantage of direct inference, favoring stepwise decomposition.

• Prompt Engineering: Best practices include clear step markers, provision of high-quality few-shot exemplars, modular task decomposition to ensure transition alignment, and maximizing per-step determinism.
• Causal Validation: Causal interventions (steering, necessity/sufficiency ablations, edit distance in dual chains) are essential to verify the computational relevance of individual steps. Both need to be checked for efficiency, transparency, and safety in monitoring model reasoning (Zhao et al., 28 Oct 2025).

6. Applications, Extensions, and Limitations

• Symbolic and Tool-Use: CoS extends beyond natural language to code and symbolic reasoning. Symbolic chains are especially effective for spatial/environment planning and cut token cost while improving clarity (Hu et al., 2023).
• Formal Verification: CoS decomposes proof search, reducing the combinatorial explosion of tactic calls and improving alignment with informal argument structure (Wang et al., 11 Dec 2025).
• Multimodal Reasoning: Step-structured decompositions combined with reward models (PRMs) drive performance improvements and allow for fine-grained RL in complex settings (Chen et al., 23 Sep 2025).
• Distillation and Transfer: Identifying and training on key steps, via dual chains and minimum edit distance, improves student model robustness on both in-domain and out-of-domain tasks (Dai et al., 2024).

Limitations: Most LLM-generated chains contain mostly decorative steps with negligible causal contribution; measures like TTS or edit distance–extracted key steps are required for faithful auditing. Manual conversion to symbolic CoS or toolkit definition burdens scaling. Some approaches require access to closed-source APIs or annotations for reward modeling.

7. Broader Implications and Recommendations

• Faithfulness and Trust: Stepwise outputs cannot be naively assumed to reflect internal computation (Zhao et al., 28 Oct 2025). Both necessity and sufficiency should be causally assessed per step before using CoS/CoT as a rationale for audit, safety, or verification.
• Efficiency: Skipping decorative or low-TTS steps can reduce LLM inference cost without degrading accuracy.
• Training Objectives: Explicit rewards or auxiliary losses (e.g., TTS, PRM) aligned to stepwise faithfulness may guide future model and training protocol design.
• Modular Design: CoS paradigms provide generalizable building blocks for complex reasoning, planning, tool use, and multi-agent workflows.

CoS unifies a broad family of explicit-step reasoning approaches, with systematic benchmarking, causal analysis, and architectural formalization emerging as critical for their continued adoption and extension in LLM and multimodal models. Key open directions include scalable faithfulness auditing, automation of symbolic chain extraction, and generalized interface design for reasoning with and about chains.