Chain-of-Guidance Reasoning in AI

• Chain-of-Guidance Reasoning is a structured framework that separates guidance generation from reasoning execution, using intermediate artifacts to steer inference.
• It encompasses methods like Step Guided Reasoning, Description-then-Decision, and cross-model transfer, which refine guidance to enhance task performance.
• The framework addresses challenges in guidance fidelity by diagnosing pathologies such as post-hoc, encoded, or internalized reasoning through targeted metrics.

Chain-of-Guidance Reasoning denotes a family of reasoning procedures in which an explicit intermediate artifact guides subsequent inference rather than merely accompanying it. Depending on the setting, that artifact may be a natural-language guidance string generated before the next reasoning step, a descriptive perceptual summary that conditions a later decision, a prefix of another model’s chain-of-thought (CoT), or a low-dimensional representation-space intervention applied directly to hidden states. Across these variants, the central premise is that reasoning can be decomposed into guidance-producing and guidance-consuming phases; the central difficulty is that successful performance does not by itself establish that the visible guidance is faithful, causally necessary, or semantically load-bearing (Cao et al., 2024, Cheng et al., 27 May 2026, Wu et al., 2023, Hu et al., 2024, Liu et al., 14 Feb 2026).

1. Conceptual foundations

In its most general form, Chain-of-Guidance Reasoning treats intermediate structure as a control signal for later computation. In Step Guided Reasoning (SGR), the model alternates between guidance generation and step reasoning, so that each reasoning paragraph is conditioned on a fresh meta-level reflection about what knowledge will be needed next (Cao et al., 2024). In complex vision-language reasoning, the same pattern appears as “Description then Decision”: the model first produces a question-relevant description of visual content and then makes a choice conditioned on that description, replacing a direct one-step mapping with a guided two-stage trajectory (Wu et al., 2023). In cross-model transfer, a provider model’s CoT becomes an external guidance artifact for a receiver model, which may either answer from the given prefix or continue reasoning from it (Cheng et al., 27 May 2026). In the Hopfieldian account, guidance is internalized further: prompts and CoT stimuli move hidden activations toward low-dimensional representation spaces associated with reasoning, and Representation-of-Thought (RoT) intervenes directly on those spaces (Hu et al., 2024).

This convergence suggests that Chain-of-Guidance Reasoning is not a single prompting recipe but a broader organizational principle. The shared structure is a separation between a guidance channel and a task-answering channel. What varies is the ontology of the guidance signal: text, descriptive latent summary, transferred prefix, or manipulated hidden-state direction. A plausible implication is that the field’s main theoretical questions concern when such signals are causally effective, how they transfer across models and modalities, and under what conditions they remain interpretable.

2. Explicit guidance at inference time

SGR provides the clearest inference-time instantiation of the paradigm. It is a pure inference-time method applied to a general instruction-tuned model, and it avoids additional fine-tuning by replacing single-pass CoT with repeated guidance–reasoning cycles (Cao et al., 2024). The first iteration uses a two-phase procedure: a Guidance Question is generated from the problem, then answered to produce the first Step Guidance. Subsequent iterations skip the question-answer split and directly generate guidance from the original question and accumulated reasoning. The reasoning step is then produced under a prompt that instructs the model to continue the next step, avoid repetition, and answer according to the previous content. A step is defined as a paragraph ending at the delimiter .\n\n and exceeding a minimum length; the paper uses a maximum number of steps $T = 10$.

The recurrence can be expressed as

$$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$

Here $Q$ is the problem, $G_t$ is the current guidance, $S_t$ is the current reasoning paragraph, and $R_t$ is accumulated reasoning. Conceptually, the method turns monolithic CoT into guided step-wise reasoning: the model first reflects on what relevant knowledge will be needed in the future, then uses that reflection to drive the next chunk of solution text.

The vision-language “Description then Decision” strategy implements the same decomposition in a multimodal setting (Wu et al., 2023). Instead of directly asking which caption matches an image or which image matches a caption, the model is instructed: “First, describe the image information relevant to the question. Then, provide your answer.” The paper contrasts the direct formulation,

$\text{P(Answer | Image, Question)},$

with a guided formulation in which an intermediate description conditions the answer,

$\text{P(Answer | Image, Question, Description)}.$

It further introduces a two-turn variant in which a vision-LLM first generates the description and a LLM then answers from that description alone, making the recognition/description and reasoning/decision modules more explicitly separable.

Both SGR and Description-then-Decision therefore instantiate the same operational principle: guidance is generated before the decisive step and inserted into the context as a structured constraint on what comes next. This differs from standard CoT, where the model is merely told to “think step-by-step” and implicitly decides for itself how to organize intermediate reasoning.

3. Guidance as a transferable external state

The provider–receiver framework studies Chain-of-Guidance Reasoning in its most literal cross-model form: one model generates a CoT trace, and another model solves the same problem conditioned on increasingly longer prefixes of that trace (Cheng et al., 27 May 2026). A provider generates a full rationale and answer, the rationale is segmented into ten ordered segments of roughly equal token length, and cumulative prefixes are formed. For each prefix, the receiver is evaluated in two modes. In force-answer mode, it must answer from the prefix as-is. In free-generation mode, it may continue reasoning before answering.

The resulting prefix trajectories show that full-trace transfer often works extremely well, but they also show that “transfer” is not a single mechanism. Averaged over all provider–receiver pairs and modes, the residual gap to provider performance at full trace is 0.17 percentage points on AIME, 0.46 on MMLU-Pro, and 3.63 on ZebraLogic, with some extra ZebraLogic gap attributed to JSON formatting errors (Cheng et al., 27 May 2026). Yet the same paper shows that high end performance does not reveal how the prefix contributes to the receiver’s answer.

Three mechanisms are empirically distinguished. First, there is answer extraction, most visible in AIME force-answer mode. The leakage curve closely tracks force-answer accuracy, and among examples where provider is correct and receiver gets correct at least once, 94.5% of first-correct force-answer steps occur at or after leakage. Second, there is reasoning scaffolding, most visible in free-generation. On AIME, 88.11% of free-generation first-correct steps occur before answer leakage, indicating that partial prefixes often help receivers construct their own solution trajectories. Among 494 prefixes where receivers first become correct before leakage, 47.57% are classified as approach-level, 47.37% as derivation-level, and 5.06% as mixed or unclear. Third, there is receiver-dependent competence. On MMLU-Pro, only 35.6% of first-correct force-answer steps occur at or after leakage, while 64.4% occur before explicit answer appearance, implying a larger role for the receiver’s own knowledge and priors. ZebraLogic exhibits a different regime: 40.1% of first-correct force-answer steps occur before the complete structured answer appears, but receivers are correct after seeing 82.8% of gold answer components on average in force-answer mode, whereas free-generation can succeed after seeing only 37.7% of components on average.

This evidence makes a central caution explicit. Successful guidance transfer does not imply that a receiver is reproducing the provider’s reasoning process. The receiver may instead be reading off leaked answers, exploiting dense partial structured information, or using the prefix only as a scaffold for its own reasoning. The same paper therefore proposes answer agreement among receivers as a gold-free stopping signal: one-step and two-step agreement rules allow provider reasoning to be stopped early when receivers converge on the same answer.

4. Mechanistic and geometric interpretations

The Hopfieldian view offers a mechanistic language for why guidance can change reasoning behavior (Hu et al., 2024). Rather than treating CoT only as output text, it interprets prompting as a stimulus that induces population-level changes in hidden activations. For a prompt $p$, the hidden state at layer $k$ is denoted $$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$0, and the stimulus-induced effect is defined as

$$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$1

where $$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$2 includes the stimulus and $$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$3 omits it. Across many prompts, these differences are collected as

$$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$4

and principal component analysis yields a representation space

$$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$5

The paper takes $$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$6, so each layer contributes a single principal direction interpreted as a CoT-related concept.

Within this framework, reasoning is a trajectory through representation space. At decoding step $$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$7, the model evaluates alignment with the learned reasoning direction through

$$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$8

followed by an average over selected layers. Correct reasoning corresponds to trajectories whose scores remain non-negative; error localization is defined by a transition where the aggregate score drops below zero after previously being non-negative. In the paper’s GSM8K case study, such a drop coincides with the token at which the model makes an arithmetic mistake involving hours and minutes.

RoT turns this interpretation into an intervention:

$$G_t = g(Q, R_{t-1}), \qquad S_t = f(Q, R_{t-1}, G_t), \qquad R_t = R_{t-1} \cup \{S_t\}.$$9

The sign of $Q$0 is chosen to reinforce existing alignment rather than flip it. The paper argues that low-dimensional representation spaces provide a more robust and interpretable control surface than prompt text alone, because they operate directly on the neural populations that CoT stimuli are presumed to activate. In the context of Chain-of-Guidance Reasoning, this extends the notion of guidance from externally visible text to internal geometric steering.

5. Pathologies, faithfulness, and diagnosis

The principal criticism of Chain-of-Guidance Reasoning is not that it never works, but that the visible guidance channel may cease to be a faithful causal substrate for the model’s answer (Liu et al., 14 Feb 2026). The paper on pathological CoT isolates three canonical failure modes. In post-hoc rationalization, the model arrives at the final answer before, or early on in, its CoT and traces an apparent reasoning path backward from that conclusion. In encoded reasoning, the model’s progress depends on conventions embedded in the CoT that go beyond normal meaning, so the true reasoning is carried by a private code rather than ordinary semantics. In internalized reasoning, the model uses the forward passes during CoT generation to compute internally while emitting semantically irrelevant or filler text.

These pathologies undermine distinct uses of guidance. Post-hoc rationalization breaks causal monitoring because editing or suppressing the CoT need not change the answer. Encoded reasoning breaks interpretability because the CoT can appear natural-language-like while its critical content resides in nonstandard lexical choices, punctuation, capitalization, or other opaque schemes. Internalized reasoning breaks the assumption that readable intermediate text reflects the computation actually being performed, since the content of the CoT may be irrelevant even when the existence of CoT tokens is still performance-critical.

To discriminate among these regimes, the paper proposes three task-agnostic, log-probability-based diagnostics. Necessity asks whether removing CoT hurts the probability of the same answer; it targets post-hoc rationalization. Paraphrasability asks whether semantic-preserving paraphrases preserve answer probability; it targets encoded reasoning. Substantivity asks whether replacing the CoT with irrelevant but well-formed reasoning from another dataset degrades answer probability; it targets internalized reasoning. All three metrics compare the original answer log-probability with the log-probability under an intervention and are scaled to lie in $Q$1.

The expected signatures are diagnostic. A healthy CoT has positive Necessity, near-zero Paraphrasability change, and positive Substantivity. Encoded reasoning has positive Necessity, negative Paraphrasability, and positive Substantivity. Internalized reasoning has positive Necessity, near-zero Paraphrasability, and near-zero Substantivity. Post-hoc reasoning has Necessity near zero, while Paraphrasability and Substantivity are unpredictable. The paper also compares pathology checkpoints to a healthy baseline with Cohen’s $Q$2, using effect sizes across metric distributions. For Chain-of-Guidance systems, this diagnostic framework is consequential because it distinguishes genuinely load-bearing guidance from decorative, private, or semantically empty guidance channels.

6. Empirical performance, scope, and unresolved questions

Empirically, guidance-based decompositions improve performance across multiple domains. In mathematics, SGR reports that Qwen2-72B-Instruct outperforms Qwen2.5-72B-Math-Instruct on MMLU-STEM with a score of 90.9%, compared to 87.3%, and that the average scores of Qwen2-7B-Instruct and Qwen2-72B-Instruct increase from 27.1% to 36.3% and from 36.5% to 47.4% on the mathematics domain, respectively (Cao et al., 2024). The same paper argues that challenging problems require multiple reasoning steps: MATH level-5 problems often need approximately two steps, while AMC23 problems concentrate around three to four steps. In vision-language reasoning, GPT-4V improves from 69.25 to 75.25 on text score, from 46.25 to 68.75 on image score, and from 39.25 to 58.75 on group score under the “Description then Decision” prompt, a relative group-score improvement of about 50% (Wu et al., 2023). InstructBLIP improves from 17.50 to 31.50 in text score with CoT, and LLaVA improves from 25.00 to 33.50.

At the same time, the cross-model transfer results show that high performance under guidance is mechanistically heterogeneous rather than uniformly interpretable (Cheng et al., 27 May 2026). Full traces can nearly close provider–receiver gaps, but short prefixes may help because they leak answers, provide partial structured solutions, or merely bias a competent receiver in the right direction. The pathology results sharpen the point further: even when visible CoT exists and performance improves, the text may be post-hoc, encoded, or semantically nonsubstantive (Liu et al., 14 Feb 2026).

Several open questions follow. One concerns control granularity: SGR and Description-then-Decision use explicit natural-language guidance, whereas RoT suggests that low-dimensional representation-space interventions may provide a more direct handle on reasoning dynamics (Hu et al., 2024). Another concerns stopping and efficiency: provider–receiver agreement offers a gold-free criterion for terminating guidance early, but the reliability of such stopping rules depends on receiver competence and task structure (Cheng et al., 27 May 2026). A third concerns safety and oversight: if training-time or deployment-time monitoring relies on visible CoT, then the pathology taxonomy implies that guidance quality must be diagnosed, not assumed (Liu et al., 14 Feb 2026).

Taken together, the literature presents Chain-of-Guidance Reasoning as a broad design space rather than a settled method. Its strongest empirical results arise when intermediate artifacts supply structured subgoals, perceptual summaries, partial derivations, or representation-level steering. Its principal limitation is that the existence of a guidance channel does not guarantee that the channel is faithful. The field’s central challenge is therefore dual: to make guidance more effective as a computational scaffold, and to make it more trustworthy as an object of interpretation and control.