Think-Then-Generate (T2G) Overview

• Think-Then-Generate (T2G) is a modeling strategy that decomposes output generation into an explicit intermediate 'think' phase followed by a final generation stage.
• It leverages varied intermediate representations such as chain-of-thought, error diagnosis, and evidence selection to improve reasoning and task-specific performance.
• Recent research highlights trade-offs between static pre-planning and dynamic token-level reasoning, prompting further study on optimal thought representations for diverse applications.

Think-Then-Generate (T2G) denotes a class of modeling strategies in which generation is explicitly decomposed into a preliminary “think” phase and a subsequent “generate” phase. The intermediate object can be an explicit chain-of-thought, a plan, an error diagnosis, a query rewrite, a retrieved reasoning trace, a set of selected evidence spans, or a latent reasoning state. In the broadest formulation, the decomposition treats the thought trace as an intermediate variable $z$, so that $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$, with the final output $y$ produced only after conditioning on the constructed or selected thought (Xia et al., 18 Apr 2025). Across recent work, T2G has become less a single algorithm than a family of staged inference protocols whose main axes are the representation of thought, the timing of thought relative to decoding, and the degree to which the thought state is fixed or dynamically updated (Li et al., 2023, Wang et al., 7 Dec 2025, Kou et al., 15 Jan 2026).

1. Conceptual scope and formal variants

The common core of T2G is staged inference: some intermediate cognitive artifact is made explicit before, or at least in service of, final generation. What differs sharply across papers is the meaning of “thought.” In text prompting, it may be an error label over an imperfect candidate; in grounded generation, it may be a set of source highlights and sentence clusters; in retrieval systems, it may be a decision about whether to rewrite or retrieve; in reasoning-intensive RAG, it may be a retrieved thinking trace; in diffusion or multimodal generation, it may be a rewritten prompt, a compact instruction sequence, or an interleaved textual-visual trajectory (Li et al., 2023, Slobodkin et al., 2024, Shen et al., 2024, Arabzadeh et al., 5 May 2026, Kou et al., 15 Jan 2026).

Domain	Think phase	Generate phase
Text prompting	Error detection or CoT	Refined answer or text
Personalized long-form	Static or token-level reasoning	Long response aligned to user history
Grounded generation	Span selection and sentence planning	Sentence sequence with local citations
RAG	Query/retrieval gating or trace retrieval	Answer with or without external context
Visual generation	Prompt rewrite, instruction CoT, or interleaved thoughts	Image, edited image, or visual trajectory

This diversity matters because later work argues that not all T2G formulations have the same inductive bias. A one-shot pre-plan can be sufficient when the task is short, well scoped, or naturally compressible into an outline, but can become mismatched when the target output is long, multimodal, or needs local correction as it unfolds. This suggests that T2G is best understood as a design space whose central question is not merely whether to think first, but what state to construct, how to supervise it, and whether it should remain fixed during downstream decoding (Wang et al., 7 Dec 2025, Guo et al., 20 Nov 2025).

2. Textual T2G: deliberation, search, and refinement

In text generation, one major line of work instantiates T2G as deliberate prompting. “Deliberate then Generate” inserts an explicit error-detection stage before the final answer by appending the instruction “Please detect the error type firstly, and provide/refine the [task] then,” often together with an empty or deliberately imperfect candidate. The model performs diagnosis and refinement in a single pass, with default decoding $T=0$ and $\text{top\_p}=1$, and the paper reports gains across machine translation, summarization, dialogue summarization, simplification, style transfer, paraphrase, and commonsense generation. Examples include CNN/DailyMail ROUGE-1 $40.17$ $(+1.30)$, SamSum ROUGE-1 $48.72$ $(+2.28)$, Asset BLEU $80.2$ $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$0 and SARI $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$1 $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$2, and CommonGen BLEU-4 $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$3 $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$4 (Li et al., 2023).

A broader theoretical framing treats T2G as the central inference pattern of “Act II” cognition engineering. In that view, the think phase produces language-level thoughts such as plans, hypotheses, derivations, critiques, verifications, backtracking, or tool-use decisions, and the generate phase emits the answer conditioned on those thoughts. Canonical T2G variants include Chain-of-Thought, self-consistency, tree-of-thought search, critique-and-revise loops, verifier-guided best-of-$$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$5, and deliberate slow thinking with extended reasoning-token budgets. The practical lesson is that T2G is not limited to a single forward chain; it can also be a compute-allocation policy over sampling, search, reflection, and verification (Xia et al., 18 Apr 2025).

A recurring misconception is that T2G in text is synonymous with standard CoT prompting. The literature does not support that equivalence. DTG uses error diagnosis over negative evidence rather than mathematical derivation; verifier-guided selection treats thoughts as candidates to be scored; reflection loops treat thoughts as iterative critique; and long-CoT systems treat them as a budgeted search process. Conversely, the same literature also records limits: gains saturate with larger $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$6, search breadth, or token budget; verifier quality often becomes the bottleneck; and overthinking can reduce accuracy on simple tasks (Xia et al., 18 Apr 2025).

3. Static pre-planning and the challenge of personalized long-form generation

One explicit formulation of T2G in personalized generation defines it as a class of methods that first produce an explicit or latent reasoning trace $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$7 from user history $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$8 and query $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$9, and then generate the full response $y$0 conditioned on that single, precomputed reasoning. Representative variants include plan-then-write, chain-of-thought or scratchpad prompting, and latent reasoning methods such as Coconut. In this formulation, T2G is a static one-shot reasoning paradigm: the entire response must be guided by a reasoning state fixed before full generation begins (Wang et al., 7 Dec 2025).

That static assumption is the target of a direct critique in personalized long-form generation. The argument is that user preferences are implicit in history, often emerge only across later segments, and may shift as content unfolds; therefore a one-shot plan must capture all relevant information in advance, creating long-range dependencies that are hard to learn and easy to lose. The reported failure mode is “context drift,” especially in later segments of long outputs. On LONGLAMP user-based splits, the proposed think-while-generating alternative, FlyThinker, exceeds static T2G baselines across Product Review, Abstract Generation, and Topic Writing. For Product Review, it reports ROUGE-1 $y$1 versus SFT $y$2, CoT $y$3, and Coconut $y$4, with BLEU $y$5 versus SFT $y$6. Position-sensitive evaluation further shows that SFT, CoT, and Coconut degrade significantly in later segments, whereas FlyThinker maintains personalization quality (Wang et al., 7 Dec 2025).

The technical distinction is precise. FlyThinker separates a Reasoner $y$7 from a Generator $y$8, with latent thoughts $y$9 satisfying

$T=0$0

and crucially depending only on $T=0$1, not on $T=0$2. Generation uses embedding-level fusion,

$T=0$3

so that reasoning remains token-local and training remains fully parallel under teacher forcing. The paper gives a concrete compute example for $T=0$4: FlyThinker costs about $T=0$5 TFLOPs, versus Coconut $T=0$6 TFLOPs and SFT $T=0$7 TFLOPs, while retaining near-SFT latency through staggered parallel decoding (Wang et al., 7 Dec 2025).

This comparison does not invalidate T2G as such; it narrows the conditions under which static T2G is well matched to the task. The reported practical guidance is explicit: for short, well-scoped responses or tasks with explicit plans, T2G is sufficient and simpler; for personalized long-form generation over hundreds of tokens, dynamic token-level reasoning is more effective (Wang et al., 7 Dec 2025).

4. T2G as evidence selection, retrieval control, and trace retrieval

Another branch of T2G shifts the think phase away from solution derivation and toward evidence organization. “Attribute First, then Generate” decomposes grounded generation into content selection, sentence planning, and sequential sentence generation. The selected highlights are verbatim source spans, the planner groups them into ordered sentence-level clusters $T=0$8, and each sentence is generated conditioned on its designated cluster, with the same cluster becoming the sentence’s local citation set. The sequential objective is written as $T=0$9. On long-form question answering, the ICL-CoT variant reports ROUGE-L $\text{top\_p}=1$0, BERTScore $\text{top\_p}=1$1, AUTOAIS $\text{top\_p}=1$2, LENGTH $\text{top\_p}=1$3, and NO ATT $\text{top\_p}=1$4, versus ALCE LENGTH $\text{top\_p}=1$5; the paper summarizes this as approximately $\text{top\_p}=1$6 shorter citations, together with improved attribution accuracy and reduced verification time by about $\text{top\_p}=1$7 (Slobodkin et al., 2024).

In retrieval-augmented generation, T2G can also mean pre-generation control of external actions. “Think-then-Act” introduces two gates before answer generation: a clarity/completeness assessment that decides whether to rewrite the query, and a capability/answerability assessment that decides whether retrieval is needed. If confidence $\text{top\_p}=1$8 exceeds threshold $\text{top\_p}=1$9, the model answers directly; otherwise it answers with retrieved context,

$40.17$0

With $40.17$1, the reported results include HotPotQA $40.17$2, 2WikiMultihopQA $40.17$3, StrategyQA $40.17$4, and FEVER $40.17$5 in EM/F1, with a ChinesePoetry retrieval ratio of $40.17$6 versus about $40.17$7 for an LM-preprocessing approach and $40.17$8 for retrieve-always (Shen et al., 2024).

A more radical retrieval-based reinterpretation appears in reasoning tasks. Instead of retrieving documents, “RAG over Thinking Traces” retrieves prior reasoning trajectories, optionally transformed by T3 into Struct, Semantic, or Reflect representations. The inference rule is simple,

$40.17$9

but the corpus $(+1.30)$0 consists of thinking traces rather than web passages. Reported gains are substantial: on AIME 2025–2026, RAG with Gemini-2-thinking traces yields relative improvements of $(+1.30)$1 for Gemini-2.5-Flash, $(+1.30)$2 for GPT-OSS-120B, and $(+1.30)$3 for GPT-5; the paper further states that T3 can incur little or no extra inference cost and can even reduce inference cost by up to $(+1.30)$4 (Arabzadeh et al., 5 May 2026).

These variants broaden the meaning of T2G. Here the think phase may be evidence selection, citation planning, action gating, or retrieval of externalized process knowledge rather than fresh on-the-fly reasoning.

5. Multimodal and visual T2G

Visual generation has produced several non-equivalent T2G formulations. One diffusion-oriented line uses an LLM encoder not merely as a text encoder but as an active planner and rewriter. In “Think-Then-Generate: Reasoning-Aware Text-to-Image Diffusion with LLM Encoders,” a Qwen2.5-VL encoder is first SFT-trained on $(+1.30)$5 prompts to produce CoT and refined prompts, and then jointly optimized with an MM-DiT backbone via Dual-GRPO. The refined prompt states $(+1.30)$6 become the conditioning signal for diffusion, and the method reports WISE overall $(+1.30)$7, nearly on par with GPT-4o at $(+1.30)$8, as well as T2I-ReasonBench Accuracy $(+1.30)$9 and Quality $48.72$0 (Kou et al., 15 Jan 2026).

A second line argues that fixed pre-planning is inadequate for visual generation just as it is for personalized long-form text. “Thinking-while-Generating” interleaves textual thoughts $48.72$1 with region-level generation $48.72$2 and optional reflection $48.72$3 inside a single autoregressive trajectory. With a static schedule $48.72$4 and uniform spatial spacing, zero-shot TwiG already improves Janus-Pro-7B on T2I-CompBench from Color $48.72$5 to $48.72$6, Shape $48.72$7 to $48.72$8, Texture $48.72$9 to $(+2.28)$0, and Complex $(+2.28)$1 to $(+2.28)$2. TwiG-SFT and TwiG-RL extend these gains; TwiG-RL reaches Color $(+2.28)$3, Shape $(+2.28)$4, Texture $(+2.28)$5, Spatial $(+2.28)$6, and Complex $(+2.28)$7 (Guo et al., 20 Nov 2025).

A third formulation makes the intermediate states explicitly multimodal and process-supervised. “Think in Strokes, Not Pixels” defines an interleaved trajectory

$(+2.28)$8

with a four-stage loop of textual planning, visual drafting, textual reflection, and visual refinement. The model is trained on $(+2.28)$9 multi-turn generation samples, $80.2$0 instruction–intermediate conflict samples, and $80.2$1 image–instruction alignment samples. On GenEval, the process-driven model improves BAGEL-7B from $80.2$2 to $80.2$3 overall, with Position $80.2$4 and Color Attributes $80.2$5; on WISE it improves from $80.2$6 to $80.2$7 overall (Zhang et al., 6 Apr 2026).

ThinkGen and TwGI extend the same theme in different architectural directions. ThinkGen decouples a Qwen3-VL-8B-Think MLLM from an OmniGen2-DiT-4B generator, retains only post-</think> states through a VGI-refine interface with $80.2$8 Prepadding States and a linear $80.2$9 connector, and trains the two modules with SepGRPO. Reported gains include WISE $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$00, RISEBench average $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$01, CVTG Acc/NED $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$02, and ImgEdit $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$03 when CoT is enabled (Jiao et al., 29 Dec 2025). TwGI instead uses a single unified autoregressive LMM, Anole-7B, to generate intermediate visual subgoals or self-critiqued image hypotheses inside one multimodal token stream. On GenEval, its subgoal model improves Two Objects from $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$04 to $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$05, Position from $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$06 to $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$07, and Color Attributes from $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$08 to $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$09; its self-critique variant improves GenEval Overall from $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$10 to $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$11 and DPG-Bench Overall from $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$12 to $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$13 (Chern et al., 28 May 2025).

Taken together, these results show that visual T2G now spans at least three regimes: pre-generation prompt reasoning, interleaved textual guidance during generation, and fully process-driven multimodal trajectories in which both text and images function as intermediate thought states.

6. Trade-offs, misconceptions, and open technical questions

One misconception is that T2G always means a single explicit chain-of-thought followed by a final answer. The surveyed work does not support that narrow reading. In some systems, the think phase is an error type over an empty candidate; in others it is a query-rewrite decision, a highlight cluster, a retrieved trace, a latent token sequence, a refined prompt state, or a sequence of intermediate images. The broad family resemblance is staged control, not any one representation (Li et al., 2023, Shen et al., 2024, Arabzadeh et al., 5 May 2026, Chern et al., 28 May 2025).

A second misconception is that more thinking is always better. Several papers document the opposite boundary conditions. Long-CoT and search scale positively but saturate; overthinking can hurt simple tasks; reflection-heavy supervision can induce over-corrections; very large $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$14 in latent-fusion systems destabilizes generation; and increasing $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$15 in retrieval over traces can introduce noise, with $$P(y \mid x)=\sum_{z \in \mathcal{Z}} P(y \mid x,z)\,P(z \mid x)$$16 reported as most stable (Xia et al., 18 Apr 2025, Guo et al., 20 Nov 2025, Wang et al., 7 Dec 2025, Arabzadeh et al., 5 May 2026).

The central engineering trade-off is between control and efficiency. Static T2G is simple and often adequate for short-form tasks or coarse global planning, but it cannot revise its thought state using partial renders or later textual context. Interleaved or process-driven variants improve local correction and long-horizon consistency, but they add reasoning tokens, dual-model memory pressure, reflection passes, or reinforcement-learning cost. FlyThinker explicitly cautions that memory increases because two models are active; TwiG notes extra textual reasoning steps and occasional local re-generation; ThinkGen notes reward-hacking risk in scenario-specific rule models; TwGI documents planning–execution disconnects in which a correct critique is not successfully implemented in the next image (Wang et al., 7 Dec 2025, Guo et al., 20 Nov 2025, Jiao et al., 29 Dec 2025, Chern et al., 28 May 2025).

Ethical and governance concerns also recur. Personalized T2G relies on implicit signals in user history and inherits any dataset biases; the authors of FlyThinker state that personalization raises privacy considerations requiring careful governance of user history collection and use. Retrieval-based T2G depends on corpus quality and decontamination. Reward-shaped visual systems inherit the failure modes of their reward models or judges. This suggests that future progress is likely to depend not only on better thought representations, but also on better control of when thoughts should be generated, retrieved, revised, or withheld (Wang et al., 7 Dec 2025, Arabzadeh et al., 5 May 2026, Kou et al., 15 Jan 2026).

In that sense, T2G has evolved from a prompt pattern into a general systems principle: allocate explicit intermediate structure before committing to generation, but choose the form and timing of that structure to match the task’s temporal, evidential, and multimodal demands.