Reasoning CPT: Enhanced LLM Reasoning

• Reasoning CPT is a method that extends continued pretraining by integrating annotated hidden thought sequences to scaffold multi-step reasoning in large language models.
• It employs synthetic data generation, prompt-engineered chain annotation, and low-rank adapters to fine-tune reasoning abilities across diverse domains.
• Empirical evaluations reveal substantial gains in difficult tasks, robust cross-domain transfer, and improved logical expressivity with controlled reasoning depth.

Reasoning CPT refers to a family of continued pretraining (CPT) methodologies in which LLMs are further trained using datasets specifically constructed to elicit and scaffold internal step-by-step reasoning processes. Unlike conventional CPT, which typically relies on domain-specific corpora or plain language modeling objectives, Reasoning CPT methods inject synthetic or curated "hidden thought" sequences that mimic the underlying cognitive chains leading to surface-level texts and answers. The defining feature is the explicit or implicit reconstruction of intermediate reasoning, enabling models to not only memorize facts or patterns but flexibly generalize robust reasoning strategies across domains and difficulty gradients (Ishibashi et al., 15 May 2025).

1. Formal Definition and Objective Structure

In Reasoning CPT, each training example is extended beyond its surface text $S$ by an associated latent reasoning chain $H$, generated or annotated to represent the plausible "hidden thought process" underlying $S$. The training sequence is thus constructed as:

$$X = \langle \texttt{<start\_of\_thought>}, H, \texttt{<end\_of\_thought>}, S \rangle$$

This sequence is used in causal language modeling to minimize the standard autoregressive LM loss:

$$\mathcal{L}_{\mathrm{CPT}}(\theta) = -\mathbb{E}_{X \sim \mathcal{D}}\left[\sum_{t=1}^L \log p_\theta(x_t | x_{<t})\right]$$

where $\mathcal{D}$ denotes the synthetic or annotated distribution over $(H,S)$ pairs (Ishibashi et al., 15 May 2025). The principal aim is to reconstruct both the latent reasoning and the final output, thereby encouraging the model to encode and deploy adaptively deep logical, mathematical, or commonsense reasoning structures.

2. Synthetic Data Generation and Reasoning Chain Annotation

The best-documented instantiations of Reasoning CPT involve synthetic generation of "hidden thoughts" for each seed text. For instance:

• In STEM and Law domains, original texts $S$ are paired with six-stage reasoning chains $H$ produced by prompt-engineered LLM calls ("goal, recall background, propose approaches, compare, commit, verify") and wrapped in explicit thought demarcation tags for robust format conditioning.
• Chains are validated for coverage and relevance; distributions are controlled to balance length and diversity (e.g., $150\mathrm{K}$ examples per domain, $>85$M tokens in STEM, $>66$M tokens in Law) (Ishibashi et al., 15 May 2025).

This process ensures that the model learns to generate and consume multi-step internal reasoning, not mere answer retrieval or shallow manipulation.

3. Training Architecture and Hyperparameter Regimes

Reasoning CPT utilizes standard transformer architectures (e.g., Gemma2-9B), with adaptation via low-rank adapters (LoRA, rank $r=64$), freezing core weights for efficiency and stability. Hyperparameters are invariant between Reasoning CPT and vanilla CPT:

• batch size: $4$
• learning rate: $3 \times 10^{-5}$, cosine decay
• optimizer: AdamW
• epochs: $6$ over synthetic corpora
• context length: $1024$ tokens
• hardware: NVIDIA A100

Vocabulary is unchanged; thought tag tokens are handled by the subword model for seamless integration (Ishibashi et al., 15 May 2025).

4. Empirical Evaluation: Reasoning Transfer, Depth Adaptation, and Robustness

Robust evaluation frameworks such as MMLU and GSM8k are utilized to measure advances conferred by Reasoning CPT.

Cross-domain transfer is manifest: STEM-trained Reasoning CPT yields significant ($+2.6$ to $+5.4$ points) gains in social sciences, while Law-trained Reasoning CPT achieves ($+4.3$ points) in STEM, highlighting the generalizability of acquired reasoning skills beyond task boundaries.

Difficulty-stratified gains: The performance gap between Reasoning CPT and conventional methods widens with problem difficulty; on MMLU, Reasoning CPT confers up to $+11.2$ points over baseline on very hard problems.

Adaptive reasoning depth: Analysis reveals that models trained with hidden thoughts increase the number and granularity of reasoning tokens for harder inputs, exhibiting dynamic adjustment of inference-chain length to complexity.

Reasoning diversity is quantified by Pass@$k$ metrics on GSM8k; Reasoning CPT models reach high accuracy ($91.7\%$ at Pass@5), outperforming instruction-tuned baselines even at lower $k$ (Ishibashi et al., 15 May 2025).

5. Reasoning CPT in Domain-Specific Adaptation and Reasoning Stability

Reasoning CPT is leveraged in specialized domains (e.g., medical LLMs for Japanese clinical decision support):

• CPT infuses deep domain knowledge via large-scale curated corpora (e.g., $>5$B tokens for Japanese medical data) and regularizes towards pretrained weights to prevent knowledge drift.
• Reasoning Preference Optimization (RPO, or DPO-style ranking loss) finetunes models so that high-quality, stepwise reasoning chains are more probable, stabilizing both factual accuracy and explanation reliability under "explain-before-answer" prompting (Kawakami et al., 25 Apr 2025).
• Empirically, CPT+RPO yields zero accuracy drop whether or not explanations are requested, while baselines and CPT-only exhibit degradation up to $11.5\%$ (Kawakami et al., 25 Apr 2025).

6. Reasoning CPT and Logical Expressivity: Choiceless Polynomial Time

Within the theoretical logic arena, "Reasoning CPT" relates to Choiceless Polynomial Time (CPT), which defines polynomial-time reasoning in a way that is invariant under automorphisms and ordering. Here, CPT is extended to witness symmetric choice operators, which allow for deterministic construction of reasoning paths from definable orbits, provided automorphism certificates are supplied for polynomial-time evaluability (Lichter et al., 2022). This symmetry-centric formalism further elevates the logic of reasoning chains, generalizing canonical forms and automating isomorphism-to-canonization steps across classes of mathematical structures.

7. Robustness and Vulnerabilities: Compromising Thought

Not all reasoning CPT approaches guarantee robust performance. If models are exposed to manipulated reasoning tokens (e.g., compromised chain-of-thoughts with tampered final results), LLMs may abandon their correct internal reasoning and adopt incorrect endpoints—a vulnerability termed "Compromising Thought (CPT)" (Cui et al., 25 Mar 2025). Systematic evaluation reveals that local endpoint manipulations exert greater compromise than structural changes. Explicit prompt-level defenses and architectural validators are necessary to mitigate such vulnerabilities.

Summary Table: Principal Reasoning CPT Findings

Method	Key Feature	Reasoning Gains (max)
Synthetic Hidden Thoughts	Paired $(H, S)$ pretraining	+3.3 points overall; +11.2 points on 'very hard'; dynamic reasoning length scaling; cross-domain transfer (STEM $\rightarrow$ Law and vice versa) (Ishibashi et al., 15 May 2025)
CPT + RPO (Medical LLMs)	Domain knowledge + preference loss	Stable explanation accuracy (0.868) under prompting; baseline drops up to 11.5% (Kawakami et al., 25 Apr 2025)
GraphPile CPT	CPT from graph reasoning tasks	Up to +21.2% in non-mathematical reasoning, +4.9% in math; robust transfer of decomposition-state-update-decision patterns (Zhang et al., 23 Jul 2025)
CPT in Logic (CPT+WSC)	Symmetric choice in CPT logic	Automates isomorphism/canonization; polynomial-time reasoning from witnessed orbits (Lichter et al., 2022, Pago, 2021)
CPT Vulnerabilities	Resistance to tampered CoT endpoints	High risk from local result manipulations; explicit prompt-level defenses required (Cui et al., 25 Mar 2025)

Outlook

Reasoning CPT, with its explicit scaffolding of internal thought processes, provides systematic enhancements in reasoning capability, adaptability, and transfer across diverse domains and task complexities. It also motivates novel approaches for robustness verification and symmetry-preserving logic representations, while exposing unique vulnerabilities that demand continued methodological innovation.