PromptCoT 2.0: Scalable Prompt Synthesis

• PromptCoT 2.0 is a scalable, learnable framework that synthesizes challenging prompts to enhance reasoning in large language models.
• It employs a latent rationale variable and an expectation–maximization loop to jointly optimize prompt generation and improve training data diversity.
• The framework supports both self-play and supervised fine-tuning, yielding measurable performance gains in mathematical and code reasoning benchmarks.

PromptCoT 2.0 is a scalable, fully learnable framework for synthesizing high-quality, challenging prompts to improve reasoning capabilities in LLMs. By leveraging a latent rationale variable and expectation–maximization (EM) loop, PromptCoT 2.0 generates harder and more diverse training data than previous prompt synthesis paradigms, enabling both self-play improvement of strong models and supervised fine-tuning of weaker models. The approach supersedes hand-crafted heuristics with an end-to-end pipeline for rationale-informed prompt construction and demonstrates measurable advancements in mathematical and code reasoning benchmarks (Zhao et al., 24 Sep 2025).

1. Conceptual Foundations and Motivation

PromptCoT 2.0 addresses a central bottleneck in scaling LLM reasoning: the scarcity of high-quality, difficult prompts. Traditional datasets rely on costly human curation, while synthetic corpora generated via simple heuristics produce problems that are typically easy or narrow. PromptCoT 1.0 introduced a rationale variable into prompt synthesis, yielding increased problem difficulty, but still depended on hand-crafted rules for rationale generation. PromptCoT 2.0 extends this paradigm by jointly optimizing rationale and prompt generation using a domain-agnostic EM framework, eliminating the need for manual heuristics and supporting the creation of fundamentally harder and distributionally distinct problems (Zhao et al., 24 Sep 2025).

2. Expectation–Maximization Framework for Prompt Synthesis

PromptCoT 2.0 models prompt synthesis as the probabilistic process:

$p(x|c) = \sum_z p_\theta(x,z|c),$

where $c$ is a set of abstract concepts, $z$ is a latent rationale ("thinking process"), and $x$ is the synthesized prompt. Direct marginalization over $z$ is intractable, so an approximate posterior $q_\phi(z|c,x)$ is introduced. The objective is to maximize the evidence lower bound (ELBO):

$$\log p_\theta(x|c) \geq \mathbb{E}_{z\sim q_\phi}[\log p_\theta(z, x | c)] - \mathrm{KL}(q_\phi(z|c,x) \,\|\, p_\theta(z|c)).$$

The optimization alternates between:

• E-step: Update $q_\phi$ to approximate the posterior by sampling K candidate rationales per $(c, x)$ and selecting $z^*$ maximizing the joint likelihood $c$0 for supervised learning.
• M-step: Update $c$1 to maximize $c$2, given $c$3 sampled from $c$4.

This EM loop is implemented over seed $c$5 triples, with fine-tuning used for initialization and repeated updates until convergence. The prompt generator $c$6 ultimately synthesizes prompts from concepts to rationales to surface forms. The architecture is domain-agnostic, and learning rates and batch sizes are chosen for stability (e.g., batch size 16, learning rate $c$7, K=8 candidate rationales per instance) (Zhao et al., 24 Sep 2025).

3. Rationale-Guided Prompt Construction

Rationales $c$8 serve as detailed scaffolding, explaining "why" and "how" the concepts combine to yield a problem statement. In PromptCoT 1.0, rationales were created with fixed, hand-crafted prompts; PromptCoT 2.0 uses a learned $c$9 refined through the EM loop. The prompt generator $z$0 learns to produce coherent, diverse, and challenging prompts $z$1 based on concept–rationale pairs $z$2.

This process ensures tight coupling between latent reasoning processes and the final prompts, and allows for sampling multiple diverse rationales per concept to enrich the resulting prompt set. The synthetic corpus achieved through PromptCoT 2.0 is structurally and statistically distinct from prior datasets, confirmed via embedding space analyses (e.g., MDS visualization of problem clusters) (Zhao et al., 24 Sep 2025).

4. Post-Training Regimes: Self-Play and Supervised Fine-Tuning

PromptCoT 2.0 enables two principal post-training setups:

• Self-Play for Strong Models: LLMs (e.g., Qwen3-30B-A3B-Thinking-2507) are further trained exclusively on synthetic data, plus a small real-problem holdout, with automatic verification of generation quality. For math, correctness is determined by majority vote over 8 generations; for code, by synthesized unit tests. Optimization is conducted via Direct Preference Optimization (DPO) or PPO, with easier problems filtered out. This removes dependence on external teacher models and favors autonomous improvement.
• Supervised Fine-Tuning (SFT) for Weaker Models: A student model (e.g., Qwen2.5-7B-Instruct) is trained on a large set of synthetic problems (4.77M prompts) with chain-of-thought rationales and answers generated by a teacher LLM (e.g., GPT-OSS-120B). Training utilizes maximum-likelihood estimation on prompt-to-rationale+answer pairs. PromptCoT 2.0 SFT yields significant improvements in math and code reasoning accuracy compared to PromptCoT 1.0 SFT and models trained on real or hybrid data (Zhao et al., 24 Sep 2025).

5. Empirical Results on Benchmark Tasks

PromptCoT 2.0 establishes new state-of-the-art results at the 30B scale and demonstrates marked improvements in both math and code reasoning tasks. Key metrics include:

Model/Regime	AIME 24	AIME 25	HMMT Feb 25	LCB v5	LCB v6	Codeforces Elo
Baseline (zero-shot) [30B]	87.7	85.0	71.4	68.1	66.0	2044
PromptCoT 2.0 [Self-Play, 30B]	92.1	89.8	76.7	74.2	71.0	2079
Gains	+4.4	+4.8	+5.3	+6.1	+5.0	+35
Qwen2.5-7B (no SFT)	12.8	8.0	2.7	14.7	13.7	706
PromptCoT 1.0 SFT [7B]	71.0	60.2	41.6	47.8	43.6	1645
PromptCoT 2.0 SFT [7B]	73.1	65.6	46.5	53.4	48.9	1815

Additional findings:

• Qwen2.5-72B zero-shot accuracy on sampled math: PromptCoT 2.0 corpus (18.5%), PromptCoT 1.0 (24.7%), OpenMathReasoning (28.9%)—indicating PromptCoT 2.0 corpus is the hardest.
• GPT-OSS-120B average chain-of-thought length: 37.4k tokens for PromptCoT 2.0 prompts vs. 29–30k for other corpora, suggesting increased complexity and diversity.
• Ablations show EM refinement and initialization are both necessary; EM-only or cold-start-only variants yield lower performance (Zhao et al., 24 Sep 2025).

6. Comparative Context: AlignedCoT and the PromptCoT 2.0 Paradigm

AlignedCoT ("PromptCoT 2.0" in the context of in-context prompting) pursues similar rationale alignment principles for few-shot learning: LLMs perform better when chain-of-thought context examples reflect the model's own "native" style rather than human-constructed demonstrations. AlignedCoT implements a three-stage pipeline—probing, refining, and formatting—applied to a small set of question–answer seeds:

1. Probing: Native-style CoT is elicited via zero-shot generation with a fixed prompt.
2. Refining: Iterative error correction aligns reasoning with the ground-truth answer.
3. Formatting: Manual normalization yields uniform CoT style and answer formats.

This procedure reliably yields more error-free, consistent, and fluently formatted in-context exemplars. Experiments on GSM8K, AQUA, SVAMP*, and other reasoning datasets show that AlignedCoT improves few-shot and retrieval-augmented accuracy metrics, establishing consistent gains over manual CoTs (e.g., GPT-3.5-turbo: 80.3% → 83.5%; GPT-4: 89.2% → 90.9% on average) (Yang et al., 2023).

A plausible implication is that scaling rationale-informed prompt synthesis (PromptCoT 2.0) and aligning in-context chains-of-thought to model-native styles (AlignedCoT) represent complementary advances: PromptCoT 2.0 optimizes prompt generation for diversity and hardness via probabilistic EM, while AlignedCoT targets maximal alignment in prompt style and logic for few-shot setups.

7. Applications, Scalability, and Future Directions

PromptCoT 2.0 synthesized 4.77 million prompts spanning mathematics and code, demonstrating that EM-driven rationale-guided prompt synthesis scales efficiently. The pipeline is domain-agnostic and supports generalization to new task domains, including future research areas like multimodal reasoning and agentic LLMs. Open-source code and data are provided for further adoption.

Practical implications include:

• Scalability: The EM loop introduces manageable inference and supervised update overheads, while supporting high-throughput synthetic data generation.
• Resource Efficiency: Shorter chain-of-thought traces in SFT setups reduce inference cost; self-play eliminates the need for proprietary or expert teachers.
• Adoption Prospects: The framework's domain-general design is suitable for integration into a wide range of research pipelines.

Potential research directions encompass:

• Further automation of rationale validation.
• Extension to broader problem domains or modalities.
• Improved style transfer networks for rationale alignment.
• Real-time retrieval and alignment for dynamic adaptive prompting (Zhao et al., 24 Sep 2025, Yang et al., 2023).

PromptCoT 2.0 positions prompt synthesis as a powerful lever for scaling LLM reasoning, providing the dual benefits of heightened training data difficulty/diversity and empirical performance gains on complex reasoning tasks.