QueST: Hard Problem Generation Framework

• QueST is a two-stage training framework for an LLM problem generator that creates challenging coding problems using graph sampling and difficulty-based fine-tuning.
• It employs a difficulty-aware graph to prompt problem generation and selects the hardest candidate using model-based scoring.
• The synthetic problems from QueST enhance downstream model performance on competitive coding benchmarks via chain-of-thought distillation and reinforcement learning.

QueST is a two-stage framework for training a LLM problem generator that specializes in producing challenging coding problems and, by extension, more general reasoning problems. It was introduced to address two constraints in contemporary reasoning-model training: scalability limits imposed by human-labeled datasets, and the lack of large-scale, difficult coding-problem corpora. Existing competitive coding datasets contain only thousands to tens of thousands of problems, and prior synthetic-data pipelines typically either augment instruction datasets or select challenging examples from human-written data. QueST instead directly optimizes specialized generators through difficulty-aware graph sampling and difficulty-aware rejection fine-tuning, then uses the resulting synthetic problems for chain-of-thought distillation or reinforcement learning (Hu et al., 20 Oct 2025).

1. Problem formulation and overall design

QueST is organized as a two-stage training procedure for an LLM “problem generator.” In the first stage, it constructs a graph whose nodes are “concepts” extracted from a seed problem set, and whose edges encode both co-occurrence frequency and human-annotated difficulty. Random walks on this graph produce concept sets that are turned into prompts. In the second stage, the generator produces multiple candidate problems for each prompt; each candidate is scored by an explicit difficulty proxy based on self-consistency among multiple model solutions; only the hardest candidate is retained for fine-tuning (Hu et al., 20 Oct 2025).

This design differs from synthetic-data methods that depend primarily on augmenting existing instruction corpora or mining difficult problems from human-labeled sources. In QueST, the generator itself is specialized to create difficult problems. The paper’s central claim is that alternating graph-based prompt sampling with rejection fine-tuning on the hardest generated candidates produces a generator that outperforms even GPT-4o at creating difficult, downstream-useful problems (Hu et al., 20 Oct 2025).

The seed data are taken from TACO, with $Q_{\text{seed}}=$ TACO containing $25\text{K}$ problems and human difficulty labels. Concepts $C$ are extracted via LLM prompts, and the resulting graph is used to sample $100\text{K}$ concept prompts. These prompts support large-scale synthesis without relying on new human annotation (Hu et al., 20 Oct 2025).

2. Difficulty-aware graph sampling

The graph-sampling stage begins with an undirected graph $G=(C,E)$, where $C$ is the set of concepts extracted from the seed corpus. For each edge $(u,v)$, QueST defines a weight that combines concept co-occurrence and human difficulty:

$$w(u,v) =\log\bigl(\alpha\cdot\mathrm{freq}(u,v)+(1-\alpha)\cdot\mathrm{diff}(u,v)+\varepsilon\bigr),$$

where $\mathrm{freq}(u,v)$ is the number of co-occurrences of $u$ and $25\text{K}$0 in $25\text{K}$1, $25\text{K}$2 is the average human difficulty over problems containing both concepts, $25\text{K}$3 balances frequency versus difficulty, and $25\text{K}$4 is a small constant for numerical stability. In the reported experiments, $25\text{K}$5 (Hu et al., 20 Oct 2025).

The pairwise difficulty term is defined over the subset $25\text{K}$6 as

$25\text{K}$7

with $25\text{K}$8. This gives the graph a mixed statistical and pedagogical structure: frequently co-occurring concept pairs are preserved, but hard concept pairs receive additional weight (Hu et al., 20 Oct 2025).

Concept prompts are obtained through random walks of up to $25\text{K}$9 steps. From node $C$0, the one-step transition probability to a neighbor $C$1 is

$C$2

The resulting sampled concept sequence $C$3 is instantiated as a prompt $C$4. This mechanism biases prompt construction toward concept combinations that are both common enough to be learnable and difficult enough to be valuable for hard-problem generation. A plausible implication is that the graph functions as a structured curriculum over concept combinations rather than as a flat sampling distribution.

3. Difficulty estimation and rejection fine-tuning

For each sampled prompt $C$5, the generator produces $C$6 candidate problems, with $C$7 in the reported data-generation pipeline. The core selection mechanism is a model-based difficulty estimate $C$8 defined through self-consistency among multiple solutions (Hu et al., 20 Oct 2025).

Given a generated problem $C$9, QueST estimates difficulty by generating $100\text{K}$0 test inputs with GPT-4o, generating $100\text{K}$1 solution candidates with GPT-4o, executing each solution on each test input, and measuring disagreement among the resulting outputs. If $100\text{K}$2 is the set of outputs obtained on test input $100\text{K}$3, and $100\text{K}$4 is the majority-voted output, then

$100\text{K}$5

A higher $100\text{K}$6 indicates greater disagreement among solutions and therefore greater estimated difficulty (Hu et al., 20 Oct 2025).

Difficulty-aware rejection fine-tuning then selects the hardest candidate. If $100\text{K}$7 is the generator with parameters $100\text{K}$8, and $100\text{K}$9, the retained problem is

$G=(C,E)$0

The pair $G=(C,E)$1 is added to the hard training set $G=(C,E)$2, and the generator is fine-tuned by maximizing the likelihood of $G=(C,E)$3 given $G=(C,E)$4, equivalently minimizing cross-entropy:

$G=(C,E)$5

This procedure makes the rejection criterion explicitly difficulty-sensitive rather than quality-sensitive in a generic sense. The paper’s interpretation is that QueST directly optimizes generators to create challenging coding problems, rather than merely plausible or stylistically correct ones (Hu et al., 20 Oct 2025).

4. Models, training protocol, and large-scale synthesis

The reported generator base checkpoint is Qwen2.5-14B-Instruct, with experiments also probing GPT-4o; after rejection fine-tuning, the specialized generator is approximately $G=(C,E)$6. For downstream distillation, the student model is Qwen3-8B-Base and the teacher is Qwen3-235B-A22B. Inference uses vLLM with $G=(C,E)$7. Supervised fine-tuning uses $G=(C,E)$8 and $G=(C,E)$9. Reinforcement learning uses the GRPO algorithm in VeRL with $C$0, $C$1, and $C$2 (Hu et al., 20 Oct 2025).

The synthesis pipeline proceeds from TACO as the seed corpus. Concepts are extracted, the difficulty-aware graph is built, and $C$3 concept prompts are sampled. For each prompt, $C$4 candidate problems are generated, difficulty is measured via $C$5, and only the hardest candidate is kept. Generated problems are then filtered to remove any $C$6 for which more than $C$7 of test executions fail (Hu et al., 20 Oct 2025).

Two downstream uses are emphasized. In chain-of-thought distillation, each synthetic $C$8 is paired with a long chain-of-thought solution from Qwen3-235B-A22B, producing $C$9 $(u,v)$0 pairs for supervised fine-tuning of Qwen3-8B-Base. In reinforcement learning, Qwen3-8B itself is used to majority-vote pseudo-labels on test cases, invalid test cases are discarded, and GRPO is applied to fine-tune Qwen2.5-7B on $(u,v)$1 QuEST synthetic problems versus baselines (Hu et al., 20 Oct 2025).

A central practical feature is that the difficulty proxy is execution-based and therefore verifiable. This matters because the framework does not treat hardness as a purely stylistic judgment; it ties hardness to disagreement under executable solution attempts.

5. Distillation, reinforcement learning, and quantitative performance

The distillation results on LiveCodeBench-V5 and USACO show that QueST-generated synthetic data can improve a smaller reasoning model beyond its base checkpoint and beyond a human-repeat baseline (Hu et al., 20 Oct 2025).

Model	LCB-Avg	USACO-Avg
Qwen3-8B (base)	58.7	43.5
OCR-8B (112K human repeats)	56.5	41.3
QueST-100K-8B (100K synth)	59.4	43.5
QueST-8B (100K + 112K)	65.2	49.9
DeepSeek-R1-671B (closed)	65.6	56.2

After fine-tuning Qwen3-8B-Base on $(u,v)$2 difficult problems generated by QueST, the model surpasses the performance of the original Qwen3-8B on LiveCodeBench. With an additional $(u,v)$3 examples, described as $(u,v)$4 human-written problems paired with multiple synthetic solutions, the resulting 8B model matches the performance of DeepSeek-R1-671B on LiveCodeBench (Hu et al., 20 Oct 2025).

The hard–medium–easy breakdown further indicates that gains are concentrated at higher difficulty levels. QueST-8B attains $(u,v)$5 on Easy, $(u,v)$6 on Medium, and $(u,v)$7 on Hard, compared with the base Qwen3-8B at $(u,v)$8. This suggests that the synthetic problems are not merely increasing volume; they are shifting the learned competence profile toward harder instances (Hu et al., 20 Oct 2025).

In RLVR, QuEST synthetic data also outperforms alternative synthetic or seed-data sources. Qwen2.5-7B-Instruct scores $(u,v)$9 LCB-Avg, rising to $$w(u,v) =\log\bigl(\alpha\cdot\mathrm{freq}(u,v)+(1-\alpha)\cdot\mathrm{diff}(u,v)+\varepsilon\bigr),$$0 with RL on TACO, $$w(u,v) =\log\bigl(\alpha\cdot\mathrm{freq}(u,v)+(1-\alpha)\cdot\mathrm{diff}(u,v)+\varepsilon\bigr),$$1 with RL on MathScale synth, and $$w(u,v) =\log\bigl(\alpha\cdot\mathrm{freq}(u,v)+(1-\alpha)\cdot\mathrm{diff}(u,v)+\varepsilon\bigr),$$2 with RL on QuEST synth. The reported conclusion is that QueST-generated problems are effective in both supervised distillation and reinforcement learning settings (Hu et al., 20 Oct 2025).

6. Ablations, interpretation, and broader significance

Ablation studies isolate the contributions of the two main components. A baseline generator with a uniform graph achieves $$w(u,v) =\log\bigl(\alpha\cdot\mathrm{freq}(u,v)+(1-\alpha)\cdot\mathrm{diff}(u,v)+\varepsilon\bigr),$$3 LCB-Avg. Adding the difficulty-aware graph raises this to $$w(u,v) =\log\bigl(\alpha\cdot\mathrm{freq}(u,v)+(1-\alpha)\cdot\mathrm{diff}(u,v)+\varepsilon\bigr),$$4, and adding rejection fine-tuning yields the full QueST score of $$w(u,v) =\log\bigl(\alpha\cdot\mathrm{freq}(u,v)+(1-\alpha)\cdot\mathrm{diff}(u,v)+\varepsilon\bigr),$$5. The quantitative gap is modest between the latter two settings, but both components contribute measurably, and the complete framework performs best (Hu et al., 20 Oct 2025).

These ablations help clarify a common misconception about synthetic hard-problem generation: difficulty is not obtained solely by post hoc filtering of outputs. In QueST, prompt construction is itself difficulty-aware through the concept graph, and output selection is difficulty-aware through rejection fine-tuning. The framework therefore modifies both the prompt distribution and the generator parameters.

The paper also emphasizes broader applicability. QueST can be used to generate arbitrarily large, curriculum-tuned problem sets for new reasoning modalities such as math or logical puzzles by swapping the seed set and the difficulty metric. Because $$w(u,v) =\log\bigl(\alpha\cdot\mathrm{freq}(u,v)+(1-\alpha)\cdot\mathrm{diff}(u,v)+\varepsilon\bigr),$$6 is verifiable through execution, the approach extends naturally to tasks with an automatic checker. The authors further suggest that, by replacing supervised fine-tuning teachers or integrating real-time difficulty rewards, QueST could support continual learning pipelines that keep “raising the bar” as models improve (Hu et al., 20 Oct 2025).

In that sense, QueST is best understood not simply as a data-generation heuristic, but as a training framework for specialized hard-problem generators. Its significance lies in reframing synthetic data creation as an optimization problem over difficulty, with direct downstream consequences for competitive coding and reasoning models.