Combinatorial Synthesis: Scaling Code RLVR via Atomic Decomposition and Recombination

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as the cornerstone for shaping the remarkable coding abilities of LLMs. However, the scalability of RLVR is severely constrained by the scarcity of sufficiently challenging verifiable code tasks that target near the model's edge of competence. Prior studies often rely on heuristic seed expansions for data synthesis, which severely limits both novelty and difficulty. Consequently, the training value of such data fails to scale proportionally with the size of its synthesis. To this end, we propose Atomic Decomposition and Recombination (ADR), a novel framework that generates verifiable code tasks via decomposition into atomic elements and controlled recombination, thereby enabling the generation of genuinely novel and challenging verifiable code tasks. Experiments and analysis demonstrate that ADR achieves superior originality, difficulty, diversity, and test quality over existing baselines, and consistently delivers greater improvements in code ability across RLVR in diverse downstream domains, including algorithmic programming, tool usage, and data science. Our work sheds light on a new paradigm for novel code task synthesis and scalable RLVR training.

• The paper introduces Atomic Decomposition and Recombination (ADR), which uses entropy- and mutual-information-guided schemas, controlled recombination, executable validation, and adversarial test refinement to generate frontier-level code tasks.
• ADR substantially improves synthetic data quality, raising originality to 28.91, difficulty to 71.89, diversity to 84.36, and test quality to 81.36, while ASSR increases average test coverage by 135.8%.
• ADR-trained models outperform real and synthetic-data baselines across code benchmarks, reaching 25.76% average Pass@1 on LiveCodeBench v5/v6 for Qwen2.5-Coder-7B and increasing Pass@8 by 4.79%, suggesting genuine capability expansion rather than sampling gains alone.

Motivation: the data bottleneck in code RLVR

Reinforcement Learning with Verifiable Rewards (RLVR) has become the dominant post-training paradigm for improving the coding capabilities of LLMs, with deterministic unit tests providing reliable reward signals. The effectiveness of this paradigm, however, depends critically on a supply of sufficiently challenging verifiable tasks positioned near the model's edge of competence. The authors argue that manually curated datasets cannot scale to meet this demand, and that existing synthesis pipelines—Evol-Instruct-style recursive prompting, KodCode, OSS-Instruct, Educational Instruct—are structurally incapable of filling the gap. Their diagnosis is precise: heuristic expansion methods preserve the compositional structure of seed problems, so they increase linguistic surface diversity without expanding logical diversity or difficulty. The practical consequence is premature reward saturation during RLVR, where training value fails to scale with corpus size.

The ADR framework

Atomic Decomposition and Recombination (ADR) replaces seed expansion with an explicit model of code tasks as compositions of orthogonal logical primitives. The pipeline proceeds in five stages:

1. Element extraction. An LLM induces a task-specific element schema $\mathcal{S} = \{e_1, \dots, e_n\}$ from seed instances. For algorithmic tasks the schema comprises elements such as core algorithmic idea, story background, strategy diversity, and difficulty level; tool-usage and data-science tasks receive analogous five-element schemas (e.g., computational objective, tool dependency set, processing logic constraints, input interface, output specification).
2. Info-guided schema optimization (ESO). Elements are embedded with all-MiniLM-L6-v2, clustered via K-Means, and scored with two information-theoretic signals: per-element entropy $H(e_i)$ over semantic clusters, which drives split/merge operations, and conditional mutual information $I(e_i; q \mid e_j)$, which filters redundant elements and prioritizes those contributing task complexity. An LLM applies structured add/remove/split/merge/redefine operations until average schema entropy converges.
3. Controlled recombination. Generation is anchored on a core element selected for high information content and low coupling with other elements. Conditioned on this anchor plus three exemplar combinations, the LLM samples new combinations, avoiding contradictory or unsolvable configurations.
4. Template-based synthesis and execution-grounded validation. Problems are instantiated through constrained templates specifying description, I/O format, and constraints. Each problem receives a reference solution and a programmatic test case generator; only problems whose solutions pass all generated tests in an isolated sandbox are retained.
5. Adversarial solution space refinement (ASSR). The framework generates near-miss solutions—plausible programs flawed by edge-case errors—and iteratively refines the test generator until the near-miss pass rate $R(\mathcal{V}, \mathcal{T})$ converges.

The component ablations support each stage. ESO iterations raise problem validity from 35.0% to 43.0% while diversity improves monotonically across three iterations. On 300 sampled LiveCodeBench-v5 instances, single-round generation yields reference solutions passing ground-truth tests at 90.62%. ASSR applied to 5K tasks increases average test case count from 14.75 to 34.78 (+135.8%) and test quality score from 72.91 to 81.36 (+11.6%).

Data quality evaluation

The paper introduces a four-dimensional taxonomy—originality (fraction of samples below cosine similarity $\tau = 0.6$ against a 144K-problem reference set), difficulty (one minus mean performance of Qwen3-4B/8B/14B reference models), diversity (uniformity of nearest-neighbor distances), and test quality (LLM-judged coverage). Under this taxonomy ADR substantially outperforms both KodCode and Educational Instruct:

Method	Originality	Difficulty	Diversity	Test Quality
KodCode	1.78	17.92	72.75	29.91
Educational Instruct	6.04	20.14	46.17	37.82
ADR	28.91	71.89	84.36	81.36

The originality gap is the most striking figure: ADR's 28.91 versus 6.04 for the strongest baseline indicates that recombination produces problems genuinely outside the reference distribution rather than paraphrases of it. t-SNE density analysis against KodCode, both seeded from TACO, shows ADR covering a broader manifold extending into long-tail regions, though the authors concede that some KodCode-only regions are not covered by ADR; notably, 47.5% of KodCode's unique samples are simple function-level completions, whereas ADR concentrates on instruction-style algorithmic tasks. In a controlled 100-step RL comparison on identical seeds, ADR-only data lifts LCB-v5 Pass@1 from 16.17 to 20.28 versus 17.89 for KodCode-only data.

RLVR results

Training uses GRPO for 10 epochs (global batch 128, 8 rollouts per question, learning rate $10^{-6}$), with 5,000 training examples per method synthesized by DeepSeek-V3.2 from 1,710 verified medium-and-above TACO seeds. Evaluation covers LiveCodeBench v5/v6, BigCodeBench, and DS-1000 across three base models.

A central empirical claim is that prior synthetic methods fail to beat real-data fine-tuning. On Qwen2.5-Coder-7B-Instruct, Educational Instruct averages 20.66% versus 23.23% for real TACO data; KodCode reaches only 23.16%, statistically indistinguishable from TACO. On Qwen3-8B, both synthetic baselines underperform TACO by 5.70% and 6.80% relatively, and on Llama-3.1-8B-Instruct, TACO and Educational Instruct suffer reward saturation with unstable gradients, degrading below the base model. This is a strong, somewhat contrarian result: it implies that scale of synthetic data alone provides no RLVR benefit unless difficulty targets the capability frontier.

ADR, by contrast, delivers consistent gains everywhere, all statistically significant at $p < 0.001$ (McNemar's test):

Model / Data	LCB-v5	LCB-v6	Average
Qwen2.5-Coder-7B base	16.17	20.21	18.19
+ TACO (real)	22.60	23.86	23.23
+ KodCode	22.75	23.57	23.16
+ ADR	25.37	26.14	25.76
Llama-3.1-8B base	9.36	15.71	12.54
+ ADR	16.84	23.00	19.92
Qwen3-8B base	22.53	21.21	21.87
+ ADR	35.85	31.43	33.64

On tool usage and data science, ADR also surpasses KodCode on BigCodeBench (41.67 vs. 41.27) and DS-1000 (42.44 vs. 39.05), demonstrating cross-domain transfer from Package Instruct seeds despite those seeds lacking verifiable signals natively.

The paper's most consequential analysis concerns whether synthetic RLVR data expands intrinsic capability or merely sharpens sampling. Measured by Pass@8 on LCB-v5, ADR-trained models improve from 28.74% to 33.53% (+4.79%) as sampling budget grows, while the best baseline gains only +0.60%. Following prior arguments that Pass@k scaling reflects genuine reasoning-capacity expansion, the authors interpret this as evidence that ADR targets boundary capabilities rather than reinforcing already-mastered skills. Training dynamics corroborate this: ADR sustains actor gradient norms (~0.25 plateau) where baselines decay, achieves larger cumulative reward improvement ($\Delta = 0.45$ vs. 0.25 for KodCode), and drives KL divergence from the SFT policy to ~0.14 versus ~0.08, indicating deeper policy exploration rather than convergence into local optima.

Case studies illustrate the mechanism concretely: given a Hamming-distance-over-sliding-windows seed, heuristic expansion preserves the core operation with altered constraints, while ADR shifts data types (binary strings to integer arrays) and algorithmic paradigms (prefix sums to monotonic queues), producing structurally novel requirements.

Limitations and open questions

The authors acknowledge that evaluation is confined to specific benchmarks and roughly 7–8B parameter models; scaling behavior at larger foundation-model scales and in multilingual settings remains unverified. The framework is demonstrated on single-turn code generation, leaving multi-turn agentic scenarios such as automated software engineering unexplored. Two further caveats bear on interpretation: the validity rate of synthesized problems plateaus around 43% even after schema optimization, meaning more than half of generated candidates are discarded—a nontrivial compute cost whose amortization at larger scale is unquantified—and the difficulty metric depends on a fixed panel of Qwen3 reference models, so difficulty estimates may not transfer to models with different capability profiles. Whether the entropy/CMI-driven schema optimization remains beneficial as schemas saturate beyond three iterations is also left open.

Conclusion

ADR reframes verifiable code data synthesis as combinatorial exploration over an information-theoretically optimized element space rather than heuristic expansion of seed problems. The empirical record supports its central claims: substantial margins over both synthetic and real-data baselines across three model families and three code domains, large improvements in originality and difficulty metrics, and—most importantly—Pass@8 gains indicating genuine capability-frontier expansion rather than sampling-density effects. The open questions are chiefly about scale: whether the synthesis-to-validity ratio and the difficulty-targeting property persist for larger models and multi-turn agent workloads.