CodeBoost: RL-Enhanced Code LLM Training

• CodeBoost is a post-training framework that uses executable Python snippets and a self-supervised RL curriculum to enhance code LLM performance.
• It incorporates maximum-clique curation, bi-directional and error-aware prediction, and heterogeneous augmentation to optimize training efficiency.
• Reinforcement learning with execution-based rewards from stdout and stderr drives consistent performance gains across various code models.

CodeBoost is a post-training framework for code LLMs that seeks to remove a central bottleneck in contemporary code-model alignment: the dependence on curated human instruction–answer pairs. Introduced in “CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL” (Wang et al., 7 Aug 2025), it treats executable Python code snippets as the primary supervision source, synthesizes training tasks from code behavior rather than from human-authored natural-language prompts, and optimizes model behavior with reinforcement learning. Its defining components are maximum-clique curation, bi-directional prediction, error-aware prediction, heterogeneous augmentation, and heterogeneous rewarding, all organized into an RL-based post-training pipeline that learns from code alone (Wang et al., 7 Aug 2025).

1. Motivation and conceptual framing

CodeBoost is motivated by an asymmetry between the abundance of raw code and the scarcity of high-quality coding instructions. Standard supervised fine-tuning and RL-based alignment for coding commonly rely on tuples of the form “human-written coding instruction/question $\rightarrow$ desired answer/code/output,” but the instruction side is costly to author, difficult to scale, and narrower than the supply of executable code in repositories and open datasets. CodeBoost therefore shifts the unit of supervision from manually written instructions to code snippets that can be turned into self-contained learning tasks (Wang et al., 7 Aug 2025).

The framework’s central premise is that executable code snippets, together with their inputs, execution environment, and observed stdout/stderr behavior, can function as an instruction-free curriculum. In this setup, “prompts” are synthesized from the snippet itself rather than sourced from annotation. RL is then used at the post-training stage so that the model explores candidate predictions and receives reward from execution-based feedback rather than from full supervised labels in the usual instruction–answer sense. This distinguishes CodeBoost from standard RLHF and other instruction-based post-training pipelines: its supervision comes from code execution and structured reward rules, not from human preference judgments or human-authored coding instructions (Wang et al., 7 Aug 2025).

The framework combines five components:

Component	Function
Maximum-clique curation	Selects a representative and diverse training corpus from code
Bi-directional prediction	Uses both forward and backward prediction objectives
Error-aware prediction	Learns from both correct and incorrect outputs
Heterogeneous augmentation	Diversifies the training distribution across code and inputs
Heterogeneous rewarding	Scores format correctness and execution feedback from successes and failures

A plausible implication is that CodeBoost reframes post-training as behavior grounding over executable artifacts rather than as imitation of annotated instructions. The paper’s own emphasis, however, remains practical: it aims to improve code LLMs purely from code snippets without relying on human-annotated instructions (Wang et al., 7 Aug 2025).

2. Data pipeline and maximum-clique curation

The implementation in the paper is Python-only. Snippets are collected from OpenCoder, CodeForces-CoTs, and Open-Thoughts-114k, and are organized under three input regimes: no-input code, stdin code, and function-input code. No-input and stdin programs are directly executable, whereas function-input code is preprocessed so that inputs are inserted explicitly before execution. Basic filtering removes snippets that fail execution, are too short, or contain visualization-related components. The supplement specifies exclusion thresholds of fewer than 10 lines or fewer than 30 characters. Sandbox execution is visualization-free, uses a 5-second timeout and an 8 GB RAM limit, and ignores over-limit runs. After filtering and curation, the final training set contains 58k code snippets (Wang et al., 7 Aug 2025).

A key preprocessing step is maximum-clique curation. The paper reports that training on merely filtered snippets gave poor results because of heavy duplication and redundancy, and rejects embedding-based deduplication as potentially weak for code. Instead, it defines a structural distance over line-level tokenized snippets. For two code strings $S_i$ and $S_j$, after applying a split function $f_{\mathrm{split}}$ that converts each string into a set of line-level substrings, the distance is defined as

$$d(S_i, S_j)=\left| ( S_i \cup S_j ) \setminus ( S_i \cap S_j ) \right|, \qquad S_i=f_\mathrm{split}(S_i),\quad S_j=f_\mathrm{split}(S_j).$$

The paper then defines an indicator for whether two snippets are sufficiently different by comparing this symmetric-difference distance against a threshold proportional to the smaller snippet size, with threshold factor $\gamma$. The notation is described as slightly inconsistent in the paper, since $S_i$ and $\mathcal{S}_i$ appear interchangeably, but the intended criterion is explicit (Wang et al., 7 Aug 2025).

From this criterion, CodeBoost constructs a graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$ in which vertices are snippets and edges connect pairs that are sufficiently distinct. The curation target is then a maximum clique: the largest subset in which every pair is mutually adjacent, so every retained pair is sufficiently diverse. Because exact maximum-clique extraction over the full graph is too expensive, the implementation uses divide-and-conquer: randomly split the dataset into $K$ subsets, with maximum subset size 400 in the supplement; find a maximum clique in each subset using NetworkX; union the resulting cliques into an intermediate dataset; and repeat this for $S_i$0 iterations. The reported empirical effect is modest but consistent: removing maximum-clique curation on Qwen2.5-Coder-7B-Instruct lowers total score from 334.6 to 331.2 (Wang et al., 7 Aug 2025).

The resulting procedural pipeline is explicit. It collects Python code snippets and inputs, filters unusable programs, curates a diverse subset via the maximum-clique procedure, transforms retained snippets into self-supervised tasks, augments code and inputs, executes them in a sandbox to obtain ground-truth stdout/stderr, and then optimizes a code LLM with GRPO using hand-designed rewards (Wang et al., 7 Aug 2025).

3. Synthetic tasks, augmentation, and error-aware learning

CodeBoost defines two synthetic RL tasks under the label bi-directional prediction. The forward task gives the model code, input, and environment context, and asks it to predict execution outputs, specifically stdout and stderr. Operationally, for a code snippet $S_i$1 and input $S_i$2, the model predicts $S_i$3, intended as the stdout and stderr after executing $S_i$4. The backward task masks randomly selected lines, asks the model to reconstruct the missing lines, reinserts the predicted lines into the program, executes the reconstructed program under the same setup, and compares the resulting outputs to ground truth. The forward task emphasizes behavior prediction from full code, whereas the backward task emphasizes code recovery under behavioral constraints (Wang et al., 7 Aug 2025).

The two directions are complementary but not equally important in the reported ablations. On Qwen2.5-Coder-7B-Instruct, removing the forward task reduces total score from 334.6 to 330.3, whereas removing the backward task reduces it to 331.8. The authors interpret this as evidence that output-oriented reasoning provides a stronger learning signal than reconstructive completion alone (Wang et al., 7 Aug 2025).

Error-aware prediction is the third major component. Rather than discarding failing programs, CodeBoost always asks the model to predict both stdout and stderr, regardless of whether execution succeeds. During augmentation, modified snippets that become erroneous are retained if the resulting error belongs to a supported built-in Python exception class. The supported errors listed in the supplement are SyntaxError, IndexError, ValueError, NameError, TypeError, KeyError, and ZeroDivisionError. The training signal then includes reward for correctly predicting stderr content. Removing “errors and rewarding stderr” lowers total score from 334.6 to 332.0; using only logical errors yields 333.0, only syntax errors 330.6, and syntax plus logical errors 334.6, while using all errors is worse at 330.5 (Wang et al., 7 Aug 2025).

To broaden the task distribution, CodeBoost applies heterogeneous augmentation before execution. The paper contrasts this with naive character-level corruption, which mostly produces syntax garbage. Its structured augmentations are digit-level augmentation, applied to isolated digits in code and inputs, and logical-level augmentation, which alters operations and conditions such as comparison operators, assignment forms, unary operations, or condition expressions. Logical augmentation is implemented by parsing the concrete syntax tree with libcst, so the perturbations are structurally valid and semantically targeted. This component interacts directly with error-awareness, because augmented code/input pairs are executed to produce new stdout/stderr targets that include both successful and failing traces. Empirically it is the most important ablated component: removing heterogeneous augmentation drops total score from 334.6 to 329.0. Digit augmentation alone reaches 331.5, logical augmentation alone 330.0, and both together 334.6 (Wang et al., 7 Aug 2025).

4. Reward design, prompting, and RL optimization

CodeBoost’s reward design extends standard code RL setups that use a format reward plus a correctness reward. It retains the format reward but decomposes correctness into stdout and stderr components. For the forward task, with predicted stdout/stderr $S_i$5 and ground-truth strings $S_i$6, the reward is

$S_i$7

where

$S_i$8

and

$S_i$9

Here $S_j$0 indicates whether the output is parsable in the required structure, $S_j$1 balances formatting against semantic reward, and $S_j$2 downweights stderr relative to stdout. The explicit hyperparameters are $S_j$3, $S_j$4, and $S_j$5. Thus 10% of the reward budget is allocated to formatting, while 90% is allocated to stdout exactness and stderr similarity, with stderr further downweighted by 0.5. The paper describes stdout reward as hard and stderr reward as soft, since stderr is scored by a Jaccard-style line overlap rather than exact match (Wang et al., 7 Aug 2025).

The backward task uses the same reward structure, except that $S_j$6 and $S_j$7 are obtained by executing the code reconstructed from the model’s predicted masked lines. The rewards are heuristic and rule-based rather than learned. This design matters in the reported comparisons: replacing the rule-based reward with an LLM-based reward model substantially hurts performance, reducing total score from 334.6 to 328.5 (Wang et al., 7 Aug 2025).

Training is prompt-based and execution-centered. The prompt includes the code snippet, input sample if any, environment information, and execution context such as file path or project directory and execution date or time. For the forward task, the required response format is markdown blocks tagged "answer_stdout" and "answer_stderr". For the backward task, the model must output markdown blocks tagged "answer_MASKED_LINE_$ID&quot;</code> for each masked line. Those predicted lines are reinserted into the code and executed in the sandbox (<a href="/papers/2508.05242" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Wang et al., 7 Aug 2025</a>).</p> <p>On the RL side, CodeBoost adopts GRPO via the EasyR1 framework, itself described as a modified version of verl. The paper does not provide the GRPO objective, advantage estimator, KL term, clipping rule, or policy optimization formula, so only the use of GRPO as the policy optimization method is documented. The reported implementation details are AdamW, learning rate $S_j$8, weight decay $S_j$9, group size 5, global batch size 128, rollout batch size 512, and 1 epoch. Training takes about 30 hours on 8 A100-80GB GPUs (Wang et al., 7 Aug 2025).

5. Evaluation, scaling behavior, and ablations

The framework is evaluated on four base code LLMs: Qwen2.5-Coder-7B-Instruct, Llama-3.1-8B-Instruct, Seed-Coder-8B-Instruct, and Yi-Coder-9B-Chat. Additional scaling experiments use Qwen2.5-Coder 1.5B and 3B variants. Benchmarks include BigCodeBench (Hard Complete and Hard Instruct), CRUXEval (Output and Input), MBPP, EvalPlus-MBPP+, and LiveCodeBench (2501–2505 slice). The paper reports benchmark scores as defined by each benchmark and also a summed “Total Perf.” score, rather than standardizing all results into a single pass@$f_{\mathrm{split}}$0 metric (Wang et al., 7 Aug 2025).

The reported gains are consistent across all tested models:

Base model	Baseline total	CodeBoost total
Qwen2.5-Coder-7B-Instruct	327.0	334.6
Llama-3.1-8B-Instruct	242.0	258.1
Seed-Coder-8B-Instruct	356.2	359.6
Yi-Coder-9B-Chat	301.8	306.9
Qwen2.5-Coder-1.5B	211.5	221.7
Qwen2.5-Coder-3B	273.2	281.6

Among the primary models, the largest absolute improvement is on Llama-3.1-8B-Instruct, which rises from 242.0 to 258.1, a gain of 16.1. The strongest baseline, Seed-Coder-8B-Instruct, still improves from 356.2 to 359.6. A supplementary data-scaling study reports that using 25%, 50%, and 100% of the data yields total scores of 330.8, 331.9, and 334.6 respectively, suggesting that the method benefits from more training data (Wang et al., 7 Aug 2025).

The ablation profile is especially informative. Starting from 334.6 on Qwen2.5-Coder-7B-Instruct, performance falls to 331.2 without maximum-clique curation, 330.3 without the forward task, 331.8 without the backward task, 329.0 without heterogeneous augmentation, and 332.0 without errors and stderr reward. This indicates that all five core components contribute, with heterogeneous augmentation exerting the largest single effect and forward prediction the next largest. The paper also reports that the format reward is easiest for the model to optimize, while stdout correctness is hardest. Response length increases during training, which the authors interpret as evidence that the model is engaging in more complex reasoning (Wang et al., 7 Aug 2025).

6. Scope, limitations, and significance

CodeBoost is explicitly designed for settings where raw code is abundant but instruction data is limited. It is therefore best suited to repository- and dataset-scale collections of executable code, especially when a sandboxed execution environment is available. Its supervision mechanism depends on execution traces, so its practical domain is bounded by executability and by the availability of inputs or input reconstruction procedures (Wang et al., 7 Aug 2025).

The framework also has explicit limitations. In the reported study it is Python-only. It assumes snippets are executable or can be made executable with provided inputs. It excludes visualization-centric code and, more broadly, the paper states that it currently cannot handle GUI/visualization-heavy or multimodal coding tasks effectively. The reward design is hand-crafted, relies on exact stdout matching and line-overlap stderr scoring, and may therefore fail to capture semantic equivalence for more open-ended behaviors. The maximum-clique curation procedure is approximate and based on a simple line-set distance rather than richer semantic similarity. In addition, the paper does not expose the precise GRPO loss, advantage estimation, or rollout details beyond core hyperparameters, leaving some optimization specifics underdocumented (Wang et al., 7 Aug 2025).

Within those constraints, CodeBoost’s significance lies in its demonstration that executable code snippets can be converted into an instruction-free RL curriculum. The method curates diverse code, asks the model to reason in forward and backward directions, preserves failure traces as supervision rather than discarding them, augments code semantically rather than randomly, and rewards format, stdout, and stderr jointly. This suggests a broader research direction in which code understanding and code generation are aligned through executable behavior instead of through manually authored instruction corpora. The empirical gains are described in the paper as broad and consistent rather than dramatic, but they establish the practical viability of code-only post-training for code LLMs (Wang et al., 7 Aug 2025).