Qwen2.5-Coder-C3: Controllable Code Completion

• Qwen2.5-Coder-C3 is a supervised fine-tuned model specialized for controllable code completion, generating missing code that adheres to explicit implementation instructions.
• It uses a two-phase data synthesis pipeline with 200K synthetic instruction-completion pairs, significantly improving scale-control and instruction-following metrics.
• The model is benchmarked with C³-Bench, which evaluates both functional correctness and precise adherence to developer-specified instructions, mimicking real-world coding workflows.

Qwen2.5-Coder-C3, written in the source paper as Qwen2.5-Coder-C$^3$, is a supervised fine-tuned variant of Qwen2.5-Coder designed for controllable code completion: the task of generating missing code between a prefix and suffix while also obeying an explicit user instruction about implementation strategy, structure, control flow, parameter usage, or completion scope. It was introduced together with C$^3$-Bench, an instruction-guided code completion benchmark intended to measure not only functional correctness but also instruction adherence during infilling, a setting the authors argue is closer to practical developer workflows than conventional completion benchmarks (Zhang et al., 22 Jan 2026).

1. Definition and model identity

Qwen2.5-Coder-C$^3$ is not a generic Qwen2.5-Coder checkpoint with a new suffix, but a task-specialized model for controllable code completion. The paper fine-tunes two variants, Qwen2.5-Coder-1.5B-C$^3$ and Qwen2.5-Coder-32B-C$^3$, using synthetic instruction-completion pairs. The immediate motivation is that real code-completion usage often involves explicit requirements such as “use BFS,” “complete only the loop body,” or “generate exactly 3 lines,” whereas standard completion benchmarks largely ignore whether those instructions are followed (Zhang et al., 22 Jan 2026).

The paper distinguishes this model from ordinary base and instruct checkpoints. Several instruct/chat variants perform worse than their base code models on code completion, especially in fill-in-the-middle settings. This motivates a completion-specific alignment strategy rather than relying on generic instruction tuning alone. In that sense, Qwen2.5-Coder-C$^3$ is best understood as a specialized infilling model whose defining property is controllability rather than broad conversational coding assistance (Zhang et al., 22 Jan 2026).

This positioning also matters historically. The Qwen2.5-Coder technical report introduces Qwen2.5-Coder-$$(0.5\mathrm{B},1.5\mathrm{B},3\mathrm{B},7\mathrm{B},14\mathrm{B},32\mathrm{B})$$ and corresponding Instruct variants, but does not mention any model called C3. The C$^3$ models therefore represent a later task-specific derivative rather than an official member of the original release taxonomy (Hui et al., 2024).

2. Controllable code completion as a task

The paper formalizes standard code completion as a tuple $(P,S,G,T)$, where $P$ is prefix code, $^3$0 is suffix code, $^3$1 is the ground-truth middle code, and $^3$2 is a unit-test set. Controllable Code Completion augments this with an instruction $^3$3, yielding $^3$4, and the target mapping $^3$5 (Zhang et al., 22 Jan 2026).

Two task families are defined. The first is Implementation-Control Completion (ICC), where the generated middle code must be functionally correct and must satisfy an implementation instruction. ICC instructions are divided into four categories: Structural Specification Requirements, Algorithmic Implementation Requirements, Control Flow Requirements, and Critical Parameter Requirements. The second is Scale-Control Completion (SCC), where the objective is to control how much code is generated rather than to produce a full implementation. SCC contains Line Span Completion, Multi-line Completion, and Statement Block Completion for IF, FOR, and WHILE blocks (Zhang et al., 22 Jan 2026).

This decomposition is central to the model’s design. ICC isolates cases where multiple functionally correct middles may exist but only one matches the requested implementation pattern. SCC isolates a different failure mode: over-generation of surrounding code, comments, or explanations when only a tightly bounded insertion is desired. The paper treats these as distinct controllability problems rather than variants of ordinary pass@1 code generation (Zhang et al., 22 Jan 2026).

A plausible implication is that Qwen2.5-Coder-C$^3$6 is optimized for developer-facing infilling systems in which correctness alone is insufficient. In such settings, the model must choose among valid implementations and must respect output granularity constraints, both of which are only weakly measured by conventional completion benchmarks.

3. C$^3$7-Bench: benchmark construction and evaluation

C$^3$8-Bench contains 2,195 Python instances, split into 1,286 ICC tasks and 909 SCC tasks. Within ICC, the benchmark includes 111 Structural tasks, 502 Algorithmic tasks, 547 Control-Flow tasks, and 126 Parameter tasks. Within SCC, it includes 97 Span tasks, 467 Multi-Lines tasks, and 345 Statement-Block tasks. The benchmark is derived from HumanEval and SAFIM, with larger middle spans extracted and multiple equivalent implementations constructed for shared prefix-suffix contexts (Zhang et al., 22 Jan 2026).

The construction pipeline has four stages: middle code extraction, equivalent implementation generation, data filtering, and instruction generation. Middle code extraction uses AST-based processing through tree-sitter-languages; in SCC, 30% of examples additionally mask 3–5 consecutive lines. For ICC, equivalent implementations are generated with GPT-4o-2024-11-20, Claude3.5-Sonnet-20241022, DeepSeek-V3, and Qwen2.5-Coder-32B-Instruct, and only implementations that pass all unit tests are retained. Over 50% of ICC cases contain three distinct implementations for the same context (Zhang et al., 22 Jan 2026).

Benchmark filtering includes readability and PEP8 checks, a length constraint requiring middle code to occupy at most 30% of total context, significant implementation diversity, and algorithmic efficiency. ICC instructions are manually crafted, whereas SCC instructions are generated by Claude3.5-Sonnet. Instruction quality is validated through expert review by five senior Python developers and automated consistency checking with Claude3.5-Sonnet (Zhang et al., 22 Jan 2026).

Evaluation uses three metrics. Pass@1 applies to ICC and is determined by unit tests. Instruction-Following Rate (IF) is the main controllability metric: for ICC it is measured only among functionally correct outputs, and for SCC it is measured through automatic structural verification. Edit Similarity (ES) is a supplementary static metric. ICC adherence is judged by Claude3.5-Sonnet using a structured prompt; the paper reports 98% agreement between this judge and senior Python developers across 10 independent assessment rounds. SCC verification is automatic, using AST-based node-type matching for statement-block tasks and length-based checks for line-count requirements (Zhang et al., 22 Jan 2026).

4. Data synthesis and supervised fine-tuning

The C$^3$9 training corpus is synthesized from Python code in GitHub repositories sourced from data associated with The Stack v2 / StarCoder2. The synthesis process is explicitly two-phase. First, Claude3.5-Sonnet generates 1,000 high-quality instruction-completion pairs through middle-code extraction and instruction generation. These seed examples are then used as few-shot demonstrations for large-scale synthesis with Qwen2.5-Coder-32B-Instruct, which performs middle-code extraction and instruction generation at scale (Zhang et al., 22 Jan 2026).

The final supervised fine-tuning dataset contains 200,000 synthetic instruction-completion pairs. The paper states that extracted middle segments are validated with pattern matching to ensure extraction accuracy, and that 10-gram decontamination against C$^3$0-Bench is performed before training. Compared with the benchmark-construction pipeline, the training-data pipeline is simpler and more automated; the paper does not claim the same level of manual curation, diversity filtering, or efficiency checking for the 200K training set (Zhang et al., 22 Jan 2026).

Fine-tuning is performed on Qwen2.5-Coder-1.5B and Qwen2.5-Coder-32B. The reported hyperparameters are Adam, learning rate $^3$1, 50 warmup steps, global batch size 1024 samples, tensor parallel size 2, and sequence truncation at 4K tokens. Training uses 64 NVIDIA A100-80GB GPUs. Open-source inference in experiments uses vLLM, and decoding is greedy with a 1024-token generation limit (Zhang et al., 22 Jan 2026).

Because Qwen2.5-Coder supports fill-in-the-middle, evaluation for Qwen-family models uses FIM formatting rather than chat-only prompting. This matters because the paper’s central claim is not merely that instruction tuning improves coding in the abstract, but that completion-specific instruction tuning in the correct serialization regime improves controllable infilling in particular (Zhang et al., 22 Jan 2026).

5. Empirical performance

The principal result is that Qwen2.5-Coder-32B-C$^3$2 substantially improves controllable completion relative to both the base and instruct Qwen2.5-Coder-32B checkpoints. On C$^3$3-Bench, it reaches ICC ES 49.3, ICC Pass@1 62.0, ICC IF 52.5, SCC ES 44.2, SCC IF 80.7, and overall average IF 66.6. Relative to Qwen2.5-Coder-32B-Instruct, the gains are especially large on SCC IF, which rises from 16.9 to 80.7, and on average IF, which rises from 22.8 to 66.6 (Zhang et al., 22 Jan 2026).

Model	ICC Pass@1 / ICC IF	SCC IF / Avg IF
Qwen2.5-Coder-32B	58.1 / 38.7	5.2 / 21.9
Qwen2.5-Coder-32B-Instruct	49.8 / 28.8	16.9 / 22.8
Qwen2.5-Coder-32B-C$^3$4	62.0 / 52.5	80.7 / 66.6
o1-2024-12-17	72.1 / 62.9	59.6 / 61.3

The same pattern appears at smaller scale. Qwen2.5-Coder-1.5B-C$^3$5 achieves ICC ES 44.7, ICC Pass@1 39.7, ICC IF 29.6, SCC ES 40.4, SCC IF 66.8, and average IF 48.2. Compared with Qwen2.5-Coder-1.5B-Instruct, ICC Pass@1 increases from 22.9 to 39.7, ICC IF from 0.7 to 29.6, SCC IF from 8.0 to 66.8, and average IF from 4.3 to 48.2 (Zhang et al., 22 Jan 2026).

The paper’s comparative claim is narrower and more interesting than a generic “state of the art” statement. Qwen2.5-Coder-32B-C$^3$6 does not exceed the strongest proprietary models on ICC Pass@1 or ICC IF, but it achieves the best SCC IF in the entire table at 80.7, and its average IF of 66.6 is higher than all listed proprietary models, including o1-2024-12-17 at 61.3. This indicates that the model’s strongest advantage lies in precise scale control rather than absolute dominance on all implementation-control tasks (Zhang et al., 22 Jan 2026).

On conventional benchmarks, performance retention is mixed rather than uniformly lossless. On CrossCodeEval, Qwen2.5-Coder-32B scores average EM 57.1 and ES 86.8, while Qwen2.5-Coder-32B-C$^3$7 scores 55.9 and 86.2. On RepoEval, the base model scores average EM 51.6 and ES 78.5, while C$^3$8 scores 51.8 and 77.0. On CrossCodeLongEval, the base model scores average EM 36.9 and ES 66.4, whereas C$^3$9 drops to 29.1 and 60.5. On SAFIM, average performance declines from 71.2 to 67.5. The paper therefore supports a targeted trade-off interpretation: strong gains in controllability, modest degradation on some standard completion settings, and larger degradation on some long-context or API-heavy tasks (Zhang et al., 22 Jan 2026).

An ablation in which instructions are removed from ICC prompts shows that IF degrades substantially while Pass@1 remains largely unchanged and can even improve slightly. This supports the benchmark’s central claim that controllability is not reducible to correctness and that instruction-sensitive completion must be evaluated separately (Zhang et al., 22 Jan 2026).

6. Relation to the broader Qwen2.5-Coder research line

Qwen2.5-Coder-C$^3$0 sits within a broader Qwen2.5-Coder ecosystem but addresses a more specific objective than most related work. The original Qwen2.5-Coder series is a code-specialized continuation of Qwen2.5, with dense models from 0.5B to 32B, explicit FIM support, repository-oriented tokens such as <|repo_name|> and <|file_sep|>, and a training pipeline spanning file-level pretraining, repo-level pretraining, and instruction tuning on over 5.5 trillion tokens. That earlier report emphasizes code generation, completion, reasoning, repair, and repository-level understanding, but does not define controllable completion as a distinct benchmarked task (Hui et al., 2024).

At the family level, the Qwen2.5 technical report describes Qwen2.5-Coder as a specialized descendant of Qwen2.5 and states that Qwen2.5 post-training incorporates instruction tuning data from Qwen2.5-Coder across nearly 40 programming languages, using static checking and automated unit testing in a multilingual sandbox. This provides the broader training philosophy from which task-specific derivatives such as C$^3$1 can emerge: code-centric supervision, executable verification, and specialized post-training layered on top of a general Qwen2.5 foundation (Qwen et al., 2024).

Two nearby lines of work illustrate what Qwen2.5-Coder-C$^3$2 is not. First, rStar-Coder improves Qwen2.5-Coder Instruct checkpoints through supervised fine-tuning on a large verified dataset of 418K competition-level problems and 580K long-reasoning solutions; its focus is verified code reasoning rather than controllable infilling (Liu et al., 27 May 2025). Second, CURE co-evolves coding and unit-test generation via reinforcement learning and derives ReasonFlux-Coder models from Qwen2.5-Instruct rather than Qwen2.5-Coder-Instruct; it is a post-training method for execution-based code generation and test-time scaling, not a completion-specific instruction-following model (Wang et al., 3 Jun 2025).

These distinctions matter because “C3” can easily be misread as a general Qwen2.5-Coder release tier, an RL-trained reasoning variant, or an official Qwen checkpoint suffix. The evidence instead supports a narrower definition: Qwen2.5-Coder-C$^3$3 is a supervised fine-tuned, completion-specialized model whose primary novelty is explicit controllability in fill-in-the-middle settings.

7. Limitations and interpretation

The first limitation is scope. C$^3$4-Bench is Python-only and focuses on in-file completion rather than repository-scale controllable completion. As a result, the paper does not directly establish controllable completion performance for Java, TypeScript, C#, or other languages in which Qwen2.5-Coder otherwise reports multilingual strength (Zhang et al., 22 Jan 2026).

The second limitation is capability dependence. The paper explicitly notes that ICC performance remains constrained by the base model’s coding ability. Instruction tuning improves the model’s ability to choose how to implement a solution, but it does not fully solve whether the underlying model can solve the problem at all. This is consistent with the empirical pattern: the largest gains are on SCC rather than ICC (Zhang et al., 22 Jan 2026).

The third limitation is evaluation methodology. ICC instruction adherence is judged by Claude3.5-Sonnet rather than by exhaustive human annotation, although the reported agreement with senior developers is 98% across 10 rounds. This supports the benchmark’s reliability, but it is still not identical to fully human evaluation (Zhang et al., 22 Jan 2026).

A final interpretive point concerns naming. In the Qwen literature, the paper introducing Qwen2.5-Coder-C$^3$5 is the only source here that explicitly uses the C$^3$6 designation. Other Qwen-related papers in the same period discuss Qwen2.5-Coder, Qwen2.5-Coder-Instruct, rStar-Coder, ReasonFlux-Coder, or Qwen2.5 Coder 14B in HPC RL settings, but none identify those models as Qwen2.5-Coder-C3. The term therefore refers specifically to the controllable code completion model and its associated benchmark regime, not to a generic Qwen2.5-Coder subfamily (Zhang et al., 22 Jan 2026).