Qwen2.5-Coder-3B-Instruct: Compact Code Model

Updated 13 July 2026

Qwen2.5-Coder-3B-Instruct is a 3-billion parameter instruction-tuned code model that efficiently generates, completes, debugs, and reasons about code.
Built on the Qwen2.5 backbone with specialized tokenizer augmentation and a multi-stage instruction tuning process, it offers strong performance relative to its size.
The model is noted for its energy-efficient code rewriting and prompt responsiveness, making it ideal for experimental local deployment in research settings.

Qwen2.5-Coder-3B-Instruct is a 3-billion-parameter instruction-tuned code model in Alibaba’s Qwen2.5-Coder series, built on the broader Qwen2.5 architecture and post-trained to function as a coding assistant for code generation, completion, debugging, reasoning, and interactive use. Within the six-model family—0.5B, 1.5B, 3B, 7B, 14B, and 32B—it occupies a mid-small position: small enough for practical local deployment, yet repeatedly reported as unusually strong relative to same-size peers and competitive with some larger open-weight baselines. Across the cited literature, it is characterized by three recurring properties: strong code-generation and code-reasoning performance for its scale, notable responsiveness to prompt structure in energy-efficiency-oriented code rewriting, and suitability as a compact open-weight model for local experimentation, although not without clear limitations in repository-scale completion, verified RL evidence, and vulnerability detection (Hui et al., 2024).

1. Family position and model definition

Qwen2.5-Coder-3B-Instruct is the 3B instruction-tuned member of the Qwen2.5-Coder family. The technical report describes the series as code-specialized models built from the Qwen2.5 backbone and further trained on a code-centric corpus, with Instruct variants then post-trained to behave as coding assistants. In that family, the 3B checkpoint sits above the 0.5B and 1.5B models and below the 7B, 14B, and 32B models. The report repeatedly emphasizes that the 3B model is especially strong relative to same-size peers, notably StarCoder2-3B, and in many tasks rivals or exceeds models several times larger (Hui et al., 2024).

The model is not presented as a new architecture separate from Qwen2.5. Rather, it inherits the Qwen2.5 architecture and tokenizer, while being specialized for coding through pretraining, tokenizer augmentation, long-context adaptation, and code-focused post-training. The same report states that the Instruct variants are post-trained to act as coding assistants, which is central to understanding why the 3B checkpoint is treated not merely as a decoder-only LLM, but as an instruction-following code model suitable for generation, completion, debugging, reasoning, and interactive workflows (Hui et al., 2024).

A closely related deployment-oriented detail appears in a later reproducibility study of Vul-RAG, where the model identifier is given as [Qwen](https://www.emergentmind.com/topics/qwen)/Qwen2.5-Coder-3B-Instruct, with a parameter count of 3.09B, a maximum context length of 32,768 tokens, and a knowledge cutoff date of February 2024 (Kaniewski et al., 3 Jun 2026). That description is operational rather than architectural, but it situates the checkpoint in open-weight practice.

2. Architecture, tokenizer, and training pipeline

The Qwen2.5-Coder technical report specifies the 3B model’s core configuration as follows (Hui et al., 2024):

Component	Value
Hidden size	2,048
Layers	36
Query heads / KV heads	16 / 2
Head size	128
Intermediate size	4,864
Vocabulary size	151,646
Tokens trained on	5.5T total

Long-context and repository-aware behavior are also part of the model family design. The report states that repository-level pretraining extended context length from 8,192 to 32,768, the RoPE base frequency was changed from 10,000 to 1,000,000, and YARN enabled extrapolation to 128K tokens. The intended implication is explicit in the source: the family is adapted for file-level completion, repo-level retrieval-like completion, and long-context code understanding, rather than being only a generic LLM with additional code data (Hui et al., 2024).

The pretraining corpus, called Qwen2.5-Coder-Data, contains five major components: source code data; text-code grounding data; synthetic data; math data; and text data. The report states that the overall corpus spans over 5.5 trillion tokens, with a final training mixture using 5.2T tokens after data selection and mixing. Source code coverage spans 92 programming languages and includes GitHub repositories, pull requests, commits, Jupyter notebooks, and Kaggle data. The text-code grounding pipeline uses coarse-to-fine hierarchical filtering with smaller models such as fastText; synthetic data is generated using CodeQwen1.5 and checked by an executor so that only executable code is retained; math data comes from the Qwen2.5-Math corpus; and general text data is retained to preserve non-code capability (Hui et al., 2024).

The reported mixture study is notable because it rejects the simplistic assumption that maximal code fraction is always optimal. For Qwen2.5-Coder-7B, the best tested Code:Text:Math ratio was 70:20:10, which improved both coding and non-coding tasks, including MATH, GSM8K, MMLU, CEval, and HellaSwag. This matters for the 3B Instruct model because the report explicitly frames the family as preserving general and mathematical skills rather than becoming narrowly code-only (Hui et al., 2024).

Instruction tuning for the Instruct variant is multi-stage. The report describes multilingual programming-language identification using a fine-tuned CodeBERT classifier, instruction synthesis from GitHub code with LLM generation and LLM scoring, multilingual collaborative instruction generation with language-specific agents and memory banks, checklist-based scoring and verification, coarse-to-fine supervised fine-tuning, mixed tuning with FIM, and offline DPO alignment. The checklist-based scoring rule is given as

$s = w_1 s_1 + \dots + w_n s_n$

where the per-dimension scores cover question-answer consistency, relevance to computer science, difficulty, code existence, code correctness, code clarity, comments, and educational value. Verification uses a multilingual sandbox with static parsing or AST checks and executable unit tests for supported languages such as Python, Java, C++, and JavaScript (Hui et al., 2024).

3. Benchmark profile and code capabilities

The most direct benchmark portrait of Qwen2.5-Coder-3B-Instruct comes from the Qwen2.5-Coder technical report and is echoed in later work using the same model family. On common code-generation benchmarks, the 3B Instruct model is reported as follows (Hui et al., 2024, Yang et al., 2024):

Benchmark	Score
HumanEval / HumanEval+	84.1 / 80.5
MBPP / MBPP+	73.6 / 62.4
BigCodeBench Full / Hard	35.8 / 14.2
LiveCodeBench Pass@1	10.8
MultiPL-E average over 8 languages	72.1
CRUXEval Input-CoT / Output-CoT	53.2 / 56.0
Aider Pass@1 / Pass@2	33.8 / 39.1

On MultiPL-E, the per-language scores are Python 83.5, Java 74.7, C++ 68.3, C# 78.5, TypeScript 79.9, JavaScript 75.2, PHP 73.3, and Bash 43.0. The report uses these results to argue that the model is unusually strong for its size across mainstream programming languages, while the lower Bash score indicates weaker performance on that language (Hui et al., 2024, Yang et al., 2024).

The code-reasoning profile is also strong for a 3B model. On CRUXEval, the model scores 53.2 on Input-CoT and 56.0 on Output-CoT, outperforming DS-Coder-1.3B-Instruct, Yi-Coder-1.5B-Chat, and StarCoder2-3B. The report specifically highlights that it beats larger baselines on code reasoning despite being only 3B parameters (Hui et al., 2024).

In code repair and editing, the strongest explicit 3B evidence is on Aider, where it attains Pass@1 33.8 and Pass@2 39.1. The report notes that this exceeds DS-Coder-1.3B-Instruct, Yi-Coder-1.5B-Chat, and CodeQwen1.5-7B-Chat, although it remains below DS-Coder-6.7B-Instruct on those metrics (Hui et al., 2024).

The model also retains measurable non-code competence. For the 3B Instruct checkpoint, the report lists MATH 58.1, GSM8K 80.7, GaoKao2023en 48.8, OlympiadBench 23.6, CollegeMath 39.7, AIME24 6.7, AMC23 25.0, MMLU 56.5, MMLU-Pro 35.2, IFEval 44.2, CEval 53.9, and GPQA 28.3. The technical report explicitly uses these results to support the claim that the model preserves useful general and math ability alongside code specialization (Hui et al., 2024).

A common misconception is that a compact coder model of this size is only viable for narrow autocomplete. The reported evidence does not support that view. The model is evaluated not only on code generation, but also on code reasoning, repair, multilingual generation, math, and general understanding. At the same time, the literature does not claim uniform dominance: smaller models still trail larger Qwen2.5-Coder variants, and function completion exact match remains low across models because multi-line generation is intrinsically hard to match exactly (Hui et al., 2024).

4. Prompt sensitivity and energy-efficient code generation

A distinct line of evidence comes from the study of energy-efficient code generation, which compares four open-source instruction-tuned small LLMs—StableCode-Instruct-3B, Qwen2.5-Coder-3B-Instruct, CodeLlama-7B-Instruct, and Phi-3-Mini-4K-Instruct—on 150 Python LeetCode problems, evenly split into 50 easy, 50 medium, and 50 hard tasks. Each problem includes a community-verified human solution, and that human-written solution serves both as input prompt and as the baseline for comparison. The evaluated prompting strategies are role prompting, zero-shot, few-shot, and chain-of-thought (CoT). All models are run locally from Hugging Face; for CoT, the strategy is first generated by the GPT-5 API from the human baseline solution and then fed to the SLM together with the original code (Ashraf et al., 12 Sep 2025).

The paper measures runtime in milliseconds with Python’s time module, memory usage in kibibytes with tracemalloc peak memory via tracemalloc.get_traced_memory(), and energy consumption in milliwatt-hours with CodeCarbon. The authors explicitly stress that these metrics evaluate the efficiency and environmental impact of the generated code during execution, not the inference cost of the model itself. The execution environment is a Google Cloud Compute Engine c2-standard-8 VM with an NVIDIA A100 40GB GPU available for model loading, Ubuntu 24.04 LTS, and Python 3.12.3. Each code sample is executed 10 times with a 5-second cooling interval, and only the first valid solution is retained when models emit multiple outputs or echo the prompt (Ashraf et al., 12 Sep 2025).

For the 150-problem average, Qwen2.5-Coder-3B-Instruct is the only SLM reported as consistently energy-efficient across all prompting styles. The paper gives the following aggregate values (Ashraf et al., 12 Sep 2025):

Setting	Runtime / Memory / Energy
CoT	0.00584 / 638.76 KiB / 1.7112 mWh
Few-shot	0.00617 / 651.99 KiB / 1.7121 mWh
Zero-shot	0.00603 / 645.33 KiB / 1.7115 mWh
Role	0.00603 / 645.33 KiB / 1.7114 mWh
Baseline	0.00606 / 648.57 KiB / 1.7122 mWh

These figures mean that the model remains slightly below the human-written baseline on energy consumption under all four prompt styles, with CoT yielding the best overall result at 1.7112 mWh. The paper describes this as the strongest model-specific story in the study. It also notes an important nuance: CoT is the best overall aggregate setting, but not necessarily the lowest-energy configuration in every difficulty slice. On hard problems, zero-shot is slightly lower than CoT, while CoT remains the strongest aggregate choice across all 150 tasks (Ashraf et al., 12 Sep 2025).

At the difficulty level, the energy results are also reported explicitly. For easy problems, the values are 1.7103 mWh under CoT, 1.7120 under few-shot, 1.7112 under zero-shot, 1.7112 under role prompting, and 1.7114 for the baseline. For medium problems, they are 1.7111, 1.7131, 1.7129, 1.7114, and 1.7138 respectively. For hard problems, they are 1.7125, 1.7114, 1.7107, 1.7119, and 1.7115. The authors interpret these patterns by stating that Qwen2.5-Coder-3B-Instruct “benefits from prompting” broadly, while CoT is the most consistently strong strategy in aggregate (Ashraf et al., 12 Sep 2025).

The study draws two broader conclusions relevant to the model. First, prompt effectiveness is model-dependent rather than universal: CodeLlama-7B and Phi-3-Mini-4K do not beat the baseline under any prompting strategy, whereas Qwen2.5-Coder-3B-Instruct does so under all four. Second, few-shot prompting is not the best strategy for any model in the study, suggesting that adding examples does not automatically improve energy efficiency and may add unnecessary complexity. The authors qualitatively attribute Qwen2.5-Coder-3B-Instruct’s behavior to unusual responsiveness to structured reasoning cues, particularly when a reasoning plan is supplied and then translated into optimized code (Ashraf et al., 12 Sep 2025).

Several subsequent papers position Qwen2.5-Coder-3B-Instruct within broader methodological trends rather than reporting a wholly new 3B checkpoint. One theme is additional instruction tuning. OpenCodeInstruct introduces a 5 million-sample open-access instruction-tuning dataset, with each sample containing a programming question, solution, test cases, execution feedback, and LLM-generated quality assessments. For the 3B scale, the paper compares baseline Qwen2.5-Coder-3B-Instruct against OCI-Qwen2.5-Coder-3B and reports 84.1 to 84.8 on HumanEval, 80.5 to 79.7 on HumanEval+, 73.6 to 81.0 on MBPP, 62.4 to 69.3 on MBPP+, 23.7 to 31.1 on LiveCodeBench average, and 35.8 to 38.1 on BigCodeBench Full. The gains are therefore mixed on HumanEval-style tasks but clearly positive on MBPP, MBPP+, LiveCodeBench, and BigCodeBench Full (Ahmad et al., 5 Apr 2025).

That same paper also clarifies the data-engineering assumptions behind such gains. The authors report that filtering by LLM judge score can outperform execution-only filtering, that larger training mixtures help, and that NL-to-Code formatting is stronger than Code-to-Code formatting. A careful reading suggests that Qwen2.5-Coder-3B-Instruct is strong enough to benefit from further supervised tuning, but not so saturated that data quality ceases to matter (Ahmad et al., 5 Apr 2025).

A second theme is repository-level completion. ExecRepoBench introduces an executable repository-level code-completion benchmark with 1.2K samples from 50 active Python repositories and a multi-level grammar-based completion methodology conditioned on the AST. Although the paper’s new completion-tuned model is Qwen2.5-Coder-Instruct-C at 7B, not 3B, it directly evaluates Qwen2.5-Coder-3B-Instruct on MultiPL-E and situates it in the same family trajectory. The paper’s formalization of in-file and repository-level completion is given as conditional infilling over prefix, suffix, and optional repository context:

$P(x_m^{L_k}) = P(x_m^{L_k} \mid x_p^{L_k}, x_s^{L_k}; \mathcal{M})$

and, with repository files $z=\{z_i\}_{i=1}^N$ ,

$P(x_m^{L_k}) = P(x_m^{L_k} \mid x_p^{L_k}, x_s^{L_k}, \{z_i\}_{i=1}^N; \mathcal{M}).$

The paper’s strongest quantitative completion claims belong to the 7B completion-tuned model, not the 3B Instruct checkpoint. A plausible implication is that repository-level completion tuning and executable evaluation are directly relevant to the 3B model family, but the paper does not establish a 3B completion-tuned analogue (Yang et al., 2024).

A third theme is verified RL and reward modeling. ACECODER builds automated test-case synthesis, pass-rate-based preference construction, Bradley-Terry reward modeling, and Reinforcement++ training around Qwen2.5-Coder backbones, especially 7B. It does not report experiments specifically for Qwen2.5-Coder-3B-Instruct. Its core reward-model formulation is

$\mathcal{L}_\phi(\mathbf{x}, s_i, s_j) = \mathbbm{1}[s_i > s_j] \log \sigma\!\left(R_\phi(\mathbf{x}, \mathbf{y}^i) - R_\phi(\mathbf{x}, \mathbf{y}^j)\right),$

with the final loss averaged pairwise over sampled programs (Zeng et al., 3 Feb 2025). The paper explicitly states that any claim about 3B is inferential. Its evidence suggests that the method is model-agnostic and that a smaller Qwen2.5-Coder model would likely benefit if reward quality were high enough, but direct empirical support is presented only for 7B.

A fourth theme is synthetic instruction synthesis with smaller datasets. Infinite-Instruct fine-tunes Qwen2.5-Coder-7B-Base and 32B-Base, then compares them against Qwen2.5-Coder-Instruct checkpoints, arguing that bidirectional synthesis plus static verification can achieve competitive results with at most 200K samples. The paper does not train Qwen2.5-Coder-3B-Instruct directly, but it frames its practical lessons—diverse and logically structured synthetic data, strict static-analysis filtering, and mixed Reverse plus Backfeed data—as likely useful for smaller Qwen coder models as well. This remains an inference rather than a direct result (Xing et al., 29 May 2025).

6. Security, deployment, adaptation, and limitations

Qwen2.5-Coder-3B-Instruct has also been studied outside pure code-generation benchmarks. In a reproducibility study of Vul-RAG, it appears in the parameter-scale study rather than the baseline reproducibility setting. The local open-weight implementation uses PyTorch and Hugging Face Transformers, with default settings max_new_tokens = 4096, temperature = 0.01, max_knowledge = 3, and retry_times = 5. The Vul-RAG pipeline comprises knowledge-base construction, online retrieval using BM25 followed by re-ranking, and iterative detection through prompts for finding vulnerability causes and fixing solutions (Kaniewski et al., 3 Jun 2026).

On the PairVul benchmark—586 test function pairs derived from 2,903 vulnerable or patched Linux-kernel CVE pairs covering the 10 most prevalent CWEs—the model achieves pairwise accuracy 0.24, balanced recall 0.54, balanced precision 0.54, and processes all 586 of 586 pairs. Within the RQ5 scale study, this places it above Qwen2.5-Coder-7B-Instruct at 0.08 pairwise accuracy, about equal to Qwen3-8B and Qwen2.5-Coder-14B-Instruct at 0.24, and below the top reported model in that comparison at 0.29. The paper’s main conclusion is that model scale alone does not drive performance, since a broader plateau around 0.30 pairwise accuracy persists across tested models (Kaniewski et al., 3 Jun 2026).

The same paper also reports CWE-wise pairwise accuracy for the 3B model: 0.26 on CWE-264, 0.20 on CWE-119, 0.14 on CWE-125, 0.36 on CWE-200, 0.22 on CWE-362, 0.35 on CWE-401, 0.20 on CWE-416, 0.21 on CWE-787, 0.27 on CWE-476, and 0.33 on CWE-20. These results indicate that the model can participate meaningfully in local open-weight RAG-based vulnerability detection but does not make the framework production-grade for autonomous vulnerability detection (Kaniewski et al., 3 Jun 2026).

A separate line of evidence concerns low-resource adaptation of the broader 3B Qwen2.5 family. LLiMba adapts Qwen2.5-3B-Instruct—not the coder checkpoint—to Sardinian using continued pretraining and supervised fine-tuning on a single 24 GB RTX 4090. The paper argues that its conclusions about rank, adapter choice, and the CPT-to-SFT division of labor are likely to generalize to other Qwen2.5 3B checkpoints, including Qwen2.5-Coder-3B-Instruct, because the decisive factor is the 3B backbone plus the adaptation recipe rather than a Sardinian-specific architectural change. This is explicitly framed as a likely generalization, not a direct experiment on the coder model. The strongest practical lesson is that continued pretraining does most of the heavy lifting, while parameter-efficient SFT choices such as rsLoRA rank 256 determine the refinement stage under tight-memory conditions (Ballore, 9 May 2026).

Across the literature, several limitations recur. Smaller models still lag the larger Qwen2.5-Coder variants on many tasks. Repository-level completion remains difficult, and function-completion exact match is low across models (Hui et al., 2024). Energy-efficiency gains, though consistent for this model, are small in absolute terms and task-dependent (Ashraf et al., 12 Sep 2025). OpenCodeInstruct reports a slight drop on HumanEval+ after additional tuning, showing that more data does not imply uniform gains (Ahmad et al., 5 Apr 2025). ACECODER does not directly evaluate the 3B checkpoint, so claims about verified RL remain inferential at that scale (Zeng et al., 3 Feb 2025). Vul-RAG shows that local open-weight deployment is feasible, but also that a compact coder model does not break the broader vulnerability-detection performance ceiling (Kaniewski et al., 3 Jun 2026).

Taken together, these studies portray Qwen2.5-Coder-3B-Instruct as a compact but technically serious open-weight code model: architecturally grounded in the Qwen2.5 family, unusually strong for its size on code generation and reasoning, notably responsive to prompt engineering in green-code settings, and sufficiently capable to serve as a local research and development backbone. The same record also shows clear boundaries: scaling within the family still matters, downstream gains often depend more on data quality and evaluation protocol than on architecture alone, and specialized domains such as repository-level completion, verified RL, and vulnerability detection remain active research problems rather than solved capabilities.