Mify-Coder: Enterprise Code LLM

• Mify-Coder is a 2.5B-parameter code-specific LLM built on the Mify-2.5B foundation model, using continued pre-training, supervised fine-tuning, and RLHF for effective code generation and function calling.
• It employs specialized tokenization with a 64K BPE tokenizer and a Fill-in-the-Middle objective to enhance structured code completion and syntax awareness.
• Empirical benchmarks reveal that Mify-Coder excels in mainstream tasks like MBPP while maintaining secure, quantized deployments fit for enterprise desktop environments.

Searching arXiv for Mify-Coder and closely related code-model papers to ground the article with current citations. Mify-Coder is a 2.5B-parameter, code-specialized LLM built on the Mify-2.5B foundation model and trained on 4.2T tokens through a compute-optimal strategy. It is presented as a compact coder model that combines continued pre-training, supervised fine-tuning, synthetic data generation, LLM-based quality filtering, and RLHF-based alignment in order to achieve comparable accuracy and safety while remaining deployable in quantized forms on standard desktop environments (Parmar et al., 26 Dec 2025).

1. Origin, scope, and design objective

Mify-Coder is defined as a decoder-style code LLM derived from the Mify-2.5B enterprise foundation model. Its stated goals are to deliver competitive performance on code generation and function calling, make training and deployment feasible for typical enterprises, build in enterprise-grade safety and secure coding behavior from the start, and provide quantized variants that run effectively on standard desktops, including CPU-only settings (Parmar et al., 26 Dec 2025).

The model is explicitly framed against the assumption that frontier coding performance requires large dense models. In that respect, it belongs to a broader research trajectory in which capability is sought through training recipe, data curation, and objective design rather than parameter count alone. Qwen2.5-Coder, for example, is a code-specific family built on Qwen2.5 and continued pretrained on over 5.5 trillion tokens, with a 70:20:10 code:text:math mix and repo-level pretraining (Hui et al., 2024). Qwen3-Coder-Next pushes a different point in the design space by using an 80B-parameter MoE model with only 3B active parameters and large-scale agentic training on executable environments (Cao et al., 28 Feb 2026). This suggests that Mify-Coder should be understood less as an isolated artifact than as one instance of a data-centric, efficiency-oriented turn in code-model research.

2. Architecture and representation

Mify-Coder inherits a standard Transformer-based, autoregressive, decoder-only architecture from Mify-2.5B. Training uses BF16 mixed precision, attention uses Grouped Query Attention, and the tokenizer is a 64K BPE tokenizer extended with specialized tokens for language-aware separators, syntax markers, structured instruction tokens, and two reasoning tokens used to mark reasoning segments (Parmar et al., 26 Dec 2025).

The code specialization is concentrated in tokenization and training objectives rather than in a novel backbone. The model supports Fill-in-the-Middle, with an optimized fraction of CPT data set to 25%. This objective is intended to improve structural code completion, especially infilling within functions and files. The paper also notes Jupyter or notebook awareness through tokenization and data that include cell-based structures, but no new architectural module is introduced for notebooks.

Function and tool use are not implemented through separate architectural heads. Instead, function calling is represented through a unified JSON schema and specialized instruction tokens. This design keeps the model within the standard decoder-only paradigm while shifting specialization into representation and supervision. In this respect, Mify-Coder remains closer to dense code models such as Qwen2.5-Coder than to sparse, agent-specialized systems such as Qwen3-Coder-Next (Hui et al., 2024).

3. Training corpus and data curation

The final training corpus contains 4.2T tokens with an empirically selected composition of 78% code, 12% text, and 10% math (Parmar et al., 26 Dec 2025). Code data is drawn primarily from Stack-v2 via Software Heritage, with copyleft and commercial licenses excluded, non-code sections and headers removed, and configuration or markup filtered where not central. The primary languages are Python, Java, C#, C++, C, and JavaScript, with additional material including SQL, HTML, XML, and Markdown. Project-level structuring uses dependency-aware topological sort across repositories, and single-file repositories are excluded.

Code quality is assessed through more than 50 signals, with the final framework emphasizing textual and structural metrics, comments and documentation signals, and model-based signals. The listed signals include alphanumeric ratio, whitespace ratio, percentage of characters in quoted strings, fraction of encoded data, duplicate n-grams, lines per function, percent “print” or “todo”-like statements, char-token ratio, comment-to-code ratio, docstring density, inline explanations, ArmoRM for readability, explainability, and complexity, and StackEduClassifier for pedagogical clarity across 15 languages.

Math data constitutes roughly 10% of the corpus. Provenance analysis trimmed about 70% of the initial pool of 30 datasets, followed by fuzzy deduplication using MinHashLSH with shingle size 5, 110 permutations, 10 LSH bands, 11 minhashes per band, and Jaccard similarity threshold 0.75, removing about 20% additional tokens. Overlap between math and text corpora was removed through URL-based filtering, eliminating about 3% overlap. Text data accounts for 12% of the corpus and includes high-quality open text corpora, scientific literature, code documentation, GitHub docs, Wikipedia, and domain-specific STEM content. Selection uses the NVIDIA Domain Classifier and EAI-Distill models.

Instruction-oriented data undergoes a multi-stage filtering pipeline: license and provenance validation retains about 50%; quality-based filtering retains 18%; dataset impact analysis leaves about 3%; decontamination then removes benchmark overlaps; metadata-driven annotation organizes samples by natural-language type, token count, and language coverage. LLM-as-judge evaluation uses Qwen2.5-Coder-70B, Llama3-70B-Instruct, and Claude 3.5 Sonnet to rate relevance, correctness, formatting, and coding standards. The paper characterizes the result as higher “data density,” meaning fewer tokens but more useful signal.

4. Continued pre-training, supervised fine-tuning, and task formation

Training is organized as a single continuous trajectory with two main phases: continued pre-training and supervised fine-tuning (Parmar et al., 26 Dec 2025). Continued pre-training begins from Mify-2.5B, first with 50B tokens of code-only CPT to obtain a code baseline, and then with large-scale CPT on the 78:12:10 code:text:math mixture. The CPT optimizer is Distributed Fused Adam with $\beta_1 = 0.9$, $\beta_2 = 0.95$, weight decay $= 0.01$, initial learning rate $= 8 \times 10^{-5}$, and 5,000 warmup steps. FIM ratio was explored at 15% and 30%, and the final choice was 25%. Epoch-count experiments led to a final choice of 3 epochs.

Supervised fine-tuning is progressively aligned with CPT checkpoints, including CPT-S3, CPT-E1.0, CPT-E1.5, and CPT-E1.875. The SFT data covers code generation, completion, translation, bug fixing, refactoring, evaluation-style tasks, multi-turn dialogs, tool and function calling, math and chain-of-thought reasoning, safety, and secure coding. Function calling is normalized through a unified JSON schema of the form {"name": ..., "arguments": {...}}, and synthetic data generation uses seeded, multi-turn dialogues, seedless templates with prompt rotation, and multi-agent role-play.

Ablations reported for SFT show strong gains from scaling curated and synthetic data. Relative to a baseline, baseline+300k improves MBPP from 42.71% to 53.77% and BFCL-v1 from 24.81% to 72.19%; baseline+800k improves MBPP to 67.09%; baseline+1M yields 69.10% on MBPP and 44.71% on BFCL-v2. Function-calling ablations with unified JSON schema show CPT-E1.5+800K at 38.71% on BFCL-v2, CPT-E1.5+960K at 43.83%, and CPT-E1.5+1.1M at 59.35%, while avoiding large regressions on MBPP and HumanEval.

The reported training cost, excluding generation experiments, is 36,864 GPU-hours for CPT and 768 GPU-hours for SFT on B200 GPUs. This is presented as modest relative to 30B-70B-scale training programs.

5. Benchmark profile and empirical performance

Evaluation uses pass@1 on code generation and function-calling benchmarks with max_gen_tokens = 1024. The reported benchmark suite includes MBPP, MBPP+, HumanEval, HumanEval+, and BFCL-v2 (Parmar et al., 26 Dec 2025).

Benchmark	Mify-Coder	Comparative note
MBPP	91.21%	higher than Qwen3-Coder-30B-A3B-Instruct 87.60% and GPT-4o 87.60%
MBPP+	89.15%	higher than Qwen3-Coder-30B-A3B-Instruct 73.50% and Claude Sonnet 4.5 64.81%
HumanEval	53.66%	below Qwen3-Coder-30B-A3B-Instruct 92.10%
HumanEval+	48.78%	below Claude Sonnet 4.5 90.24%
BFCL-v2	55.14%	above DeepSeek-Coder-33B-Instruct 45.40%, below Llama-3.3-70B-Instruct 77.30%

The empirical profile is therefore asymmetric. On MBPP and MBPP+, Mify-Coder is reported as best overall, exceeding much larger open and closed models. On HumanEval and HumanEval+, it remains competitive for its size but does not reach the strongest 30B-123B or closed-model results. On BFCL-v2, its 55.14% score substantially exceeds OLMo-3-7B-Instruct at 12.03% and DeepSeek-Coder-33B-Instruct at 45.40%, while remaining below the 70%-77% range reached by the strongest larger models.

These results support a specific interpretation of its capability: Mify-Coder is strongest on mainstream code generation and strong relative to its size on structured tool use, but not universal state of the art across all coding benchmarks. The paper itself identifies MBPP-style tasks and function-calling workflows as especially favorable settings.

6. Safety, alignment, quantization, and deployment

Safety is treated as part of training rather than as a post-hoc layer. The paper distinguishes harmlessness and helpfulness, and describes safety-focused SFT with mixtures of harmful and general content. Initial experiments with high harmful ratios improved detection but caused excessive refusals, leading to a final ratio of 1:4 harmful:general in safety-focused training. Secure-code supervision mixes vulnerable versus fixed code pairs with general utility code, and CyberSecEval-Autocomplete is used to validate secure code generation, where the model is described as achieving parity with leading models for secure code (Parmar et al., 26 Dec 2025).

Alignment uses an internal RLHF platform called Alignment@Scale. Human feedback is collected from three independent annotators per prompt, with labels, rankings, spans, free-text comments, and code fields used to train preference models and run RLHF. Stanford AIR-Bench is used for safety evaluation, and the paper states that Mify-Coder is competitive with frontier models across regulatory risk categories.

Quantization and deployment are central to the model’s practical identity.

Quantization	Accuracy change vs FP16
FP16	baseline
8-bit SmoothQuant	–0.37%
8-bit AWQ	–1.99%
FP8	–1.35%

FP8 is selected as the primary deployment variant because of the best throughput and latency under TensorRT-LLM. Reported inference metrics are a time-to-first-token of 0.10 seconds and throughput of 6,645.3 tokens/sec on a single NVIDIA H100 80GB GPU. GGUF-based CPU inference is also noted, reducing memory from 4.67 GB to as low as 1.43 GB in some quantized variants. The intended deployment settings include local IDE integration, code assistants and chatbots, and agent frameworks and tool-calling pipelines.

7. Limitations and relation to adjacent systems

Mify-Coder’s limitations are explicit. Performance on HumanEval and HumanEval+ is used to indicate remaining weakness in complex algorithmic reasoning and multi-step decomposition. The paper does not emphasize extremely long-context, large-codebase tasks as a strength. The model is text-only and cannot directly handle screenshots, diagrams, or UI mocks. Coverage is focused on a set of primary languages, and the paper acknowledges that some niche languages or frameworks may be less well supported. It also notes that, despite strong CyberSecEval and AIR-Bench results, subtle insecure patterns and edge-case vulnerabilities remain possible (Parmar et al., 26 Dec 2025).

The proposed future directions are architectural refinements for complex reasoning, RLVR using automated test suites and static analysis, multimodal encoders for UI-to-code and visual debugging, and stronger production-grade benchmarks. Those directions place Mify-Coder in a wider field of specialized coder systems. MultiCoder studies multi-programming-lingual pre-training with PL-level MoE routing for low-resource programming languages (Gong et al., 2022). MemoCoder introduces a multi-agent repair loop with a Fixing Knowledge Set and Mentor Agent for persistent bug-fix reuse (Jia et al., 24 Jul 2025). MalCodeAI specializes a Qwen2.5-Coder-3B-Instruct backbone for vulnerability detection and remediation with CVSS-based risk scoring across 14 languages (Gajjar et al., 15 Jul 2025). M$^2$-Coder extends the space further toward multilingual multimodal code generation from UML diagrams and flowcharts (Chai et al., 11 Jul 2025). Qwen3-Coder-Next emphasizes coding agents, executable environments, and reinforcement learning with an 80B-parameter MoE model that activates only 3B parameters during inference (Cao et al., 28 Feb 2026).

This suggests that Mify-Coder occupies a specific niche within contemporary code-model research: a dense, small, data-centric coder model optimized for strong mainstream coding and function-calling performance, quantized deployment, and enterprise-oriented safety, rather than for multimodal inputs, explicit multi-agent memory, or repository-scale agentic reinforcement learning.