IFIM: Instruction-Aware Fill-in-the-Middle

• The paper presents IFIM, which integrates natural-language instructions as a core conditioning variable to enhance precision in code infilling.
• By incorporating an explicit instruction channel, IFIM significantly boosts Pass@1 metrics on benchmarks while preserving bidirectional infilling capabilities.
• IFIM achieves these gains with minimal architectural changes, offering backward compatibility and extending applicability to multimodal inpainting tasks.

Instruction-aware Fill-in-the-Middle (IFIM) is a paradigm for enhancing fill-in-the-middle (FIM) code completion and generative models by integrating explicit developer intent, as signaled by natural-language instructions or surrogate modalities. IFIM extends conventional FIM by treating instructions as a core conditioning variable—allowing models not only to synthesize plausible code conditioned on surrounding context but also to respect concise user intentions. This approach achieves significant gains in instruction following while preserving bidirectional infilling capabilities and low-latency inference, with minimal architectural changes and backward compatibility for deployed systems (Sun et al., 29 Sep 2025).

1. Motivation and Formal Definition

Standard FIM models, widely used for code completion, are trained to predict a missing contiguous span of code $x_{\rm mid}$ given a left prefix $x_{\rm pre}$ and a right suffix $x_{\rm suf}$, optimizing the objective: $$\mathcal{L}_{\rm FIM} = - \sum_{(x_{\rm pre},x_{\rm mid},x_{\rm suf})\in\mathcal{D}} \log p_\theta(x_{\rm mid} \mid x_{\rm pre},\,x_{\rm suf})$$ However, this strategy fails when developer intent is underspecified by code context alone. Developers often supplement context with natural-language instructions (“filter out negative numbers”), yet base FIM models do not utilize these cues effectively. Instruction-tuned models (e.g., trained on (instruction → complete-function) pairs) often disrupt FIM’s context usage, producing non-infill-compliant outputs.

IFIM introduces an explicit instruction segment $x_{\rm inst}$ into the input, resulting in a quadruplet $$(x_{\rm pre},\,x_{\rm inst},\,x_{\rm mid},\,x_{\rm suf})$$ and the revised training objective: $$\mathcal{L}_{\rm IFIM} = -\sum_{(x_{\rm pre},\,x_{\rm inst},\,x_{\rm mid},\,x_{\rm suf})\in\mathcal{D}} \log p_\theta(x_{\rm mid}\mid x_{\rm pre},\,x_{\rm inst},\,x_{\rm suf})$$ This explicit instruction channel enables models to leverage developer-provided intent when available, while preserving conventional FIM behavior when absent (Sun et al., 29 Sep 2025).

2. Dataset Construction and Training Protocols

The IFIM methodology constructs large-scale training datasets by extracting infill spans from open-source code and generating concise, intent-focused instructions using GPT-4o. Each example is reviewed to ensure instruction relevance and uniqueness, yielding 122.9K multi-language instruction-infill quadruplets (with a 70% Python share). Overlap with standard evaluation sets (e.g., HumanEval, MBPP) is explicitly filtered.

Fine-tuning protocols use Adafactor with a learning rate of $5 \times 10^{-5}$, two epochs over the IFIM dataset, and do not alter model depth or attention. A single added special token (e.g., <INS>) delimits the instruction. The best-performing “IFIM mode” places the instruction immediately before the masked span (“I-before-M”) (Sun et al., 29 Sep 2025).

3. Evaluation Benchmarks and Quantitative Results

IFIM’s efficacy is evaluated via IHumanEval (derived from HumanEval-infilling, 1,640 Python FIM tasks) and IRepoMasterEval (IRME, 256 tasks from real-world repositories with full test suites). The primary metric is Pass@1, the fraction of first completions passing all tests.

Key empirical results for Deepseek-Coder and Qwen2.5-Coder are summarized in the following table (Pass@1, %, with delta shown):

Model	IHumanEval (Instr.)	IRME (Instr.)	IHumanEval (No Instr.)	IRME (No Instr.)
Deepseek FIM	84.6	10.9	68.6	7.4
Deepseek IFIM	93.6 (+9.0)	21.1 (+10.2)	78.2 (+9.6)	16.0 (+8.6)
Qwen2.5 FIM	91.0	18.4	76.0	10.2
Qwen2.5 IFIM	95.8 (+4.8)	20.3 (+1.9)	76.3 (+0.3)	13.3 (+3.1)

These results confirm that IFIM consistently boosts instruction-following while also improving or maintaining infilling performance with no instructions (Sun et al., 29 Sep 2025).

4. Comparative Paradigms and Expansions

Conventional instruction-tuning (e.g., WizardCoder, Magicoder) trains on (instruction→completion) pairs and breaks FIM’s bidirectional conditioning, often leading to suffix repetition and stylistically incongruent completions. In contrast, IFIM preserves core infilling by simply inserting a delimited instruction, with no architectural changes beyond the added token.

Search-and-Replace Infilling (SRI) (Zhang et al., 19 Jan 2026) generalizes this approach by structuring infilling as a two-stage search (span identification) and replace (edit application) within a single autoregressive pass. The SRI framework incorporates explicit diff-style instructions, further broadening context-aware editing—enabling bug correction and security alignment at minimum latency. SRI-formatted tuning avoids degradation of general coding ability, unlike natural-language-infilling formats (Zhang et al., 19 Jan 2026).

Outside text, diffusion-based inpainting models (e.g., “Fill in the ____”) (Gebre et al., 2024) realize an instruction-aware FIM framework for images, with the “instruction” instantiated as an image patch, modulated at each reverse denoising step. This demonstrates the modality-abstraction potential of IFIM, where the core concept—explicit intent-channel conditioning—readily generalizes.

5. Ablations, Limitations, and Robustness

Ablation studies on IFIM training mode show that the I-before-M pattern (instruction before masked) is essential, with alternative orders degrading instruction adherence by up to 17 percentage points. Varying the share of IFIM-formatted data indicates monotonic improvement in instruction-following up to 100% IFIM presence. A plausible implication is that the gains are not primarily from extra data, but from explicit utilization of the instruction channel; inclusion of instructions is necessary for generalization (Sun et al., 29 Sep 2025).

Limitations include: (i) Python-centric datasets, (ii) reliance on synthetic instructions from GPT-4o, and (iii) demonstration only on ∼7B-parameter models. This suggests future extensions should address language diversity, instruction provenance, and scaling.

6. Future Directions

Unexplored IFIM directions proposed include:

• Extension to additional programming languages (Java, C++, etc.) and larger models (>30B parameters),
• Incorporation of “wild” instruction sources (e.g., inline comments, Q&A, usage logs) with robust denoising,
• Adaptive IFIM: dynamically inferring at inference time whether a comment should be treated as an instruction.

IFIM also provides a backward-compatible upgrade path for existing FIM code completion systems, aligning model conditioning channels with real-world developer practices (Sun et al., 29 Sep 2025). In multimodal domains (e.g., image inpainting), the underlying principle—explicit conditional control via an “instruction” channel—appears extensible without retraining, as shown by the convex-combination mechanism in image inpainting (Gebre et al., 2024).

7. Broader Impact and Theoretical Significance

Instruction-aware Fill-in-the-Middle bridges the gap between developer intent expression and code model consumption. By conditioning on explicit intent statements, IFIM advances the fidelity of code completion systems, directly addressing underspecification and reducing the trade-off between infilling competence and instruction adherence. The generalization to multimodal “instructions,” as seen in diffusion-based pipelines for images, highlights the paradigm’s broader theoretical relevance for controlled generative inference across domains. Emerging paradigms such as SRI further unify instruction-compliance and FIM flexibility, with no practical latency penalty and robust resilience to typical vulnerabilities such as code injection (Zhang et al., 19 Jan 2026).

IFIM thus constitutes a foundational step toward context- and intent-aware generative systems with minimal intervention in system architecture or deployment, unifying both completion and editing under a rigorous, extensible probabilistic training objective.