---
title: 'SRI-Coder Series: Single-Pass Code Infilling'
url: https://www.emergentmind.com/topics/sri-coder-series
type: topic
---

# SRI-Coder Series: Single-Pass Code Infilling

The SRI-Coder Series designates a family of large language models (LLMs) for source code infilling and editing, based on the Search-and-Replace Infilling (SRI) framework. SRI-Coder models advance beyond the prevalent Fill-in-the-Middle (FIM) paradigm by operationalizing a single-pass, context-aware verification-and-editing process. This is enabled by a structured output of explicit “search” and “replace” blocks, harmonizing instruction-following capabilities of chat-oriented models with the efficiency and practicality required in automated code completion and assisted development tasks [2601.13384].

## 1. Formalization of Search-and-Replace Infilling

Classic Fill-in-the-Middle (FIM) infilling generates a missing code segment $m$ by maximizing conditional likelihood given the prefix and suffix context:
$$
\hat m = \arg\max_{m} P(m \mid \text{prefix}, \text{suffix}) = \arg\max_{m} \prod_{t=1}^{T} P(w_t \mid \text{ctx}, w_{<t}).
$$
SRI recasts infilling as an integrated “search” and “replace” operation executed in a single generative pass. Given context $C = (\text{prefix}, \mathtt{/*\ MIDDLE\ CODE\ TO\ COMPLETE\ */}, \text{suffix})$, the model emits a sequence of the form:
$$
y = [\texttt{<<<<<<< SEARCH},\,y^s,\,\texttt{=======},\,y^r,\,\texttt{>>>>>>> REPLACE}],
$$
where $y^s$ designates the lines to be edited (“search block”), and $y^r$ is the corrected or completed code (“replacement block”). The formal objective is:
$$
(\hat y^s, \hat y^r) = \arg\max_{(y^s, y^r)} \prod_{t=1}^{T_s} P(w_t^s \mid C, w_{<t}^s) \cdot \prod_{t'=1}^{T_r} P(w_{t'}^r \mid C, y^s, w_{<t'}^r),
$$
implicitly reducible to a single $\arg\max_y P(y\mid C)$. This explicit diff-style formalism enables SRI-Coder models to identify and correct context-sensitive errors while maintaining the efficiency of single-pass inference [2601.13384].

## 2. Dataset Construction and Patch-Style Supervision

The SRI-200K dataset underpins SRI-Coder pretraining. It is derived from "The Stack v2" via tree-sitter, extracting logical AST blocks such as function bodies, loops, single lines, and random spans. The curation ensures 200,000 examples with balanced distribution: function bodies (2 units), multi-line blocks (1), random spans (1), and single lines (1). Fine-tuning uses a 20,000-sample high-star-count subset, with the remainder employed for ablation studies.

Each sample consists of a code file featuring a single `/* MIDDLE CODE TO COMPLETE */` marker. The label is a unified patch:
```replace
<<<<<<< SEARCH
// 10 lines of context including the marker
=======
 // same 10 lines but with ground-truth code
>>>>>>> REPLACE
```
This synthetic diff provides supervised guidance for grounding and correcting non-contiguous or multi-line code edits in a unified framework, directly aligning model behavior with practical developer workflows [2601.13384].

## 3. Architectural Design and Training Procedure

SRI-Coder builds on Qwen2.5-Coder and Qwen3-Coder checkpoints, covering a wide parameter range: 0.5B, 1.5B, 3B, 7B, 14B, 32B, 30B, 235B, and 480B. No additional layers or adapters are introduced; the Transformer backbone with SwiGLU activations and RMSNorm remains unchanged. The tokenizer is inherited from Qwen-Coder.

Key training features include:

- Data mixture: 20,000 SRI patches + 60,000 general code-assistant instructions (Glaive-Code-Assistant) + 100 safety alignments.
- Instruction-tuning by minimizing standard cross-entropy loss:
  $$
  \mathcal{L}_{\rm CE} = -\sum_{t=1}^{T_s+T_r}\log P(y_t^* \mid C, y^*_{<t}),
  $$
  with $y^*$ as the ground-truth search-and-replace diff.
- Optimization: Adam (weight decay 0.1, gradient clipping 1.0), BF16 precision, 16×A100 GPUs, initial LR $5 \times 10^{-5}$ with scheduled decay.
- Prompt conventions: input marker `/* MIDDLE CODE TO COMPLETE */`, output delimiters `<<<<<<< SEARCH`, `=======`, and `>>>>>>> REPLACE` [2601.13384].

## 4. Evaluation Methodology and Performance Analysis

SRI-Coder models are benchmarked against base models (standard FIM), natural-language FIM variants (standard, dialogue, template prompts), and a spectrum of proprietary/open LLMs (GPT-4, Claude, Gemini, DeepSeek, Qwen3, Grok):

- Similarity-based benchmarks: Exact Match (EM) and Edit Similarity (ES) on CrossCodeEval, RepoEval, CrossCodeLongEval.
- Execution-based: Pass@1 and unit-test success on ExecRepoBench and SAFIM.
- Latency: average inference time.

Empirical findings:

- SRI-Coder achieves up to +20 EM/ES over NL-FIM chat baselines.
- SRI-Coder-32B surpasses its Qwen2.5-Coder-32B base by +46.3 EM (CrossCodeEval) and +49.3 ES (CrossCodeLongEval).
- Single-pass SRI-Coder matches FIM latency to within a few percent, remaining significantly faster than agentic multi-step tools.
- SRI-Coder retains general coding competencies, with no observed pass@1 drop on MBPP, HumanEval, BigCodeBench, or LiveCodeBench—a marked improvement over NL-FIM fine-tuning which causes a 5–10 point loss [2601.13384].

## 5. Algorithmic Workflow of SRI Inference

The SRI paradigm consists of four principal steps executed in a single-shot generative manner, as shown in the following high-level pseudocode:
```python
function SRI_Infill(file_contents):
    # 1. Locate marker and extract 10-line window W
    p, marker, s = slice_around_marker(file_contents, window=10)
    prompt = format_system_prompt() + p + marker + s 

    # 2. Model generates diff in one shot
    diff_output = Model.generate(prompt) 

    # 3. Parse SEARCH and REPLACE blocks
    search_block, replace_block = parse_diff(diff_output) 

    # 4. Optionally convert to standard git patch or apply directly
    patched_file = apply_search_replace(p, s, search_block, replace_block)
    return patched_file
```
This mechanism enables practical generation of structured diffs, facilitating downstream integration with development tools and version control ecosystems [2601.13384].

## 6. Security Alignment, Limitations, and Prospective Directions

SRI-Coder inherits safety benefits from instruction-following chat models: the SAL benchmark attack success rate drops from approximately 100% (base, unaligned) to below 20% (Table 1), significantly improving over FIM models. Inference latency remains competitive, and SRI-Coder exhibits robust generalization on unseen coding tasks.

Limitations include current evaluation being offline and absence of real-world IDE/user studies, as well as modest gains on models ≤1.5B parameters. This suggests that effective transfer of the SRI paradigm to lightweight architectures may require knowledge distillation or curriculum learning. The SRI-Coder series is slated for open-source release and integration into code editors for iterative community-driven enhancement [2601.13384].

Source: https://www.emergentmind.com/topics/sri-coder-series