---
title: Structured Fill-In-the-Middle (SFIM)
url: https://www.emergentmind.com/topics/structured-fill-in-the-middle-sfim
type: topic
---

# Structured Fill-In-the-Middle (SFIM)

Structured Fill-In-the-Middle (SFIM) is a class of sequence modeling objectives and architectures in which a model is explicitly trained to generate or infill a central missing segment, or multiple internal masked sections, of a sequence with consideration for structural or semantic boundaries. SFIM stands apart from generic fill-in-the-middle (FIM) by its use of underlying structure—syntactic, semantic, or user-defined—to select, constrain, or guide both the span to be infilled and the manner of infilling. SFIM methodologies have been adopted across natural language, code, protein sequence, and tabular application domains, with implications for controllability, diversity, and real-world usability in editing, reasoning, and code synthesis tasks.

## 1. Structural Approaches to SFIM: Syntax, Trees, and Block-Level Masking

The key innovation in SFIM lies in moving beyond masking arbitrary spans to leveraging structural information. In code domains, SFIM implementations employ Abstract Syntax Tree (AST) analysis to select infill regions that correspond to complete, independently meaningful blocks—such as functions, control-flow statements, or logical groupings—rather than character-aligned or token-aligned random spans. Methodologies involve parsing code (e.g., via Tree-sitter), sampling a non-leaf internal node (representing an entire syntactic subunit), and ensuring that both the prefix and suffix present the global context on both sides of the omitted structure [2410.13187, 2506.00204, 2508.19532].

In natural language, SFIM can include specifying a middle word (as in middle-out decoders for captioning [1810.11735]), or splitting documents at semantically coherent boundaries for structured editing tasks.

For protein engineering, SFIM uses biologically meaningful residue windowing, ensuring that both local and global sequence context around the mutable region supports function and structure [2303.16452].

Table: Span Selection Strategies in SFIM

| Domain         | Span Selection Strategy          | Structural Unit         |
|----------------|---------------------------------|------------------------|
| Code           | AST subtree extraction           | Functions, if/while... |
| Protein design | Region respecting secondary structure | Residue window        |
| Text           | User- or classifier-specified token | Word, phrase          |

SFIM's reliance on structural alignment ensures infills are syntactically and semantically plausible, mimicking real-world editing operations and promoting syntactic validity, as opposed to models trained on random span masking which may sever code elements or disrupt language flow [2506.00204].

## 2. Training Objectives, Data Transformations, and Losses

SFIM training objectives universally require reformatting the training data to enforce model exposure to bidirectional context accompanied by structure-aware masking. In code, this involves extracting syntactically complete spans and reordering training examples, commonly using the "Prefix-Suffix-Middle" (PSM) or "Suffix-Prefix-Middle" (SPM) formats with sentinel tokens marking each boundary [2207.14255, 2410.13187]. SFIM loss is typically formulated as cross-entropy over the infilled region, but many models (e.g., aiXcoder-7B, AST-FIM) include dual loss formulations for PSM and SPM to enhance flexibility.

In mathematical reasoning, MathFimer employs a decomposition of solution chains into prefix, missing step, and suffix, with loss applied only to the reconstructed middle step, using supervised fine-tuning on expanded reasoning chains [2502.11684].

SFIM training increasingly incorporates auxiliary objectives for planning (e.g., horizon-length prediction [2410.03103]) or curriculum-based strategies, in which the complexity of masked regions is gradually increased according to program or reasoning structure [2412.16589, 2508.19532].

## 3. Evaluation Benchmarks and Quantitative Performance

SFIM methods are evaluated across domains with real-world and synthetic benchmarks designed to test the quality of infill and the preservation of structure:

- **Code**: Real-FIM-Eval (code deltas from real commits covering multiple languages [2506.00204]), HumanEval and MBPP for function completion [2508.19532], SAFIM for syntax-aware infilling, CCEval with cross-file context [2412.16589].
- **Protein**: SEIFER benchmark requires not only sequence matching but strict conservation of secondary structure in the infilled region [2303.16452].
- **Mathematics**: MathFimer-expanded datasets with inserted intermediate reasoning steps, evaluated on GSM8K, MATH, and MetaMathQA [2502.11684].

Performance gains reported for SFIM-trained models over baseline FIM/random mask models are substantial: up to 5 points improvement on code FIM benchmarks (AST-FIM over Rand-FIM [2506.00204]), up to 24% relative gain in FIM tasks with horizon-length prediction [2410.03103], and 3–9 percentage-point advancement on mathematical reasoning benchmarks through chain expansion [2502.11684]. SFIM also reduces decrements in left-to-right generative capability compared to standard FIM; for example, joint FIM/L2R training yields "FIM-for-free" without degrading conventional perplexity metrics [2207.14255].

## 4. Architectural and Serving Considerations

Implementing SFIM at scale necessitates architectural adaptations for both training and inference stages:

- **Dual Decoding and Attention**: Middle-out decoding instantiates two decoders expanding in opposite directions from a central token, each with dual self-attention mechanisms to consolidate context from both generated sides [1810.11735].
- **Bidirectional Masked Language Modeling**: Models like FiLM generalize SFIM capabilities to any mask position via a masked language modeling objective with adaptive mask scheduling (Beta-distributed mask probabilities) [2310.09930].
- **KV Cache Efficiency**: For efficient serving in interactive environments, EFIM proposes a prompt reordering (moving the infill to the end) and fragment tokenization training, enabling both prefix and suffix KV caches to be reused across requests, demonstrating up to 98% throughput gain and 52% latency reduction [2505.21889].
- **Syntax-Constrained Decoding**: Early-rejection incremental parsers with quotient grammars force syntactic validity during decoding, achieving near 90% syntax-correctness rate in Python FIM tasks [2402.17988].
- **Robust Subtoken/Character-Level Handling**: FIM-SE eliminates the need for subtoken prediction at span boundaries by aligning infill units to line boundaries and applying strong character constraints, reducing perplexity and error rates in character-level SFIM tasks [2405.17103].

## 5. Real-World Applications and Broader Implications

SFIM has enabled new capabilities and performance gains across several real-world tasks:

- **Code Completion and Editing**: SFIM-trained models produce infills that fit within realistic code editing operations, supporting IDE-assisted patching, automated refactoring, and multi-line suggestion [2506.00204, 2410.13187, 2508.19532].
- **Form Filling and Structured Data Entry**: Domain-agnostic form filling combines multi-faceted contextual inputs and structured output maps, supporting real-time autofill across arbitrary web forms [2310.17826].
- **Mathematical Reasoning Expansion**: MathFimer demonstrates that FIM-based reasoning insertion can systematically improve both solution chain detail and final accuracy in LLM reasoning [2502.11684].
- **Protein Engineering**: By infilling the middle of proteins with context-aware mutations, SFIM-guided pLMs such as ProtFIM generate functional, structurally consistent designs for protein engineering [2303.16452].

Table: Domains and Notable SFIM Application Contexts

| Domain       | SFIM Output Type               | Benchmark/Deployment           |
|--------------|-------------------------------|--------------------------------|
| Code         | Block/statement infill        | HumanEval, SAFIM, Real-FIM-Eval|
| Language     | Phrase/paragraph insertion    | WikiText-103, ROCStories       |
| Protein      | Sequence segment infill       | SEIFER                         |
| Mathematical | Reasoning step insertion      | GSM8K, MATH, MetaMathQA        |
| Form filling | Field-by-field autofill       | OmniFill user studies          |

A plausible implication is that SFIM-mediated models facilitate controllable, semantically-aware editing and completion for a range of tasks where preservation of global structure is critical, and provide a principled route to fine-grained curriculum design, planning objectives (e.g., horizon-length), and cache-efficient deployment.

## 6. Limitations and Open Challenges

Despite its strengths, SFIM faces several ongoing challenges and limitations:

- **Subtoken/Fragment Generation**: Prompt formats exposing incomplete tokens at the boundary can degrade model performance unless special fragment tokenization procedures are employed [2505.21889, 2405.17103].
- **Boundary Awareness**: Without auxiliary objectives (such as HLP), models may fail to terminate generation at the correct integration point, requiring dataset-specific post-processing for evaluation and sometimes for deployment [2505.18789, 2410.03103].
- **Domain Generalization**: While SFIM generalizes well within code and block-structured domains, cross-language or cross-schema generalization relies on robust structural parsing and annotation pipelines, and may require adaptation for new programming languages or data schemas [2506.00204].
- **Inference Efficiency Under High Concurrency**: Although prompt reordering can boost KV cache reuse, resource constraints (e.g., limited GPU memory) still limit scaling under extreme concurrency [2505.21889].
- **Evaluation Complexity**: Meaningful evaluation of SFIM completions must reconcile diverse structural units, requiring benchmarks that reflect real-world editing patterns and standardized post-processing heuristics when "raw" outputs remain imperfect [2505.18789].

## 7. Future Directions and Research Trajectory

Recent SFIM research suggests several directions:

- **Generalizing SFIM to Multimodal and Non-Code Domains**: Strategies employing structure-aware masking (e.g., visual or tabular block masking) may extend SFIM to multimodal settings.
- **Integration with Preference and Reward Optimization**: SFIM paired with preference optimization (such as DPO) over granular structural units enables more data-efficient learning from limited but high-quality test cases [2508.19532].
- **Planning and Lookahead**: Explicit modeling of horizon or boundary (as in HLP) may be vital for applications requiring structured insertion or alignment beyond straightforward L2R objectives [2410.03103].
- **Human-in-the-Loop and Transparency**: SFIM deployment in user-facing systems (e.g., OmniFill) reveals the importance of context transparency and user-driven task specification, highlighting the need for attribution and prompt construction interfaces [2310.17826].
- **Standardization of Benchmarks and Post-Processing**: The proliferation of SFIM tasks and specialized data formats (PSM, SPM, line/block-aligned splitting, AST masking) underscores the need for shared evaluation standards and tooling [2207.14255, 2506.00204].

SFIM-based architectures and objectives are shaping a trajectory towards structure-aligned, contextually adaptable, and computationally efficient language models tailored for a wide range of infilling, editing, and reasoning tasks.

Source: https://www.emergentmind.com/topics/structured-fill-in-the-middle-sfim