To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing

Abstract: LLMs are increasingly used for code editing, yet the prevalent full-code generation paradigm suffers from severe efficiency bottlenecks, posing challenges for interactive coding assistants that demand low latency and cost. Despite the predominant focus on scaling model capabilities, the edit format itself has been largely overlooked in model training. In this paper, we begin with a systematic study of conventional diff formats and reveal that fragile offsets and fragmented hunks make generation highly unnatural for LLMs. To address it, we introduce BlockDiff and FuncDiff, two structure-aware diff formats that represent changes as block-level rewrites of syntactically coherent units such as control structures and functions. Furthermore, we propose AdaEdit, a general adaptive edit strategy that trains LLMs to dynamically choose the most token-efficient format between a given diff format and full code. Extensive experiments demonstrate that AdaEdit paired with structure-aware diff formats consistently matches the accuracy of full-code generation, while reducing both latency and cost by over 30% on long-code editing tasks.

• The paper introduces structure-aware diff formats, BlockDiff and FuncDiff, that significantly improve code edit accuracy by aligning changes with AST structures.
• The paper demonstrates that AdaEdit, an adaptive format selection mechanism, achieves over 90% accuracy in token-efficient format selection while reducing latency and cost by over 30%.
• The paper validates these methods across Python and JavaScript benchmarks, offering a robust framework for scalable and efficient LLM-based code editing.

Structure-Aware and Adaptive Output Formats for Efficient LLM-Based Code Editing

Motivation and Background

LLMs have become central to automated code editing in modern software engineering. The standard paradigm is full-code generation—models output the entire code for even minor edits—which introduces significant latency and token inefficiency, especially in interactive settings such as IDE-driven pair programming. The focus in prior work has been on model scaling and instruction-following power, with limited attention to the role of edit format in optimizing efficiency and edit accuracy. Existing diff formats—number-indexed (e.g. unified diff) and content-addressed (e.g. context diff and search/replace)—are leveraged only superficially via prompting and lack systematic evaluation for LLM training.

Systematic Analysis of Edit Formats

The paper conducts a rigorous comparison between conventional diff formats:

• Number-indexed diffs (e.g., MinUniDiff, UniDiff): Rely on precise line numbers and offsets. LLMs consistently perform poorly due to fragile numerical indexing; even with line-augmented source inputs, degraded edit accuracy is observed.

Figure 1: Examples of conventional diff formats and their reliance on line-numbered context and fragile offset assignment.

• Content-addressed diffs (e.g., MinContentDiff, ContentDiff): Utilize unique anchor contents to locate modified regions. These formats alleviate numerical fragility, but LLMs struggle with fragmented hunks, resulting in unnatural output generation and compromised accuracy.

Structure-Aware Diff Formats

To resolve syntactic fragmentation, the authors introduce BlockDiff and FuncDiff:

• BlockDiff: Edits at any fine-grained AST node level (including control structures and non-AST segments).
• FuncDiff: Restricts edits to function and class boundaries, favoring broader structural stability.

These formats are generated by mapping diff hunks to the AST-derived block tree (using tree-sitter), then expanding anchor content until unique, merging overlapping hunks, and consolidating edits within logical modules.

Figure 2: Overview of structure-aware diff formats and AdaEdit, showing block-level and function-level anchoring.

The patching process is purely textual search-and-replace, with robustness mechanisms for whitespace and blank line ambiguity.

AdaEdit: Adaptive Format Selection

Recognizing that diff formats lose token efficiency for pervasive edits, AdaEdit enables the model to choose, per sample, between full code and diff format, training on the most token-efficient format for each instance. The adaptivity is internalized via SFT, allowing LLMs to optimize both latency and cost.

Empirical Results

Experiments span Python and JavaScript datasets, multiple LLM architectures (Qwen2.5-Coder-7B, Qwen2.5-Coder-14B, DeepSeek-Coder-6.7B), and diverse benchmarks (EditEval, CanItEdit, HumanEvalFix, Aider).

• Edit accuracy: Structure-aware formats (BlockDiff, FuncDiff) consistently outperformed ContentDiff and number-indexed formats. FuncDiff and BlockDiff with AdaEdit achieved accuracy comparable to or exceeding full-code generation, especially with larger model variants.

  Figure 3: Edit usability comparison, demonstrating improved patch success and code linting from structure-aware formats.

• Latency and cost: AdaEdit and structure-aware formats reduced both latency and token cost by over 30% for long-code editing tasks. For short-function edits, diff formats added anchor-induced overhead, but AdaEdit mitigated this via dynamic switching.

  Figure 4: Latency-accuracy landscape of edit formats and AdaEdit; AdaEdit shifts towards optimal latency and accuracy.

  

  Figure 5: Edit cost comparison across code scales, evidencing significant token savings with BlockDiff/FuncDiff and AdaEdit.

• Format selection mechanism: AdaEdit achieved >90% correctness in token-efficient format selection; larger models (e.g., GPT-5) exhibited inferior selection capability without explicit adaptation logic (see appendix).

  Figure 6: Accuracy of AdaEdit's format selection, showing high reliability in adaptive editing.

Cross-Language and Robustness

Experiments validated the cross-language robustness of structure-aware formats, with BlockDiff and FuncDiff outperforming line-level formats in JavaScript as well. Simple adjustment of AST node configurations enables generalization.

Implications and Future Directions

Practical: The findings demonstrate that optimizing edit format—rather than solely scaling model—delivers substantial efficiency gains for real-world code assistants. AdaEdit provides a robust framework for adaptively minimizing latency and token costs without sacrificing edit accuracy or usability. Structure-aware diff formats, with block-level rewriting, enhance synthetic coherence, facilitating more natural generation aligned with LLM pretraining.

Theoretical: The results highlight the inherent limitations of LLMs in arbitrary fragment generation and the necessity for syntactic alignment in supervised edit learning. They also emphasize the importance of data-driven adaptive mechanisms that internalize cost-benefit logic, rather than relying on few-shot prompting or meta-instruction alone.

Future Work: Incorporating reinforcement learning with reward balancing for correctness and efficiency could further optimize edit strategies. Fluid granularity—beyond fixed AST boundaries—presents challenges in diff and patching but could yield more flexible models capable of repository-level editing.

Conclusion

This work establishes the critical role of edit format in LLM-based code editing, introducing structure-aware diff formats and an adaptive edit strategy (AdaEdit) that systematically improve efficiency and accuracy. The empirical evaluation validates these contributions across models and languages. The implications extend to the design of cost-effective, scalable, and robust coding assistants, motivating future exploration of granularity adaptation and reward-driven optimization.