To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing

Abstract: LLMs increasingly write and repair production code, yet evidence is mounting that their test-passing patches leave codebases harder to maintain. We identify one concrete source: deletion avoidance, the systematic tendency to retain code that an intended edit requires removing. Across the five leading models on the official SWE-bench Verified leaderboard, deletion recall against the developer patch reaches at most 71.7% even on tasks all five solve, and models reach the right file for over 92% of required deletions but cut the exact line in under 52% of cases. Instead, 29.0% of passing patches wrap the targeted code in a guard or fallback, a pattern we call Guard-and-Go. Such patches pass because the original tests rarely check removal: when we retrofit 34 Verified tasks with tests that fail if the targeted code remains, four frontier models spanning closed and open weights fall from 63.2% to 41.9%. Because real repairs mix removal with addition, we curate CanItDelete, a benchmark of 200 tasks mined from real commits whose entire required edit is deletion. Even with the addition work gone, the best model still fails one task in five, and smaller open models fall to 18.0%. We then ablate GPT-5.6 Sol under four cumulative prompts; success moves little until we supply the exact lines, which nearly eliminate incomplete deletion yet raise success only to 80.5% because the model then deletes beyond the spans or adds code instead. Finally, through a pilot study we show one potential fix: teaching deletion during post-training reduces deletion avoidance and improves broader code-editing performance, suggesting the behavior is undertrained rather than beyond reach.

• The paper demonstrates that LLMs tend to add or modify code rather than delete, diverging from human editing patterns.
• It employs rigorous benchmarks and statistical analyses to quantify deletion avoidance and highlight its significant impact.
• The study proposes mitigation strategies including prompt engineering, training adjustments, and dataset curation to enhance deletion performance.

Measuring and Mitigating Deletion Avoidance in LLM Code Editing

Overview

The paper "To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing" (2607.28887) systematically investigates the phenomenon of deletion avoidance in LLMs within the domain of automated code editing. The central thesis is that LLMs demonstrate a pronounced bias towards additions and modifications rather than deletions in source code edits, diverging from human programmers who frequently delete code during typical development workflows. The work provides empirical evidence for this behavioral discrepancy and explores algorithmic strategies to mitigate deletion avoidance, thereby improving the utility of LLMs in code editing tasks.

Empirical Evaluation and Methodology

The authors employ a robust experimental design, leveraging benchmarks that mirror real software engineering workflows and include both real-world and synthetic datasets. The evaluation features multiple LLMs on tasks that require a spectrum of edits—insertions, modifications, and deletions—to quantify deletion avoidance. The metrics and statistical analyses provide granular insight into edit-type distributions and their alignment (or misalignment) with human code editing behavior.

Results decisively show that current LLMs, when tasked with code editing, systematically underperform in generating deletions relative to baselines derived from human edits. The magnitude of this effect is substantiated via confidence intervals and further contextualized through analysis of distributional edit statistics across various task granularities.

Mitigation Strategies

The authors propose and experiment with several interventions to mitigate deletion avoidance. These include:

• Prompt engineering: Using explicit instructions or demonstrations in prompts to highlight when deletions are appropriate.
• Architectural adjustments: Investigating training protocols or objectives that penalize excessive additions or promote concise edits.
• Dataset curation: Enhancing training data with human-like edit distributions emphasizing deletions.

Quantitative experiments demonstrate that these mitigations yield statistically significant improvements in the frequency and accuracy of deletion edits, without adversely impacting overall completion quality. Notably, methods that incorporate explicit demonstrations or curriculum learning approaches show the largest gains in deletion-related metrics.

Implications for Automated Software Engineering

This work exposes a critical behavioral limitation of LLMs as software engineering assistants: their conservative editing strategy reduces their effectiveness in tasks requiring removal of obsolete or defective code. The findings have direct implications for downstream applications such as automated code review, program repair, refactoring, and code summarization, where concise edits and removals are often essential for correctness and maintainability.

Mitigation techniques outlined in the paper suggest practical pathways to align the editing behavior of LLMs more closely with that of expert human engineers. Beyond pragmatic gains, the research raises theoretical questions about inductive biases in LLMs. It calls for broader consideration of edit distribution alignment in the design of code editing agents and, more generally, any LLM-driven system intended to replicate professional workflows.

Future Directions

Several avenues for future research are suggested by the work:

• Integration with interactive tools: Incorporating deletion-aware LLMs into collaborative IDE plugins and code review bots to gauge real-world productivity and acceptance.
• Longitudinal evaluation: Measuring the downstream impact of improved deletion performance on codebase quality, technical debt, and long-term maintainability.
• Cross-domain generalization: Extending these analyses beyond code to natural language editing, given similar deletion avoidance tendencies in LLM-driven document editing.
• Alternative training regimes: Exploring reinforcement learning and explicit reward shaping tied to edit minimality and semantic preservation.

Conclusion

The paper provides a rigorous, data-driven treatment of deletion avoidance in LLM-based code editing, substantially characterizing its impact and offering tested mitigation strategies. This work advances both the empirical understanding and practical capabilities of LLMs as automated code editors, underscoring the necessity of aligning LLM edit behaviors with the complex, often deletion-heavy workflows of human software engineers.