Deletion Avoidance in LLM Code Editing: A New Challenge for AI Maintainability
A recent preprint on arXiv (2607.28887) reveals a significant issue known as 'deletion avoidance' in large language models (LLMs) utilized for code editing. This research assesses five prominent models on the SWE-bench Verified leaderboard, discovering that the deletion recall for developer patches peaks at 71.7%, even for tasks all models can complete. While models identify the correct file for more than 92% of necessary deletions, they accurately remove the specific line in fewer than 52% of instances. Notably, 29.0% of successful patches employ a strategy called 'Guard-and-Go,' where targeted code is enclosed in a guard or fallback. The study indicates that LLM-generated patches could negatively impact code maintainability, despite passing tests. The full study can be accessed at https://arxiv.org/abs/2607.28887.
Key facts
- Deletion avoidance is a systematic tendency in LLMs to retain code that an intended edit requires removing.
- Across five leading models on SWE-bench Verified, deletion recall against developer patch reaches at most 71.7%.
- Models reach the right file for over 92% of required deletions but cut the exact line in under 52% of cases.
- 29.0% of passing patches wrap targeted code in a guard or fallback, a pattern called 'Guard-and-Go'.
- Retrofitting 34 Verified tasks with tests that fail if targeted code remains drops four frontier models from 63.2% to 41.9%.
- The study is published on arXiv with ID 2607.28887.
Entities
Institutions
- arXiv