To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study

Abstract: LLM-based autonomous coding agents have reshaped software development. While these agents excel at code generation, open questions persist about the long-term maintainability of AI-generated code. This study empirically investigates the maintenance extent, human involvement, and modification types of AI-generated files versus human-authored code. Using the AIDev dataset of AI-generated pull requests and GitHub, we analyzed over 1,000 files and approximately 3,200 changes from 100 popular repositories. Our findings show that: (i) AI-generated files receive less frequent maintenance than human-authored code, with updates affecting only a small fraction of file size; (ii) the most frequent modifications to AI code are feature extensions, whereas human updates focus on bug fixes, and (iii) human developers perform the large majority of this maintenance.

• The paper finds that agent-generated code requires significantly fewer and smaller maintenance commits in the early months compared to human-generated contributions.
• It employs the AIDev dataset and systematic committer heuristics, normalizing by file size to robustly compare maintenance activities.
• The study highlights that human developers perform over 80% of maintenance on AI-generated code, emphasizing current limitations of autonomous agents.

Empirical Analysis of Maintenance in Agent-generated Code

Introduction

The proliferation of LLM-based autonomous coding agents has shifted both the scale and nature of code generation in modern software repositories. While considerable effort has been devoted to evaluating the quality and complexity of agent-produced code, post-generation maintenance demands and the division of maintenance labor between human and artificial contributors remain underexplored. "To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study" (2605.06464) provides an extensive quantitative analysis of maintenance profiles for AI- and human-generated code contributions across a curated subset of active GitHub repositories.

Dataset and Methodology

The study leverages the AIDev dataset, which aggregates AI agent pull requests from 2,807+ high-profile repositories (≥100 stars), focusing on 100 randomly selected repositories to ensure computational tractability and balanced sampling. AI-generated files were identified via committer heuristics informed by official bot account identifiers across four major agents (Copilot, Claude, Devin, Cursor), with Commits classified using the Conventional Commits Classification System (CCS).

Figure 1: Schematic detailing the end-to-end data collection pipeline and central research questions posed by the study.

A six-month post-creation window was uniformly applied, capturing 508 AI-generated and 508 human-generated files. Each file’s subsequent commit timeline was examined, with frequency and magnitude of changes indexed at monthly granularity. Manually constructed baselines and normalization by file size ensured robustness to subclass distribution artifacts and file scale confounds.

Maintenance Frequency and Magnitude

A critical claim substantiated by the analysis is that AI-generated files receive significantly less maintenance—both in frequency and in the size of code modifications—than human-generated files during the initial months post-creation. The median number of maintenance commits for AI files is approximately half that for human files during the first month. This commit deficit remains pronounced for three months, with the gap diminishing after four months. Importantly, maintenance for AI files persists but generally at a reduced, steady-state rate.

The magnitude of maintenance, when normalized by lines of code, is also consistently lower for AI files, indicating patches and revisions are mostly minor adjustments, whereas human files experience larger, more disruptive interventions.

Figure 2: Per-file commit frequency distributions in the first six months post-creation, contrasting agentic and human code.

These findings are material, as they suggest the introduction of agentic code does not immediately impose unsustainable downstream maintenance overheads on human collaborators. However, the authors are careful to note alternative causal explanations—such as diminished comprehensibility of AI code leading to avoidance, or limited deployment in production—cannot be ruled out without deeper qualitative or runtime analysis.

Nature of Maintenance Activities

An unequivocal observation is the predominance of feature extension work in the maintenance of AI-generated files. Using the CCS framework, 21.8% of maintenance commits for agentic files are classified as new feature additions, whereas only 11.7% are direct bug fixes. For human-authored files, bug fixes and documentation changes represent a much larger fraction. This points to an important distinction: agentic code, while initially functional, is often incomplete with respect to requirement coverage, so post hoc augmentation is required. Bug fixing—either due to initial code quality or test composition—appears less urgent for AI code in this sample, but more research is needed to ascertain whether this is because agents produce fewer logic errors or because such files are not as heavily exercised.

In both cohorts, code style, CI/CD, and test improvements are equally infrequent, suggesting those dimensions are likely driven by broader project processes rather than by the generative modality of the source file.

Human vs Agent Participation in Maintenance

Despite advancements in agent capabilities, humans remain overwhelmingly responsible for the maintenance of agent-generated code. The study logs that 83.2% of commits to AI-generated files post-creation are authored by human developers; AI agents are responsible for only 16.8% of such activity. On human-authored files, agentic maintenance is even lower at 7%. This bifurcation highlights both the current limitations of non-interactive agent deployment—where code generation is largely a one-shot activity—and the nascent adoption of agents for iterative, contextually sensitive maintenance operations.

Agents are thus most impactful in initial codebase expansion and scaffolding, while the routine practices of sustaining, correcting, and growing codebases remain vested in human maintainers.

Implications for Practice and Research

The study’s empirical results carry several key implications:

• Maintenance Burden: Initial use of agents does not, in the current ecosystem, offload significant long-term maintenance responsibilities from humans, though agentic code also does not generate a disproportionate maintenance burden within six months.
• Functional Coverage: Agents tend to produce code that is stable but not fully featured, shifting human developer time toward feature extension as opposed to corrective tasks.
• Agentic Limitations: Presently, autonomous agents are not orchestrated for ongoing, requirement-driven maintenance and adaptation. Their limited presence in post-generation commits underscores the gap between code generation and full-lifecycle code stewardship.

From a theoretical perspective, these findings align with prior work indicating higher code smells [DBLP:journals/corr/abs-2510-03029], security vulnerabilities [DBLP:journals/corr/abs-2508-21634], and maintainability challenges [DBLP:journals/corr/abs-2504-12608, KravchukKirilyuk2025] in agentic code, yet nuance these results by showing the downstream impact (in terms of explicit maintenance operations) is mitigated by selective adoption and ongoing human oversight.

Future Directions

Owing to the short operational history of agentic coding, the study’s six-month window limits the extrapolation of results; agentic code may manifest distinct maintenance characteristics over multi-year horizons. Longitudinal studies spanning tool generations and organizational practice shifts are warranted. The sharp deficit in agent involvement in maintenance suggests substantial opportunity for research on interactive, context-aware, and longitudinally active agents capable of handling non-trivial refactoring, requirement tracking, and corrective iteration.

Efforts to develop such agents must integrate robust code understanding, project context assimilation, and the ability to incrementally align with evolving human intent—not just static requirement fulfillment.

Conclusion

The paper delivers one of the first systematic, large-scale empirical comparisons of post-creation maintenance between human and agent-generated code artifacts. AI-generated code requires fewer and smaller maintenance interventions in the short term, with feature augmentation as the primary mode of post-hoc update and humans executing the vast majority of maintenance tasks. These findings provide a nuanced baseline for ongoing evaluation of agentic workflows, underscoring both the productivity benefits and extant limitations of current agentic software engineering practice. Unlocking the next phase of agentic tooling will depend on integrating agents into not just the generative, but also the maintenance and evolution phases of production software systems.