The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study

Abstract: AI coding assistants have become prolific in recent years. Through a longitudinal mixed-methods investigation, we examined how professional software engineers perceive the effects of AI coding assistants in regard to task focus, developer experience, and productivity. Two questionnaires were administered six months apart, yielding 158 eligible participants at the first time point, 101 at the second, and a matched longitudinal cohort of 95. Participants reported spending less time on most development tasks, with 82% reporting less on writing code. We find broader shift in focus from creation to verification activities. We propose a new category of work we term supervisory engineering work, encompassing the direction, evaluation, and correction of AI output. We also identified a productivity-experience paradox: productivity perceptions held stable, with 84% reporting improvement at both time points, yet among matched participants, the proportion reporting worsened developer experience in at least one dimension nearly doubled from 14% to 27%, with flow state and cognitive load eroding while feedback loops improved. These findings suggest that AI coding assistants are impacting both the nature of software engineering work and how engineers experience it.

• The paper demonstrates a significant shift from code creation to verification, emphasizing the emergence of supervisory engineering work.
• The methodology combines mixed methods and longitudinal surveys with 95 developers to analyze tool adoption, productivity, and task reallocation.
• The study reveals a productivity-experience paradox where measurable gains coexist with reduced developer flow and increased cognitive friction.

The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study

Study Overview and Methodological Approach

This paper investigates the longitudinal effects of AI coding assistants on professional software engineering, focusing on task focus shifts, developer experience (DevEx), and productivity. The methodology employs a mixed-methods design, leveraging two comprehensive questionnaires administered six months apart (Q1 and Q2) to a global sample of professional engineers (matched longitudinal cohort: $n=95$). Quantitative measures were captured via Likert-scale items, cross-sectional and paired non-parametric testing, and longitudinal movement analyses. Qualitative thematic analysis contextualized these findings, enabling nuanced interpretation of lived developer experience.

Adoption and Usage Patterns

AI coding assistants are widely adopted, with the study documenting rapid diversification and tool landscape shifts. GitHub Copilot and ChatGPT led initial adoption but were joined by specialized environments like Cursor and numerous emerging tools, reflecting a mean increase in tools per user from 1.9 to 2.9 over the study window.

Figure 1: AI coding assistant adoption rates at Q1 and Q2. Top 10 tools by maximum usage at either time point shown.

Daily usage rates exceeded 60% across both time points. While initial impressions were predominantly positive, anticipated disappointment if tools were removed was more evenly distributed.

Figure 2: Distribution of initial impressions of AI coding assistants at Q1.

Figure 3: Distribution of anticipated disappointment if AI coding assistants were removed at Q2.

Concerns about quality and security persisted, with a notable rise in maintainability concerns over time.

Figure 4: Primary concern distribution at Q1 and Q2. "Other" excluded.

Task Focus Shifts: Creation-to-Verification Transition

Survey results indicate a pronounced reduction in effort expended on traditional creation tasks, most prominently code writing, with 82% reporting less time spent by Q2 and only 2% reporting more. Task focus analyses revealed a statistically significant shift toward verification-centric activities (reviewing, testing, debugging) rather than creation (designing, writing, refactoring), with verification activities overtaking creation as the locus of perceived activity ($r_{rb}=0.39$, $p_{adj}=0.006$).

Figure 5: Task focus shifts across time points.

Qualitative responses underscore routine compression—AI handling boilerplate, scaffolding, and rapid trial/error debugging. Participants describe a workflow transformation wherein information seeking and comprehension tasks are rerouted away from external sources to AI interactions, and supervisory engineering work emerges: engineers increasingly direct, evaluate, and correct AI-generated output rather than solely produce code.

Supervisory Engineering Work

A novel work category is proposed, supervisory engineering work, encompassing:

• Directing AI: Prompt engineering, articulating task intent and context.
• Evaluating Output: Assessing correctness, maintainability, integration viability.
• Correcting Errors: Modifying AI output, integrating with legacy codebases.

This role, not adequately captured in conventional SDLC categorizations, absorbs effort previously allocated to creation and verification tasks, raising questions about evolving skill requirements and professional identity.

The Productivity-Experience Paradox

Productivity perceptions remained robustly stable, with 84% of matched participants reporting improvement at both time points.

Figure 6: Perceived productivity impact across time points.

Figure 7: Productivity perception transitions between Q1 and Q2.

However, developer experience diverged: while quantitative feedback loops improved ($\Delta M=+0.21$, $r_{rb}=0.38$, $p=0.038$), cognitive load and flow state declined non-significantly. The proportion reporting worsened experience in at least one dimension nearly doubled from 14% to 27%, contradicting the traditional productivity-experience link. Flow state disruptions and increased cognitive friction were especially pronounced, attributed to increased context switching and the demands of evaluation/verification cycles inherent in supervisory engineering.

Figure 8: Perceived developer experience impact across time points.

Figure 9: Developer experience perception transitions between Q1 and Q2.

Cross-sectional correlations between DevEx metrics and productivity remained moderately strong (Q1: flow $\rho=0.49$, Q2: feedback $\rho=0.37$), but longitudinal changes were decoupled.

Practical and Theoretical Implications

Professional and Organizational Dynamics

• Reallocation, not Reduction: AI coding assistants restructure engineering work, shifting focus from creation to verification and supervision. Productivity gains are partly absorbed in ongoing evaluation and trust calibration, not simply yielding time savings.
• Skill Evolution: Supervisory engineering skills—prompt articulation, critical evaluation, collaborative verification—become increasingly central, requiring adaptation in engineering education and organizational training.
• Monitoring Developer Experience: Organizations should monitor experiential indices, not just output metrics, as sustained productivity can camouflage eroding developer experience and risk burnout.
• Tool Ecosystem Fluidity: Continued experimentation with diverse coding assistants is indicated, as rapid landscape evolution precludes stability and uniformity.

Educational Implications

Curricular realignment toward supervisory skills, prompt engineering, and human-AI collaboration principles is essential, as students can now generate code with AI without deep comprehension. Assessment should emphasize understanding and evaluative skill rather than output alone.

Sustainability and Future Directions

• SDLC Model Update: Supervisory engineering work may warrant recognition as a distinct SDLC activity.
• DevEx Framework Revision: The productivity-experience paradox suggests the need for new DevEx dimensions or reweighted causal pathways in AI-mediated workflows.
• Satisfaction in Supervision: Sustained enjoyment and fulfillment in supervisory engineering work remains an open empirical question, with experiential costs potentially accumulating.
• Team-level Patterns: The aggregation of creation-to-verification shifts and supervisory work across teams and orgs merits further empirical scrutiny.

Conclusion

This study documents a fundamental reallocation of effort within software engineering, with AI coding assistants accelerating code creation, rerouting information gathering, and prompting the emergence of supervisory engineering work. Productivity perceptions are stable and positive, yet developer experience, particularly flow state, shows signs of erosion—a productivity-experience paradox. The study’s findings inform engineering practice, organizational policy, and curricular innovation, highlighting the need for adaptive skill development, renewed focus on experiential quality, and ongoing empirical validation as AI tools evolve (2605.23135).