Contribution of experimental artifacts to observed slowdown
Establish whether and to what extent experimental artifacts or biases in the study design account for the measured slowdown when AI is allowed, and quantify their contribution relative to genuine effects of AI assistance on developer productivity.
References
Although the influence of experimental artifacts cannot be entirely ruled out, the robustness of the slowdown effect across our analyses suggests it is unlikely to primarily be a function of our experimental design.
— Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
(Becker et al., 12 Jul 2025) in Section 1, Introduction