Find Open Problems
Find Open Problems
Search for open problems in our database
Submit a Problem
Submit a new open problem to our database (not available yet)
Dice Question Streamline Icon: https://streamlinehq.com

Net effect of task selection on AI’s productivity impact

Determine the net effect of selecting shorter, better-scoped issues on measured AI speedup or slowdown, accounting for both potentially improved AI performance on clearer tasks and potentially improved human performance on the same tasks.

References

Existing literature would suggest that this means AIs would perform better on these issues, which we might expect to speed up developers (in contrast to our observed slowdown). However, better scoped issues may also be issues where these expert developers perform better as well, making AIs less relatively useful, so the net effect of this selection is unclear.

— Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (Becker et al., 12 Jul 2025) in Factor Analysis: Factors with an unclear effect on slowdown — Subsubsection “Unrepresentative task distribution”