Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity

Abstract: In 2025, coding agents have seen a very rapid adoption. Coding agents leverage LLMs in ways that are markedly different from LLM-based code completion, making their study critical. Moreover, unlike LLM-based completion, coding agents leave visible traces in software repositories, enabling the use of MSR techniques to study their impact on SE practices. This paper documents the promises, perils, and heuristics that we have gathered from studying coding agent activity on GitHub.