Codetations: Intelligent, Persistent Notes and UIs for Programs and Other Documents

Abstract: Software developers maintain extensive mental models of code they produce and its context, often relying on memory to retrieve or reconstruct design decisions, edge cases, and debugging experiences. These missing links and data obstruct both developers and, more recently, LLMs working with unfamiliar code. We present Codetations, a system that helps developers contextualize documents with rich notes and tools. Unlike previous approaches, notes in Codetations stay outside the document to prevent code clutter, attaching to spans in the document using a hybrid edit-tracking/LLM-based method. Their content is dynamic, interactive, and synchronized with code changes. A worked example shows that relevant notes with interactively-collected data improve LLM performance during code repair. In our user evaluation, developers praised these properties and saw significant potential in annotation types that we generated with an LLM in just a few minutes.