Code4MeV2: Open-Source Code Completion
- Code4MeV2 is a research-oriented, open-source code completion platform designed for transparent and reproducible studies on human–AI coding interaction.
- It employs a modular client-server architecture with a JetBrains IDE plugin and dedicated backend, enabling advanced telemetry and customizable data collection.
- The platform achieves industry-comparable code completion latencies and supports community-driven extensibility for scalable empirical research.
Code4MeV2 is a research-oriented, open-source code completion platform designed to provide a transparent, modular, and extensible environment for academic paper and empirical research on human–AI coding interaction. Developed as a JetBrains IDE plugin with a dedicated backend, Code4MeV2 addresses the core challenge of proprietary data and opaque interfaces in commercial code assistants. Its architecture and data collection capabilities enable researchers to conduct reproducible experiments and large-scale analyses without building bespoke tooling. The platform is notable for achieving industry-comparable latencies in code completion, an advanced telemetry framework, and a commitment to community-driven extensibility.
1. Motivation and Objectives
Code4MeV2 was conceived in response to the limitations of mainstream AI-powered code completion tools, whose user interaction data remains closed and inaccessible to the research community (Koohestani et al., 4 Oct 2025). Major commercial platforms restrict deep paper by concealing both behavioral logic and raw data, forcing researchers to implement custom solutions—an approach ill-suited to large-scale, reproducible investigation. The principal objectives for Code4MeV2 include:
- Providing an open-source alternative to proprietary systems, specifically tailored for the needs of academic and empirical research.
- Enabling fine-grained control over telemetry and code context data.
- Facilitating comprehensive studies on human–AI interaction within software development workflows.
- Lowering the barrier for reproducible research by offering modular extensibility and configurable experimental environments.
2. Architectural Design and Core Features
Code4MeV2 employs a client–server architecture that distinctly separates user interface concerns from heavy computational logic. The components are:
- Client Plugin: A lightweight JetBrains IDE plugin renders inline code completions as "ghost text" and exposes a context-aware chat assistant for interactive disambiguation and extension of generated code.
- Backend Server: Responsible for AI model inference, user authentication, and durable data storage. All computationally intensive operations are offloaded to the server, minimizing impact on IDE performance.
- Modular Analytics Dashboard: Researchers can access an analytics dashboard for post hoc inspection and analysis of captured interaction data.
The architectural diagram (Figure 1 in (Koohestani et al., 4 Oct 2025)) illustrates: (a) backend processing and storage, (b) plugin interaction pathways for code suggestion and telemetry capture, and (c) the analytics dashboard. The conversational feature supplements inline suggestions by enabling code clarification and extension through chat, thus supporting both reactive and proactive code assistant use cases.
3. Modular Data Collection Framework
A core innovation in Code4MeV2 is the modular data collection framework implemented on the client side. The system is hierarchical, allowing for individual telemetry modules—termed "aggregators"—to be independently employed for specific data collection purposes, such as:
- Typing speed
- Time elapsed since the last completion event
- Contextual details (e.g., location in source code, size of code snippets)
- User actions (e.g., copy-paste events, code edit patterns)
Researchers have fine-grained control over which aggregators are active. The architecture supports straightforward augmentation: new modules can be appended without extensive modification of the central codebase. This flexibility is critical for designing experiments that require precise telemetry or adapting to emergent research requirements.
4. Performance Evaluation
Evaluation of Code4MeV2 focused on both code completion latency and conversational module performance. Key observed metrics include:
| Task | Mean Latency (ms) | Standard Deviation (ms) |
|---|---|---|
| Code Completion | 186.31 | ±139.50 |
| Chat Assistant | 8369.78 | — |
End-to-end latency for code completions is competitive with industry standards and significantly below the threshold typically associated with workflow disruption. Although the chat assistant exhibits higher latency, this is attributed to the complexity of conversational inference relative to atomic code suggestion requests. This suggests the platform is well-suited for deployment in research environments where minimizing interruption and ensuring usability are priorities.
5. User and Expert Evaluation
Empirical assessment included both expert researcher review and a targeted user paper (eight participants). All expert participants judged the platform’s modularity, extensibility, and research suitability as strengths. Specific observations included:
- Straightforward reconfiguration for differing experimental needs.
- User interface for module management requires refinement for greater intuitiveness.
- Timing of suggestions generally met expectations, though further improvements in latency were desired.
Daily user feedback corroborated the efficacy of the design but indicated ongoing demand for expedited suggestion times and more advanced features (e.g., an Agent feature). Later iterations addressed the majority of usability concerns, but future releases may further optimize responsiveness and interactivity.
6. Significance for the Academic Community and Open Source Ecosystem
Code4MeV2 is positioned as both a research infrastructure and a collaborative platform. Community adoption and contribution are actively encouraged, with a focus on collective advancement in the following areas:
- Development of new telemetry modules, context providers, and analytics tools.
- Enhancement of the modular framework to enable wider experimental coverage.
- Improving research reproducibility and transparency by sharing data and experiment configurations.
A plausible implication is that widespread deployment and contribution could result in an ecosystem where empirical studies on human–AI collaboration in coding are readily scalable and comparable. The commitment to open source allows tailoring to specific research agendas and facilitates methodological consistency across studies.
7. Context within the Landscape of AI-assisted Development
Within the broader field of AI-assisted code generation and completion, Code4MeV2 differentiates itself by prioritizing transparent research enablement over closed commercial utility. For instance, platforms like GPT-4 (see (Moussiades et al., 2023)) demonstrate notable prowess in code generation, question answering, and debugging, with performance sufficient to suggest impending reorganizations of software development practice. However, those commercial tools do not provide accessible interaction data, impeding empirical research.
By contrast, Code4MeV2’s open, configurable, and data-centric design reframes the paper of human–AI coding workflows. It supports both research and practical integration with established development environments—namely JetBrains IDEs—and achieves performance metrics analogous to leading commercial offerings. This enables experimental work that is otherwise infeasible with proprietary tools.
In summary, Code4MeV2 is an extensible, low-latency code completion and human–AI interaction research platform. Its client–server architecture, hierarchical telemetry system, and commitment to modularity and openness substantively address reproducibility and data transparency challenges. The platform’s adoption holds potential to accelerate empirical inquiry, foster methodological standardization, and ultimately elucidate the evolving role of AI in professional software engineering practice.