MutaGReP: Execution-Free Repository-Grounded Plan Search for Code-Use (2502.15872v1)

Abstract: When a human requests an LLM to complete a coding task using functionality from a large code repository, how do we provide context from the repo to the LLM? One approach is to add the entire repo to the LLM's context window. However, most tasks involve only fraction of symbols from a repo, longer contexts are detrimental to the LLM's reasoning abilities, and context windows are not unlimited. Alternatively, we could emulate the human ability to navigate a large repo, pick out the right functionality, and form a plan to solve the task. We propose MutaGReP (Mutation-guided Grounded Repository Plan Search), an approach to search for plans that decompose a user request into natural language steps grounded in the codebase. MutaGReP performs neural tree search in plan space, exploring by mutating plans and using a symbol retriever for grounding. On the challenging LongCodeArena benchmark, our plans use less than 5% of the 128K context window for GPT-4o but rival the coding performance of GPT-4o with a context window filled with the repo. Plans produced by MutaGReP allow Qwen 2.5 Coder 32B and 72B to match the performance of GPT-4o with full repo context and enable progress on the hardest LongCodeArena tasks. Project page: zaidkhan.me/MutaGReP

MutaGReP: Execution-Free Repository-Grounded Plan Search for Code-Use

The paper introduces MutaGReP, a sophisticated framework designed for navigating extensive code repositories to enhance code-use functionality without execution. This execution-free, repository-grounded planning approach leverages tree search methodologies to create detailed plans that facilitate LLM guided code generation, particularly in complex codebases where traditional methods might struggle due to context window limitations.

Key Contributions

1. Repository-Grounded Plan Search: MutaGReP formulates the plan search as a neural-guided tree search problem. It starts from a user's coding request and breaks it down into executable steps by exploring the plan space through mutations, all the while ensuring each step is grounded in symbols available within the codebase. This method operates without the need for code execution, thus making it resource-efficient.
2. Neural Tree Search Efficiency: The framework employs a symbolic retrieval system that adds precision to the search by narrowing down the available symbols within the repository that might be pertinent to the required tasks. Through this, MutaGReP successfully condenses the context, reducing typical resource usage to less than 5% of the standard 128K context window of models like GPT-4o while maintaining a level of performance comparable to full-repository context scenarios.
3. Performance Metrics: Utilizing the LongCodeArena benchmark, the plans designed by MutaGReP rival the coding performance of broader context infusion models. The proposed plans significantly enhance the capability of smaller LLMs, such as Qwen 2.5 Coder 32B and 72B, enabling them to perform comparably to GPT-4o filled with complete context.

Methodology

The system’s design comprises several integral components:

• The successor function for generating plan mutations, which either add new steps in a monotonic fashion or allow for broader transformations.
• A symbol retriever that grounds codes with repository-specific symbols, allowing the search process to remain grounded and feasible.
• Tree traversal algorithms—primarily best-first search—are used to effectively navigate through the alternatives.
• The task proficiency is further enhanced with a plan ranker that uses symbol diversity and other criteria to prioritize search paths.

This methodology ensures a scalable and efficient search process that facilitates complex reasoning tasks preferred in computational tasks.

Implications and Future Directions

The repository-grounded approach proposed by MutaGReP introduces a transformative tool in the AI-driven software engineering landscape, breaking away from traditional exhaustive methods by advocating for context-driven, efficient execution-free procedures. The ability to create actionable, detailed plans with limited context has significant implications not only in improving the efficiency of AI systems but also in fostering the development of more intelligent tools that can autonomously navigate complex code architectures.

In terms of future developments, the authors hint at potential uses in enhancing the robustness of smaller, open-source models through integration with the MutaGReP framework. There's also a strong indication that this technique could be refined further by incorporating more sophisticated search algorithms, possibly drawing elements from strategies like Monte Carlo Tree Search (MCTS), to elevate performance and applicability further.

In conclusion, the presented work on MutaGReP signifies a substantial step forward in AI code generation, specifically in its application to real-world, extensive coding tasks where efficient, context-limited operations are crucial. This could lead to more pronounced developments across various applications, potentially shaping the future paradigms of code use and generation in software development.