CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents

Abstract: A prerequisite for coding agents to perform tasks on large repositories is code localization - the identification of relevant files, classes, and functions to work on. While repository-level code localization has been performed using embedding-based retrieval approaches such as vector search, recent work has focused on developing agents to localize relevant code either as a standalone precursor to or interleaved with performing actual work. Most prior methods on agentic code search equip the agent with complex, specialized tools, such as repository graphs derived from static analysis. In this paper, we demonstrate that, with an effective reinforcement learning recipe, a coding agent equipped with nothing more than a standard Unix terminal can be trained to achieve strong results. Our experiments on three benchmarks (SWE-Bench Verified, Pro, and Lite) reveal that our models consistently achieve superior or competitive performance over 2-18x larger base and post-trained LLMs and sometimes approach performance provided by closed models like Claude Sonnet, even when using specialized scaffolds. Our work particularly focuses on techniques for re-purposing existing coding agent environments for code search, reward design, and RL optimization. We release the resulting model family, CodeScout, along with all our code and data for the community to build upon.

• The paper's main contribution is introducing a language-agnostic RL framework that leverages a simple bash terminal to accurately localize code components.
• The study details a scalable training pipeline using Group Sequence Policy Optimization, achieving competitive F1 scores on multiple benchmarks with far smaller models.
• The research demonstrates that minimal, command-line-based agent scaffolds can simplify engineering complexity while enhancing efficiency and generalization across diverse programming languages.

CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents

Problem Formulation and Motivation

The paper presents CodeScout, a reinforcement learning (RL) paradigm for training code localization agents leveraging only a generic bash terminal interface for software repository navigation and search. Code localization—the identification of relevant files, classes, and functions pertaining to a software issue—is the bottleneck for multi-stage tasks in automated program repair and agentic coding workflows. Previous state-of-the-art methods predominantly utilize complex agent scaffolds augmented with language-specific static analysis, code graphs, and graph-tool navigation, which limit their practical deployment due to implementation overhead and restricted language generality.

The central question addressed is: With suitable RL methodology, can a standard coding agent using minimal toolsets (specifically, a bash terminal without specialized plugins) rival or outperform existing approaches employing static analysis and multiple custom tools?

CodeScout Methodology

The proposed solution comprises (1) scalable RL environment design, (2) a strictly language-agnostic agent scaffold interfaced via terminal commands, (3) fine-grained reward structuring across multiple localization granularities, and (4) an efficient RL training pipeline using Group Sequence Policy Optimization (GSPO).

Figure 1: Overview of CodeScout—given a GitHub issue, an LLM agent navigates the codebase via terminal actions and outputs predicted files, modules, and functions; reward is F1-based at all three granularities.

The RL environment is instantiated for each training instance by (a) cloning the relevant pre-pull-request state of a repository, (b) extracting ground-truth localization triplets (modified files/modules/functions) from issue resolution patches, and (c) cycling agent interactions entirely through bash commands (e.g., rg, grep, find, cat, sed).

The agent's actions are strictly limited—other than bash, the only tool is a structured localization_finish used for answer submission. RL reward is the sum of F1 scores between predicted and ground-truth sets at every granularity; for the largest model, an auxiliary termination reward encourages completion within a step budget to remedy non-submissive behavior.

Agent training utilizes GSPO with asynchronous rollout generation, as implemented in SkyRL. The training process is highly scalable—the architecture does not require code graph preprocessing or language-specific parsing and is amenable to any programming language target, subject to availability of ground-truth labels.

Experimental Results

Extensive evaluation is performed across three challenging benchmarks: SWE-Bench Verified, SWE-Bench Lite, and SWE-Bench Pro. The results highlight three major points:

1. Parameter Efficiency: CodeScout-trained models achieve performance matching or surpassing LLMs 8–18$\times$ larger, even when those are post-trained or augmented with specialized tools.
2. Closed-Model Parity: On certain datasets, CodeScout approaches or exceeds the localization efficacy of closed-source models like Claude Sonnet and GPT-5, particularly at the function granularity.
3. Scaffold Simplicity: The use of a bash-only toolbox grants language-agnosticism and massive engineering simplification while retaining high precision and recall.

Figure 2: CodeScout performance is competitive with or superior to larger SOTA open-source LLMs, and closes the gap with proprietary frontiers on SWE-Bench Verified.

Detailed, granular results indicate that all three CodeScout checkpoints (1.7B, 4B, and 14B) yield absolute F1 gains over equivalent and much larger base models, with consistent behavior across all localization levels. The 4B and 14B checkpoints also outperform stronger base and post-trained models on complex scaffolds (e.g., RepoNavigator-32B and CoSIL-32B), and are competitive even in function-level localization.

Analysis of Agent Behavior and Downstream Utility

Ablation studies confirm that CodeScout's efficacy is robust to choice of RL algorithm, with reward shaping and scaffold minimalism taking precedence over optimizer nuances.

Investigation of the evolving command usage throughout training shows rapid specialization: models start with a broad set of bash utilities but converge to a core subset by late training—principally rg and sed, with rare invocation of extraneous shell commands.

Figure 3: CodeScout-14B initially explores diverse Unix commands, but by convergence almost exclusively uses rg and sed.

Furthermore, equipping downstream issue resolution agents with CodeScout-derived localization context yields higher bug-fix rates and improved efficiency in terms of trajectory length and token usage, compared both to vanilla exploration and to oracle-provided locations.

Practical and Theoretical Implications

Pragmatically, CodeScout demonstrates that RL can efficiently repurpose generic agent infrastructure, substantially reducing engineering burden and accelerating generalization to underrepresented languages or repositories for which static analysis tools are absent. Additionally, the ability to post-train even extremely compact LLMs (e.g., 1.7B) to SOTA performance levels on downstream coding tasks opens new directions for lightweight, robust agent deployments in resource-constrained settings.

Theoretically, results suggest that the principal bottleneck in code localization-via-LLM is not scale or tool complexity, but rather data curation, reward design, and careful RL training. The observed convergence in command usage implies that highly capable policies can be discovered even with minimal actuators, and that much of the engineering effort in prior work can be replaced by a reinforcement learning signal strong enough to drive robust generalization.

Conclusion

CodeScout sets a new state of the art for repository-level code localization using a minimal, domain-agnostic agent scaffold, outperforming considerably larger models and matching closed-source LLMs without specialized toolkits or contextual graphs. Its results indicate that appropriately designed RL recipes, when paired with precise reward and scalable environment construction, suffice to achieve competitive code search in real-world settings. The agent’s convergence to a few core Unix tools highlights both the efficiency and the operational simplicity of this approach. The release of models, code, and data will facilitate further research in RL-based agentic coding and code retrieval.