FastCode Framework: Efficient Code Reasoning

• FastCode is a framework for repository-scale code reasoning that separates lightweight metadata scouting from targeted full code retrieval.
• It employs a semantic-structural graph to capture code relationships, enabling efficient multi-hop debugging and cross-file comprehension.
• Empirical evaluations show FastCode reduces token costs and improves accuracy, outperforming traditional retrieval methods on large-scale benchmarks.

FastCode is a framework for repository-scale code understanding and reasoning, designed to maximize the accuracy of LLM-assisted workflows while sharply reducing the computational and token costs of context construction. The central innovation of FastCode is its separation of repository exploration (termed “scouting”) utilizing lightweight semantic-structural information, from the final consumption stage (retrieving full code units to present to the LLM). This paradigm enables repository-level tasks—such as cross-file comprehension and multi-hop debugging—to be conducted efficiently by tracing only the most relevant parts of a codebase and minimizing unnecessary code ingestion. FastCode operationalizes a cost-aware, structure-driven approach that outperforms state-of-the-art methods in both accuracy and efficiency across several large-scale code reasoning benchmarks (Li et al., 1 Mar 2026).

1. Motivation and Problem Formulation

Repository-scale code reasoning presents fundamentally greater challenges than snippet-level tasks, primarily due to the necessity of integrating fine-grained local detail with global software architecture. LLM-powered systems must frequently traverse multi-hop dependencies, such as class hierarchies or function call chains, distributed across many files. Naïve approaches—such as supplying the entire repository to the model’s context—rapidly exceed token limits and lead to performance decline when irrelevant code fragments are introduced. While standard retrieval-augmented generation (RAG) pipelines mitigate context overflow by chunking code into fragments, they sever vital structural relationships (e.g., imports, inheritance) essential for precise reasoning. Agentic methods that iteratively search and open files can recover some structural connection but at the expense of escalating token and compute costs due to repeated loading of irrelevant files. FastCode seeks to resolve this intrinsic tension: maximizing the semantic relevance and completeness of the context while minimizing its associated computational overhead.

2. Framework Architecture and Component Overview

FastCode decouples repository traversal from code ingestion, implementing three principal components:

• Semantic–Structural Representation: Constructs a hybrid index and a multi-layer dependency graph over the codebase, capturing not only lexical and semantic similarity but also structural code relationships.
• Codebase Context Navigation: Utilizes LLM-driven tool calls (DirectoryTraversal, CodebaseSearch) and graph-guided expansion to identify and prioritize high-value code units solely via metadata.
• Cost-Aware Context Management: Governs exploration termination and snippet selection to maximize reasoning confidence under hard token or line-count constraints.

This pipeline ensures that only the most relevant code fragments are ingested, with ordered context construction guided by both semantic content and repository structure.

3. Structural Scouting Mechanism

FastCode formalizes the code repository as a semantic-structural graph $G = (U, E)$, where $U$ is the set of hierarchical code units (files, classes, functions, documentation), and $E$ encodes key structural relations: imports ($G_{dep}$), inheritance ($G_{inh}$), and call-site links ($G_{call}$). Each unit $u \in U$ is annotated with lightweight metadata—signatures, docstrings, line counts—allowing deferred loading of full implementation bodies. Multi-grained hybrid indexing is utilized, combining a sparse BM25 index over lexical tokens (names, APIs) with a dense embedding index (vector similarity) targeting semantic matches and pseudocode hints.

Structural exploration and retrieval proceed via tool-assisted mechanisms: the DirectoryTraversal tool enumerates file paths, and the CodebaseSearch tool applies regex/keyword scans to report match statistics and code signatures—all without requiring full-text loading. The adaptive scouting workflow is orchestrated in rounds, alternating among query augmentation (rephrasing and keyword expansion using LLMs), retrieval and tool calls, graph expansion to include multi-hop neighbors, and candidate screening based on provenance, signature, and cost metrics, followed by an agentic decision step (retain/discard candidates, terminate/invoke further rounds).

4. Cost-Aware Policy and Optimization Objective

Context acquisition in FastCode is cast as a joint optimization problem:

$$C^* = \underset{C \subseteq U}{\arg\max} \; [ \mathrm{Rel}(C \mid q) - \lambda \cdot \Omega(C) ]$$

Here, $\mathrm{Rel}(C\mid q)$ quantifies the semantic adequacy of candidate context $C$ to resolve the query $q$, $\Omega(C)$ measures its computational overhead (notably, token budget), and $\lambda$ modulates the relevance-cost tradeoff. The agent’s state at round $t$ is $S_t = \{D_q, H_r, L_t, t, \kappa_t \}$ with:

• $D_q$: query complexity score (0–100)
• $H_r$: repository entropy (0.5–2.0)
• $L_t$: cumulative lines of code selected
• $t$: iteration index
• $\kappa_t$: epistemic confidence in sufficiency of current context

A dynamic line budget $B$ is set proportional to $D_q \cdot H_r$. The policy $\pi(S_t)$ selects between “fast path” two-round verification (if $\kappa_0 \geq \tau$) and iterative exploration, monitored by information gain rate,

$$\mathrm{IGR}_t = \frac{\kappa_t - \kappa_{t-1}}{L_t - L_{t-1}}$$

Scouting terminates when confidence reaches threshold ($\kappa_t \geq \tau$), efficiency falls below threshold ($\mathrm{IGR}_t < \epsilon$), or budget is exhausted ($L_t > B$).

5. Context Construction and Ingestion

At the end of scouting, FastCode selects and concatenates full texts of prioritized code units in a one-shot operation. Units are evaluated according to

$$P(u) = w_1 \cdot \mathrm{Rel}(u) + w_2 \cdot I_{tool}(u) + w_3 \cdot \mathrm{Density}(u)$$

where $\mathrm{Rel}(u)$ is relevance (retrieval/graph), $I_{tool}(u)$ signals explicit tool discovery, and $\mathrm{Density}(u)$ prefers finer granularity (favoring functions and smaller code blocks). Highest scoring units are accumulated until the cumulative line budget $B$ is met. The concatenation schema preserves logical code flow: import statements first, then class definitions, followed by function implementations. The full context is then submitted to the LLM for query resolution.

6. Empirical Evaluation

FastCode’s performance was comprehensively evaluated on four diverse benchmarks:

Benchmark	Metric/Tasks	Key FastCode Result
SWE-QA	576 QA pairs / 12 Python repos	Total 43.28, \$0.032/query
LongCodeQA	443 MCQs (32K–1M tokens)	+17.9% accuracy, uniform low cost (\$0.04–\$0.07/query)
LOC-BENCH	274 issue–patch pairs (file localization)	Acc@1 86.13%, \$0.0364—down to \$0.0038 (Qwen3-30B)
GitTaskBench	54 GitHub tasks (exec.-based metrics)	TPR 57.41%, up to –99.86% cost vs. baseline

Baselines included direct LLM prompting, established RAGs (FuncRAG, SlidingRAG, FileBM25RAG), established agentic frameworks (SWEQA-Agent, OpenHands, LocAgent), and leading commercial tools (DeepWiki, CodeWiki, Gemini Code, Claude Code, Cursor). FastCode consistently achieved top-tier reasoning accuracy, with token and computational costs reduced by one to three orders of magnitude. Ablation studies revealed that hybrid retrieval, graph-based extension, and cost-aware management each independently contribute to performance (losses of –0.6%, –0.8%, and –0.9% respectively on removal), but query augmentation and tool scouting are most critical (–4 to –5% impact).

7. Implementation Considerations and Limitations

FastCode relies on standard BM25 libraries for sparse retrieval, embedding-based similarity search (e.g., code-specialized encoders with FAISS), and AST-based static analysis to extract dependency, inheritance, and call relationships. Tool wrappers expose directory traversal and codebase search via metadata without requiring full text. Query augmentation, candidate generation, and selection policies are implemented on a backbone LLM, with agent decision logic encapsulated in Python.

The framework is most effective for queries involving multi-hop dependency tracing or when stringent context budgets are required. In tasks reducible to simple symbol lookup, the overhead of building semantic-structural graphs and running cost-aware agentic policies may offset potential gains. FastCode’s reliance on static AST analysis introduces brittleness for code employing dynamic or reflective features. Transfer to other languages or extremely large codebases can require manual tuning of $\lambda$, threshold, and weight parameters. Incorporation of dynamic analysis or reinforcement learning-based policy optimization is identified as promising future work (Li et al., 1 Mar 2026).